⋯ full post (183 more characters) ⋯ show less
Sort of. I think there is an early project on nostr attempting to handle this. The issue is that you can't really break up an LLM across distributed compute yet. So unless you have a big enough system to run a useful model at usable speeds were sol.
I currently have a big enough system to share, but not enough to handle the concurrence and model variation that a community might need. Id also have to charge 6x what the most expensive tokens are to serve it.
Sort of. I think there is an early project on nostr attempting to handle this. The issue is that you can't really break up an LLM across distributed compute yet. So unless you have a big enough system to run a useful model at usable speeds were sol.
I currently have a big enoug