76
I'm using AI and I feel terrible now
(lemmy.world)
Privacy has become a very important issue in modern society, with companies and governments constantly abusing their power, more and more people are waking up to the importance of digital privacy.
In this community everyone is welcome to post links and discuss topics related to privacy.
much thanks to @gary_host_laptop for the logo design :)
host your own AI using: https://ollama.com/
this paired with "AnythingLLM" is pretty powerful, and you dont have to worry about data being sent anywhere
I actually got it to run reasonably on a mini PC in a docker container, but my setup also lets me use CPU power as a psuedo GPU.
Llamacpp, Koboldcpp, and TabbyAPI are also popular local backends for local AI. SillyTavern or RisuAi are good frontends for a chat/RP style experience. Or LM Studio for a simple, all in one solution.
I have been suggested to self host in the past. But as I'm concerned for ecology, and let's be a honest, a bit of my money too, I've thought that self hosting, meaning having the PC always on, was not a good thing. So I left the dilemma in this state, not knowing how "bad" this would actually be in terms of electric consumption, and if it was worth the trade with privacy or not.
That's pretty dependant on hardware. If you host small stuff , like a pihole or something, that can be done relatively cheaply, by using a micro-PC or a Raspberry Pi. Some services don't need to be always-on either, you only need stuff all the time if it's mission-critical, otherwise you can turn stuff on and off as necessary, for power-saving purposes. Self-hosting doesn't necessitate a huge rack and switches, or even your big gaming rig, my favorite thing to do with old laptops is throw Debian on it and find something I'd like to self-hosting from this list.