Photograph: Fredrik Varfjell/NTB/AFP/Getty Images
Guide to Self Hosting LLMs with Ollama.
- Download and run Ollama
- Open a terminal, type
ollama run llama3.2
+1 for the main risk to my service reliability being me getting distracted by some other shiny thing and getting behind on maintenance.
The Debian thong made me laugh. Who is buying this? For themselves, their partners? I'm imagining Christmas morning when I'm trying to explain the value of this gift you've just opened.
Infrastructure:
- Proxmox VE - everything's virtualised on Debian, mostly in docker inside LXC's for neat backup/restore and moving between nodes
- NGINX Proxy Manager - in front of most of my homelab services so they have https certificates
- Tailscale - access everything, everywhere, including on phone, securely
- Uptime Kuma - monitoring, with ntfy notifications
- apt cacher NG - unnecessary caching of apt updates
Apps:
- AudioBookShelf - Audio books (download them from here into BookPlayer (iOS) to listen in the car)
- Kavita - eBooks.
- BOINC - curing childhood cancer
- Forgejo - git repository. I do a bit of dev, but everyday self-hosters should probably still learn git to manage their configs
- Jellyfin - TV/Movies
- Syncthing - I replaced dropbox with Syncthing and FileBrowser, and Evernote with Syncthing and SilverBullet
Currently in testing on the dev server:
- neko - virtualised browser. Been experimenting with this in a container with a VPN for really simple secure browsing - ie launch it, do your online banking and then destroy the container.
- Dashy - I go through periods of wanting a pretty home page with all my services, set it all up, then fail to actually use it and eventually delete it, then hear about another cool one...
- Sharry - securish file sharing. I don't love just emailing my accounts off to the accountant.
- LimeSurvey - survey software (like Survey Monkey) - just something I'm testing for work
- Omada controller - I've got a TP-Link switch and WAP that don't really need centrally controlled, but you know, can be.
- A couple of development environment LXCs I use VS Code in
I still have not landed on a music system. I've put some of my library on Jellyfin, and tried a couple of apps with, but haven't hit on a good combination yet. [edit:formatting}
I read somewhere that GoPros and other action cameras are one of the least used purchases, so I figured "that should mean there's plenty on eBay". So grabbed up second hand bargain, played around with it for a couple of weeks, bought some extra batteries and other accessories, and since then it's sat in the cupboard except for a single occasion.
Turns out you don't need an action cam if you're not getting any action.
Me too. I've been carrying it around in my head as "the time we listened to scientists, and almost everyone worked together on some short term pain for worldwide long term gain". I was even hoping we might do something like that again.
Tailscale is a very cool way of seamlessly creating a private network spread out geographically. Devices sign into the Tailnet from anywhere. It's very big in the selfhosted community (it has a generous free tier). For example my home servers are signed in, so I'm able to stream from my media server to my phone over my private Tailnet tunneled through the internet. I also have an offsite backup location with another server connected to the Tailnet for accepting automated backups.
The underlying technology is Wireguard. It is very smart about figuring out the most effective route - If I'm on my laptop in my home wifi, trafffic from my servers is direct, if I'm away somewhere, it's piped though the net securely. What Tailscale adds is ease of setup and native apps for each device.
The privacy angle is that I'm able to get rid of all the cloud services I used to rely on. For example I don't want my CCTV system connected to a cloud provider, but with Tailnet I can connect to my cameras over the internet without having to expose the system to a data mining corporation.
Build anything small into a container on your laptop, push it to DockerHub or the Github package registry then host it on fly.io for free.