216
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 03 Jun 2026
216 points (100.0% liked)
Programming
27148 readers
758 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 3 years ago
MODERATORS
I run Qwen 3.6 27B at home. For “free”. It is extremely useful.
My point being that I’m not going to be priced out of using it
Don't worry, they want to replace your hardware with a "cloud based computing solution" as well.
When did that absurdity come back? I thought we killed the cloud computer nonsense a decade ago.
Well you see... subscriptions.
Well you aren't a brain dead business man then.
...yet
What hardware that needs? My issue with running local models was that it's too much of a resource hog to be able to do gamedev on the same machine, and any sensible model needs pretty expensive hardware to just get a server for it. Especially with current prices.
Geforce 3090 with 24TB should be able to run a "Q5 version" of it. Maybe get a second older computer, or maybe you can run two cards in one PC.
64GB unified memory. I run it (and a lot more) on a dgx spark, but a Mac mini would suffice also.
You could prob run 4-bit version on a RTX card with 32g. Maybe even 24g. Like a 5090 or 4090 or such.
So much info out there.
Mac Minis top out at 48GB and are 1.8k when configured like that. It's going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.
Running local isn't free, but at least it's just a single upfront payment.
The M4 Pro Mac Mini caps out at 64GB RAM. Whether or not Apple can sell you that SKU right now is a different question with the ongoing DRAM shortage.
That (64GB) doesn't appear on the site at the moment.
most people are going to destroy their home servers running these workloads
Destroy as in the fan bearings are going to wear out quicker?