134

submitted 20 hours ago by SuspiciousCarrot78@aussie.zone to c/selfhosted@lemmy.world

165 comments fedilink hide all child comments

Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

you are viewing a single comment's thread
view the rest of the comments

[-] e0qdk@reddthat.com 3 points 12 hours ago

If you just pulled the default version of qwen3.5 from ollama's repo you downloaded a mediocre one that only uses ~6GB.

Check ollama show qwen3.5 and see if you get something like this in the result:

  Model
    architecture        qwen35    
    parameters          9.7B      
    context length      262144    
    embedding length    4096      
    quantization        Q4_K_M

This is the default version I got when I first tried using ollama without any experience. It worked, but it's a heavily quantized, lower parameter version of the model -- i.e. it's pretty dumb -- compared to what you can actually run on your hardware.

[-] Steve@startrek.website 2 points 31 minutes ago

I will check it later. I loaded whichever one cluade suggested lol

this post was submitted on 24 Jun 2026

134 points (100.0% liked)

Selfhosted

60093 readers

808 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.
Promotion posts require your active participation in selfhosting or related communities, or the post will be removed. No more than 10% of your posts or comments may be self-promotional, or your post will be removed. F/LOSS Exception: If your post is about a project that is completely open source & can be self-hosted in full without payment, your post is exempt from this rule as long as you continue to engage in comments.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world