835
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 17 Aug 2026
835 points (100.0% liked)
Technology
87280 readers
2920 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
i wish we could focus on making small LLMs better.
some companies are doing this. some definitely aren't
I'm keeping an eye on Hailo for this purpose. I think the future is small embedded AI systems, not massive data centers.
Woth thé Qwen 3.6 3.8 something nice happened most US company LLM got beat by miles. NVIDIA nemotron 27b isn't as good as qwen two version prior than the one just released. US are beaten on Local LLMs already.
if you start excluding the 1000+ B param models ..
using smaller models, would initially ease hardware demand by 60% .
OFC you cant.. and probably shouldnt, ignore and disrespect SOTA flagship models
Even "big" open source models like DSV4 and Ling/Ring are very efficient. They're big, but (seemingly) sparser than US models, so they're cheap.
They run surprisingly well with hybrid CPU+GPU inference on desktops. And thats not even getting into the efficient attention mechanisms.
I can run DSV4 Flash, barely quantized, with ~1M context on my Ryzen desktop at ~11 tokens/s. If you told me that two years ago, I would not have believed you.