566
you are viewing a single comment's thread
view the rest of the comments
[-] kamen@lemmy.world 18 points 2 days ago

Local AI it is then. Not that I'm using all that much now anyway...

[-] melfie@lemmy.zip 12 points 2 days ago* (last edited 2 days ago)

Where I work, Chinese models are banned due to legal concerns. Not just in production, on any company-owned machine. That basically eliminates all the decent open weight models. I’m imagining this type of policy will be more widespread. I suppose it’s because of the potential for legal woes if systems and people are dependent on these models and then federal or state laws impose harsh penalties with little time to react.

[-] kamen@lemmy.world 8 points 2 days ago

I meant for private use.

As for work, we do use AI quite a lot and I don't have a say over what's available.

[-] eleitl@lemmy.zip 2 points 2 days ago

Copilot? Copilot.

[-] favoredponcho@lemmy.zip 8 points 2 days ago* (last edited 2 days ago)

To use one equivalent to Claude Opus, you need like 800GB GPU memory. The chips that will get you there run $20-30k a pop, and you’d need 4 or more of them.

[-] boonhet@sopuli.xyz 9 points 2 days ago

That's the biggest baddest model out there. There are models that get you 90% of the way there with a significantly smaller parameter count and thanks to MoE offloading you don't need the entire model active at once.

A 5090 and a beefy CPU and tons of RAM won't be cheap or even affordable to most, but you could run very big models and have a beefy PC for other activities. But even a 16 GB card could do plenty.

[-] kamen@lemmy.world 4 points 2 days ago

Yeah, I'm aware. I have realistic expectations and I'm looking into running something simpler and less demanding.

this post was submitted on 02 Jun 2026
566 points (100.0% liked)

Technology

85138 readers
3675 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS