submitted 2 weeks ago by Lugh@futurology.today to c/futurology@futurology.today

25 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[-] copygirl 3 points 2 weeks ago

Wouldn't running on a CPU (while possible) make it less energy efficient, though?

[-] pennomi@lemmy.world 3 points 2 weeks ago

It depends. A lot of LLMs are memory-constrained. If you’re constantly thrashing the GPU memory it can be both slower and less efficient.

this post was submitted on 01 Dec 2024

49 points (100.0% liked)

Futurology

1851 readers

47 users here now

founded 1 year ago

MODERATORS