132
Is AI Facing a Trough of Disillusionment?
(spectrum.ieee.org)
This is a most excellent place for technology news and articles.
but LLMs do represent a significant technological leap forward. i also share the skepticism that we haven’t “cracked AGI” and that a lot of these products are dumb. i think another comment made a better analogy to the dotcom bubble.
ETA: i’ve been working in ML engineering since 2019, so i can sometimes forget most people didn’t even hear about this hype train until ChatGPT, but i assure you inference hardware and dumb products were picking up steam even then (Tesla FSD being a classic example).
We definitely haven't cracked AGI, that's without a doubt.
But yeah, LLMs are big (I'd say really Transformers were the breakthrough). My point though was that Deep Learning is the underlying technology driving all of this and we certainly haven't run out of ideas in that space even if LLMs may be hitting a dead end.
I feel like LLMs didn't hit a deadend so much as people started trying to use them in completely inappropriate applications.
I think it's a mixture of that and the fact that when OpenAI saw that throwing more data drastically improved the models, they thought they would continue to see jumps like that.
However, we now know that bad benchmarks were misleading how steep the improvements were, and much like with autonomous vehicles, solving 90% of the problem is still a farcry away from 100%.