18
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 30 Dec 2024
18 points (100.0% liked)
TechTakes
1563 readers
153 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 2 years ago
MODERATORS
Nobody outside the company has been able to confirm whether the impressive benchmark performance of OpenAI's o3 model represents a significant leap in actual utility or just a significant gap in the value of those benchmarks. However, they have released information showing that the most ostensibly-powerful model costs orders of magnitude more. The lede is in that first graph, which shows that for whatever performance gain o3 costs over ~$10 per request with the headline-grabbing version costing ~$1500 per request.
I hope they've been able to identify a market willing to pay out the ass for performance that, even if it somehow isn't over hyped, is roughly equivalent to an average college graduate.
I'm wondering about the benchmark too. It's way above my level to figure out how it can be gamed. But, buried in the article:
The most expensive o3 version achieved 87.5%