381

AI coders think they’re 20% faster — but they’re actually 19% slower (pivot-to-ai.com)

submitted 3 months ago by dgerard@awful.systems to c/techtakes@awful.systems

89 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[-] mspencer712@programming.dev 6 points 3 months ago

The N=16 keeps getting buried. Deliberate?

[-] Feyd@programming.dev 31 points 3 months ago

You're acting like this is a gotcha when it's actually probably the most rigorous study of AI tool productivity change to date.

[-] blakestacey@awful.systems 21 points 3 months ago

Paragraph 2:

METR funded 16 experienced open-source developers with “moderate AI experience” to do what they do.

[-] HedyL@awful.systems 27 points 3 months ago

... and just a few paragraphs further down:

The number of people tested in the study was n=16. That’s a small number. But it’s a lot better than the usual AI coding promotion, where n=1 ’cos it’s just one guy saying “I’m so much faster now, trust me bro. No, I didn’t measure it.”

I wouldn't call that "burying information".

[-] swlabr@awful.systems 15 points 3 months ago

. Debate me bro? (jk)

[-] dgerard@awful.systems 17 points 3 months ago

this user has been removed for commenting without reading the article

being from programming dot dev is just the turd on top

[-] froztbyte@awful.systems 12 points 3 months ago

programming.dev: statistical sampling excellency (worst edition)

[-] self@awful.systems 11 points 3 months ago

programmers learned what N means in statistics and immediately realized that “this N is too small” is a cool shortcut to sounding smart without reading the study, its goals, or its conclusions. and you can use it every time N is smaller than the human population on earth!

[-] blakestacey@awful.systems 15 points 3 months ago

This N is too small: ~N~

[-] OpenStars@piefed.social 6 points 3 months ago

Skill issue - this N is even smaller:

spoiler

[-] Tar_alcaran@sh.itjust.works 4 points 3 months ago

The colon-space-subscript bothers me Immensely

this post was submitted on 11 Jul 2025

381 points (100.0% liked)

TechTakes

2277 readers

86 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 2 years ago

MODERATORS

dgerard@awful.systems