I’m always shocked by how LLM illiterate professionals or even leading “science influencers” are.
…But I know why.
It’s on purpose
Literally everything about the most common (Big Tech) interfaces obfuscates everything about them, to the maximum degree, to present them as “sycophantic oracles” and maximize engagement. If that’s all one is exposed to, there is literally no way to get a cold ass bucket of water in the face by peeking under the hood.
It’s all very… anti science.
On a semi related note, see this absolutely unhinged but not wrong rant:
The Conspiracy Against High Temperature Sampling
https://gist.github.com/Hellisotherpeople/71ba712f9f899adcb08b94bce20d5397
It is related. Anyone, even Hank Green, will rely on AI more if they never get a chance to poke it and peel it apart.
Like, if I ware in charge of a university, I would mandate a course where everyone gets a untuned base model in raw completion mode, and gleefully watch the slow realization. “Wait… all “AI” is just this?”