494
my gleeby deeby ass
(lemmy.blahaj.zone)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 06 Jan 2025
494 points (100.0% liked)
196
16840 readers
1743 users here now
Be sure to follow the rule before you head out.
Rule: You must post before you leave.
founded 2 years ago
MODERATORS
I'm not OP, but my guess is they're referring to the Intel math bug that some i5's had. I'm struggling to track it down, but it's basically an issue with doing long division where the floating point math would produce a very wrong result.
You can see more here at least for the bug/issue that existed in the 90's here
I’m not actually, just that a binary integer that overflows at 60 couldn’t exist, hence the 5.907 whatever bit length
oh, that's actually clever. And I'm saying that as a software engineer. I missed that possibility :)
I should have phrased it differently, like “Ah yes, the well known 5.9068905956 bit integer.” But thanks