438
you are viewing a single comment's thread
view the rest of the comments
[-] merc@sh.itjust.works 3 points 1 year ago

The difference is that when you input a specific, precise floating point number, the number that's stored isn't what you entered.

When you enter integers and store them in ints, as long as the number is small enough, what's stored is exactly what you entered.

If you tell your program that the radius of the circle is 0.2 units exactly, it says OK and stores 0.200000000000000011102230246251565404236316680908203125.

Of course everybody knows that there's a limit to how many digits get stored. If you tried to store Pi, there's obviously some point where it would have to be cut off. But, in life we're used to cutting things off at a certain power of 10. When we say Pi is 3.14 the numbers after the 4 are all zero. If we choose 3.14159 instead, it's the numbers after the 9 that are zero. If we represent these as fractions one is 314/100 the other is 314159/100000. The denominator is always a power of 10.

Since computers are base 2, their denominator is always a power of 2, so there's a mismatch between the rounded-off numbers we use and the rounded-off numbers the computer uses.

this post was submitted on 17 Sep 2023
438 points (100.0% liked)

Programmer Humor

19817 readers
570 users here now

Welcome to Programmer Humor!

This is a place where you can post jokes, memes, humor, etc. related to programming!

For sharing awful code theres also Programming Horror.

Rules

founded 2 years ago
MODERATORS