678
submitted 11 months ago by FauxPseudo@lemmy.world to c/skeptic@lemmy.world
you are viewing a single comment's thread
view the rest of the comments
[-] cynar@lemmy.world 16 points 11 months ago

For those confused

2.25+2.25=4.5 rounds to 2+2=5

2.5+2.5=5 truncates to 2+2=5

Both can crop up in programming, depending on the situation.

[-] echodot@feddit.uk 3 points 11 months ago

2.25 + 2.25 = 4.5

If you add two floats together then the output is a float, if you add an int and a float together the output is a float. Computers will always perform the calculation as is, unless you explicitly tell them to perform a rounding operation.

[-] cynar@lemmy.world 6 points 11 months ago

However, if you stuff them into an int at the last minute, you can get that effect.

Under the hood, it's floats. On the output, it's ints.

It's obvious and silly with small examples. The problem can creep in when you are using larger libraries or frameworks.

[-] afraid_of_zombies@lemmy.world 0 points 11 months ago

A few months back I had a floating point that had a single 1 like 16 digits past the decimal place and I couldn't get rid of it.

this post was submitted on 27 Nov 2023
678 points (96.2% liked)

Skeptic

1281 readers
1 users here now

A community for Scientific Skepticism:

Scientific skepticism or rational skepticism, sometimes referred to as skeptical inquiry, is a position in which one questions the veracity of claims lacking empirical evidence.

Do not confuse this with General Skepticism, Philosophical Skepticism, or Denialism.

Things we like:

Things we don't like:

Other communities of interest:

"A wise man proportions his belief to the evidence." -David Hume

founded 1 year ago
MODERATORS