Floating Point
Everybody knows 0.1 + 0.2 does not give 0.3, and nearly everybody blames the addition. Add the two stored values exactly, in whole numbers, with nothing rounded at all, and you land past 0.3 anyway. The addition was the second mistake.
New to how computers store fractions? Start here
A computer stores a fraction the way a scientist writes a big number: a handful of digits, and separately how far to shift the point. That is what floating point means -- the point floats rather than sitting in a fixed column. The digits are called the significand, the shift is the exponent, and both are in binary.
Binary is where the trouble starts. One tenth in binary is a repeating fraction, exactly as one third is in decimal, so it cannot be written down in any finite number of digits. The computer keeps the closest value it can and carries on. What is stored for 0.1 is not 0.1, and it never was.
So there are two separate places an answer can go wrong: what was stored in the first place, and what happened when values were combined. This page separates them, and the surprising half is that the first one has usually already happened before any arithmetic runs.
Numbers in a fixed-size box
A number on paper can be as long as it needs to be. A number in a computer gets a fixed number of digits and no more, so there is always a largest one it can hold and always a smallest difference it can tell apart.
Almost everything surprising about computer arithmetic follows from that. What happens when you count past the largest value depends on which kind of number it is: a fixed-width integer wraps round to the smallest, and a floating-point number goes to infinity instead. A value too fine for the gap is not rejected, it is rounded to the nearest one that fits, and then the rounding is added to the next rounding. These are not faults in the machines here. They are the edges of the box, and the machines are what people built to live inside it.
The machine for this idea on its own is Two's Complement, if you would rather press it than read about it.
A number, what the machine kept of it, and what that costs
1 What the machine stored when you typed it
Type any decimal. Nothing here is rounded for display: a double is a whole number times a power of two, so its decimal expansion always ends, and the page writes out every digit of it.
- you typed
- 0.1
- it holds
- 0.1000000000000000055511151231257827021181583404541015625
- which is not what you typed
Sixty-four bits: one of sign, eleven of exponent, fifty-two of fraction. With the bias taken off, the exponent field reads -4, and that is the power of two multiplying the leading 1.fraction. Counted as a whole number instead, the significand is 7205759403792794 and its scale is 2^-56. That pair multiplies back to the value above, exactly.
2 Its neighbours, and the gap at that size
There is no number between a double and the next one. The set is finite, so it can be counted rather than described.
- the double below
- 0.09999999999999999167332731531132594682276248931884765625
- the double above
- 0.10000000000000001942890293094023945741355419158935546875
- the gap here
- 0.00000000000000001387778780781445675529539585113525390625
Doubles between 1 and 2: 4,503,599,627,370,496. Doubles between 0.1 and 0.2: 4,503,599,627,370,496. the same count, in an interval a tenth of the width. Not because a tenth to a fifth is one power of two, which it is not: 0.1 sits in one and 0.2 in the next. It is because the stored 0.2 is exactly twice the stored 0.1, and doubling a value adds one to the exponent field, which moves the bit pattern by exactly two to the fifty-second. Any two NORMAL values a factor of two apart have that many representable numbers between them. Below the smallest normal value the exponent has nowhere left to go, and the subnormals that fill the gap to zero are spaced evenly instead: from the smallest positive number to twice it is one step, not two to the fifty-second.
3 Two roundings, and which one did the damage
The sum below is computed in whole numbers from the two stored values, so it is what the machine would return if it had unlimited room.
- first holds
- 0.1000000000000000055511151231257827021181583404541015625
- second holds
- 0.200000000000000011102230246251565404236316680908203125
- their exact sum
- 0.3000000000000000166533453693773481063544750213623046875
- what you get
- 0.3000000000000000444089209850062616169452667236328125
- which holds
- 0.299999999999999988897769753748434595763683319091796875
- and they are
- not equal
the addition rounded once more, by 0.0000000000000000277555756156289135105907917022705078125
4 Where the counting stops
Whole numbers are exact until the gap between neighbours grows past one. Slide up to that edge and one step over it.
- the number
- 9,007,199,254,740,992
- one more
- 9,007,199,254,740,992
- the gap here
- 2
these are the same number: 2^53 + 1 is not representable
Why the addition is not the culprit
The usual telling stops one step early. It says 0.1 is not exactly representable, which is true, and then leaves you thinking the addition went wrong.
Watch what actually happens. Reading the text 0.1 rounds it to the nearest double, which is a little above a tenth. Reading 0.2 rounds it to a little above a fifth. Those two stored values are now fixed. Add them the way a mathematician would, in whole numbers with nothing discarded, which is what the third readout above does, and the answer is already larger than three tenths. That is before the machine has added anything.
Then the machine adds, and it does the one thing the standard promises: it returns the representable value nearest the true sum of its operands. That is called correct rounding, and it is not the same as exact. It rounds once, in the direction the arithmetic demands, and lands on a double further from three tenths than the true sum was. Two roundings, and the second is the small one.
So the machine is not sloppy and the addition is not broken. The answer is wrong because the question was asked in a notation the machine does not have. That is the same shape as a scratch that is not damage until it reaches a decoder, and the opposite shape to a check that is exact right up to a boundary and useless one step past it.
Why the gaps are where they are
A double is a whole number times a power of two. Between any power of two and the next one, the exponent is fixed and only the fifty-two fraction bits vary, so every such interval holds the same count: two to the fifty-second, whatever its width. That is why one and two are as finely divided as a tenth and a fifth, and why the absolute gap grows as the numbers do until, at two to the fifty-third exactly, it becomes two and consecutive whole numbers stop being distinguishable: that is the first integer whose successor has no representation of its own.
These ran in this browser when the page loaded. Each claim, whether it held, and the number behind it.
| claim | held | measured |
|---|---|---|
| 0.1 plus 0.2 is exactly 0.3000000000000000166533453693773481063544750213623046875, and what a double can hold is 0.3000000000000000444089209850062616169452667236328125 | yes | both written out in full with BigInt rather than rounded; the answer is not representable and neither were the two addends |
| all 12 expansions parse back to the identical double, including the smallest subnormal and the largest finite | yes | a rounded expansion cannot do this; 12 of 12 round-tripped |
| over 202 sums and differences spanning 2^-50 to 2^50, the exact arithmetic agrees with the machine every time | yes | 202 of 202, every pair close enough in magnitude that the true answer is representable; an alignment that shifted the wrong way would disagree at the first unequal exponent |
| add 1.25 times 2^-60 to 1.5 and the stored answer is 1.5 again, though the true sum has 64 digits and is larger | yes | sixty powers of two apart is past the fifty-odd this format can hold at once, so the smaller addend falls off the bottom entirely rather than being rounded |
| stepping 500 doubles up from 1.0 and back down lands on the same value every time | yes | adjacent doubles are adjacent integers when the bit pattern is read as unsigned, so the step is an increment |
| there are 4503599627370496 doubles between 1 and 2, and exactly the same number between 1024 and 2048 | yes | every octave holds 2^52 of them, which is why the gap between neighbours grows with the value |
What is real here, and what is not
There is no simulation on this page, which is unusual here
Every other machine on this site models something and says where the model stops. This one has nothing to model. A JavaScript Number is a double: ECMA-262 defines the Number value as a "double-precision 64-bit binary format IEEE 754-2019 value", so the numbers above are the browser's own, and the bits are read straight out of them with a DataView. The arithmetic the page shows you is the arithmetic it is running.
The exact decimals are exact, and that is a fact about the format
A double is an integer significand times a power of two, so for a negative exponent its value is that integer divided by a power of two, which always terminates in decimal. The page computes those digits with BigInt: multiply by five to the same power and place the point. Nothing here uses toFixed or toPrecision, because both round, and a rounded digit in a readout about rounding would be the one thing this page must not do.
The exact sum is computed in whole numbers, not in floats
Adding the two stored values with the machine's own addition would round, which is the thing being measured. So the sum is done by shifting both significands to a common exponent and adding the integers, in BigInt. That result has no rounding in it at all, and the difference between it and what the machine returns is the rounding, isolated.
The counts are counted, not derived from a formula
The claim that an interval a tenth of the width holds the same number of doubles would be worth nothing as an assertion. Adjacent doubles are adjacent integers when their bit patterns are read as unsigned, for finite positive values, so the count between two of them is a subtraction on those patterns. The page does that subtraction rather than printing two to the fifty-second and hoping.
This is binary64 only, and only the default rounding mode
The standard defines binary32, binary128, decimal formats, and five rounding modes. A browser gives you one of each: binary64, rounding to nearest with ties to even. Everything above is true of that one and is not claimed of the others. Subnormals are handled in the decoder here but there is no control that reaches them.
Nothing on this page dates from 1985
The date is the standard's approval. The format was in silicon before that, in the Intel 8087 of 1980, which implemented a draft; the committee ran from 1977 and the proposal it settled on came from Kahan, Coonen and Stone in 1978. No hardware is simulated here and none of the arithmetic is period. What is from 1985 is the agreement that this is what a number means.
Sources
- Standard ECMA-262, 15th edition, June 2024 — the specification that makes this page possible, and the reason it needs no model. It defines a Number value as a “double-precision 64-bit binary format IEEE 754-2019 value”, which is quoted here because the archived copy can be checked for it. The edge in stage four is stated in the same document, under
Number.MAX_SAFE_INTEGER, and is not quoted: the source gate stores 400,000 characters of a cited document and that section of a 500-page specification falls past the cut, so a quotation from it could not be verified against anything on disk. The page demonstrates it instead, which is the stronger move here anyway. Free to download, and it normatively references the standard below, which is not. - IEEE 754-2019, IEEE Standard for Floating-Point Arithmetic — the document everything here rests on. Not read. It is behind the IEEE, and the format, the bias and the rounding rule are taken from ECMA-262 above and from Goldberg and Kahan below, all three of which are free and agree.
- David Goldberg, “What Every Computer Scientist Should Know About Floating-Point Arithmetic”, ACM Computing Surveys 23(1), March 1991, in full, for rounding error, units in the last place, and why the standard chose what it chose.
- W. Kahan, Lecture Notes on the Status of IEEE Standard 754 for Binary Floating-Point Arithmetic, Berkeley. Kahan designed the thing, and these notes are him saying what it was for and what people get wrong about it.
- IEEE Milestone: IEEE Standard 754, 1985, for the history in the last ledger entry: the subcommittee from 1977, the Kahan-Coonen-Stone proposal of 1978, and eight years to approval.