Skip to Main Content

Java Programming

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Java double floating point precision vs MS Excel

807588May 8 2009 — edited May 10 2009
Hello,

I know that in floating point computations there are precission issues related to decimal-to-binary conversions and also related to IEEE 754 specification restrictions (basically the number of bits used for fp numbers) shared with a lot of languages and computer systems and that Java language makes simply more evident because it shows doubles with full precision by default when you print them to screen.

A good sample is this calculation: 100.39 - 100.35 which doesn't gives the exact result that a human (thinking in base 10 arithmetic) could expect.

This substraction is a problem that hapens also in MS Excel ... but you can see it only if you shows the result with Scientific full precision or if you do a conditional formula that compares the cell with the substaction result with 0.04

But now I've at hand another calculation that shows a little (decimal) inaccuracy in Java and shows an EXACT result in Excel, and I dont understand why. This is the sample:

double a = 0.21;
double b = 28.5;
double c = a * b;

The result, converted to decimal, shows in screen like 5.984999999999999 but in MS Excel 2000 this time I get the exact result even showing it at full scientific precision and even if I do a conditional formula that compares the result with 5.985.

Both Java and Excel use the IEEE 754 floating point format with double precission, so I could expect the same roundoff problems in both.

Some hint?

Thanks.
Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Jun 7 2009
Added on May 8 2009
20 comments
3,856 views