Rounding behavior JDK 8 vs. JDK 7

Karl EilebrechtDec 11 2015 — edited Dec 13 2015

UPDATE / RESOLVED

In the meantime I have found a good explanation given by the guys at Open JDK: https://bugs.openjdk.java.net/browse/JDK-8029896

The problem is indeed related to IEEE 754 number representation. Older JDKs like JDK 7 incorrectly implemented rounding double value representations close to ties (like 0.005).

All the <input> cases I listed earlier can be explained by adding System.outprintln((new BigDecimal(<input>)).toString()).

As you can see none of them can be represented exactly.

Here are some examples:

input	closest representation	JDK7 round HALF_EVEN	JDK8 round HALF_EVEN
0.005	0.005000000000000000104083408558608425664715468883514404296875	0.00	0.01
0.015	0.01499999999999999944488848768742172978818416595458984375	0.02	0.01
0.025	0.025000000000000001387778780781445675529539585113525390625	0.02	0.03
0.065	0.065000000000000002220446049250313080847263336181640625	0.06	0.07
0.075	0.07499999999999999722444243843710864894092082977294921875	0.08	0.07
0.085	0.08500000000000000610622663543836097232997417449951171875	0.08	0.09
0.155	0.1549999999999999988897769753748434595763683319091796875	0.16	0.15
0.165	0.1650000000000000077715611723760957829654216766357421875	0.16	0.17
0.175	0.174999999999999988897769753748434595763683319091796875	0.18	0.17

If you now apply rounding with RoundingMode.HALF_EVEN it turns out that JDK7-behavior is wrong while JDK8 behavior is correct. This is the problem with IEEE 754, not with Java. Programs that now show different (unacceptable) behavior compared to JDK7 should be adjusted to using BigDecimal.

The change affects > 4% of all rounding operations from n decimals to n-1 decimals with n > 1.

------------------------ original post ---------------------------------------

I have read some articles about the improved rounding behavior of the NumberFormat class in JDK 8. This was for example discussed here: http://stackoverflow.com/questions/22797964/is-inconsistency-in-rounding-between-java-7-and-java-8-a-bug

As far as I understand, the new behavior shall better align to the well-known problems with IEEE 754 (https://en.wikipedia.org/wiki/IEEE_floating_point) regarding number representation (eliminate some errorneous corner cases). This should be a minor change, but it isn't somehow (see below).

I ran the code given below with JDK 7 (1.7.0_10) and JDK 8 (1.8.0_65) on Mac OS.

It simply compares rounding 1 decimal first performed on a regular double and then on the corresponding BigDecimal. The default RoundingMode is HALF_EVEN.

Surprising to me is that compared to JDK 7 the number of cases where the rounding mechanism of JDK 8 deviates from the JDK 7 behavior is much higher than I expected.

For JDK 7 all output is 0% (means same result for BigDecimal double). For JDK 8 results look different:

Range [0.0..100000.0), step=0.1 -> diff=0.00%, diffAtTie=0.00%

Range [0.00..10000.00), step=0.01 -> diff=4.00%, diffAtTie=4.00%

Range [0.000..1000.000), step=0.001 -> diff=4.80%, diffAtTie=4.80%

Range [0.0000..100.0000), step=0.0001 -> diff=4.96%, diffAtTie=4.96%

Range [0.00000..10.00000), step=0.00001 -> diff=4.99%, diffAtTie=4.99%

Range [0.000000..1.000000), step=0.000001 -> diff=5.00%, diffAtTie=5.00%

Range [0.0..-100000.0), step=-0.1 -> diff=0.00%, diffAtTie=0.00%

Range [0.00..-10000.00), step=-0.01 -> diff=4.00%, diffAtTie=4.00%

Range [0.000..-1000.000), step=-0.001 -> diff=4.80%, diffAtTie=4.80%

Range [0.0000..-100.0000), step=-0.0001 -> diff=4.96%, diffAtTie=4.96%

Range [0.00000..-10.00000), step=-0.00001 -> diff=4.99%, diffAtTie=4.99%

Range [0.000000..-1.000000), step=-0.000001 -> diff=5.00%, diffAtTie=5.00%

Here my questions:

Has this non-trivial change been announced anywhere (maybe explained in detail)?
Is this also surprising to you or am I on the wrong page?
Can anybody explain the error distribution phenomenon? Starting with the second decimal suddenly the error ratio grows from 4% up to 5 % - and it is always the "HALF" (tie) which gets rounded to the "wrong" side.

Thanks!

Karl

EDIT: As requested here some example values JDK7 (1.7.0_10) vs. JDK8 (1.8.0_65) on Mac OS El Capitan 10.11.2.

For generating the small sample below I used a simpler generated source code, which can be found at the bottom of the table.

The sample only shows rounding from 3 to 2 decimals, but the phenomenon can be seen on any rounding from n to n-1 decimals with n > 1.

As you can see the values in the red columns are different compared to JDK7. This is what makes me nervous (round about 5%).

The BigDecimal columns are just for information. The difference between passing a double vs. passing a String to BigDecimal is plausible (precision loss).

input	JDK7 nf.format( input)	JDK7 nf.format( Double. parseDoub le( "input"))	JDK7 nf.format( new BigDecimal( input))	JDK7 nf.format( new BigDecimal( "input"))	JDK8 nf.format( input)	JDK8 nf.format( Double. parseDoub le("input"))	JDK8 nf.format( new BigDecimal( input))	JDK8 nf.format( new BigDecimal( "input"))
0.005	0.00	0.00	0.01	0.00	0.01	0.01	0.01	0.00
0.015	0.02	0.02	0.01	0.02	0.01	0.01	0.01	0.02
0.025	0.02	0.02	0.03	0.02	0.03	0.03	0.03	0.02
0.065	0.06	0.06	0.07	0.06	0.07	0.07	0.07	0.06
0.075	0.08	0.08	0.07	0.08	0.07	0.07	0.07	0.08
0.085	0.08	0.08	0.09	0.08	0.09	0.09	0.09	0.08
0.155	0.16	0.16	0.15	0.16	0.15	0.15	0.15	0.16
0.165	0.16	0.16	0.17	0.16	0.17	0.17	0.17	0.16
0.175	0.18	0.18	0.17	0.18	0.17	0.17	0.17	0.18
0.215	0.22	0.22	0.21	0.22	0.21	0.21	0.21	0.22
0.225	0.22	0.22	0.23	0.22	0.23	0.23	0.23	0.22
0.235	0.24	0.24	0.23	0.24	0.23	0.23	0.23	0.24
0.265	0.26	0.26	0.27	0.26	0.27	0.27	0.27	0.26
0.295	0.30	0.30	0.29	0.30	0.29	0.29	0.29	0.30
0.325	0.32	0.32	0.33	0.32	0.33	0.33	0.33	0.32
0.355	0.36	0.36	0.35	0.36	0.35	0.35	0.35	0.36
0.385	0.38	0.38	0.39	0.38	0.39	0.39	0.39	0.38
0.405	0.40	0.40	0.41	0.40	0.41	0.41	0.41	0.40
0.415	0.42	0.42	0.41	0.42	0.41	0.41	0.41	0.42
0.435	0.44	0.44	0.43	0.44	0.43	0.43	0.43	0.44
0.445	0.44	0.44	0.45	0.44	0.45	0.45	0.45	0.44
0.465	0.46	0.46	0.47	0.46	0.47	0.47	0.47	0.46
0.475	0.48	0.48	0.47	0.48	0.47	0.47	0.47	0.48
0.495	0.50	0.50	0.49	0.50	0.49	0.49	0.49	0.50
0.505	0.50	0.50	0.51	0.50	0.51	0.51	0.51	0.50
0.525	0.52	0.52	0.53	0.52	0.53	0.53	0.53	0.52
0.545	0.54	0.54	0.55	0.54	0.55	0.55	0.55	0.54
0.575	0.58	0.58	0.57	0.58	0.57	0.57	0.57	0.58
0.595	0.60	0.60	0.59	0.60	0.59	0.59	0.59	0.60
0.615	0.62	0.62	0.61	0.62	0.61	0.61	0.61	0.62
0.645	0.64	0.64	0.65	0.64	0.65	0.65	0.65	0.64
0.665	0.66	0.66	0.67	0.66	0.67	0.67	0.67	0.66
0.685	0.68	0.68	0.69	0.68	0.69	0.69	0.69	0.68
0.695	0.70	0.70	0.69	0.70	0.69	0.69	0.69	0.70
0.715	0.72	0.72	0.71	0.72	0.71	0.71	0.71	0.72
0.735	0.74	0.74	0.73	0.74	0.73	0.73	0.73	0.74
0.765	0.76	0.76	0.77	0.76	0.77	0.77	0.77	0.76
0.785	0.78	0.78	0.79	0.78	0.79	0.79	0.79	0.78
0.805	0.80	0.80	0.81	0.80	0.81	0.81	0.81	0.80
0.815	0.82	0.82	0.81	0.82	0.81	0.81	0.81	0.82
0.835	0.84	0.84	0.83	0.84	0.83	0.83	0.83	0.84
0.855	0.86	0.86	0.85	0.86	0.85	0.85	0.85	0.86
0.885	0.88	0.88	0.89	0.88	0.89	0.89	0.89	0.88
0.905	0.90	0.90	0.91	0.90	0.91	0.91	0.91	0.90
0.925	0.92	0.92	0.93	0.92	0.93	0.93	0.93	0.92
0.955	0.96	0.96	0.95	0.96	0.95	0.95	0.95	0.96
0.975	0.98	0.98	0.97	0.98	0.97	0.97	0.97	0.98
0.995	1.00	1.00	0.99	1.00	0.99	0.99	0.99	1.00
1.015	1.02	1.02	1.01	1.02	1.01	1.01	1.01	1.02

NumberFormat nf = NumberFormat.getInstance(Locale.US);
assertEquals(RoundingMode.HALF_EVEN, nf.getRoundingMode());
nf.setGroupingUsed(false);
nf.setMaximumFractionDigits(2);
nf.setMinimumFractionDigits(2);
System.out.println("0.005;" + nf.format(0.005) + ";" + nf.format(Double.parseDouble("0.005")) + ";"
+ nf.format(new BigDecimal(0.005)) + ";" + nf.format(new BigDecimal("0.005")));
System.out.println("0.015;" + nf.format(0.015) + ";" + nf.format(Double.parseDouble("0.015")) + ";"
+ nf.format(new BigDecimal(0.015)) + ";" + nf.format(new BigDecimal("0.015")));
System.out.println("0.025;" + nf.format(0.025) + ";" + nf.format(Double.parseDouble("0.025")) + ";"
+ nf.format(new BigDecimal(0.025)) + ";" + nf.format(new BigDecimal("0.025")));
System.out.println("0.065;" + nf.format(0.065) + ";" + nf.format(Double.parseDouble("0.065")) + ";"
+ nf.format(new BigDecimal(0.065)) + ";" + nf.format(new BigDecimal("0.065")));
System.out.println("0.075;" + nf.format(0.075) + ";" + nf.format(Double.parseDouble("0.075")) + ";"
+ nf.format(new BigDecimal(0.075)) + ";" + nf.format(new BigDecimal("0.075")));
System.out.println("0.085;" + nf.format(0.085) + ";" + nf.format(Double.parseDouble("0.085")) + ";"
+ nf.format(new BigDecimal(0.085)) + ";" + nf.format(new BigDecimal("0.085")));
System.out.println("0.155;" + nf.format(0.155) + ";" + nf.format(Double.parseDouble("0.155")) + ";"
+ nf.format(new BigDecimal(0.155)) + ";" + nf.format(new BigDecimal("0.155")));
System.out.println("0.165;" + nf.format(0.165) + ";" + nf.format(Double.parseDouble("0.165")) + ";"
+ nf.format(new BigDecimal(0.165)) + ";" + nf.format(new BigDecimal("0.165")));
System.out.println("0.175;" + nf.format(0.175) + ";" + nf.format(Double.parseDouble("0.175")) + ";"
+ nf.format(new BigDecimal(0.175)) + ";" + nf.format(new BigDecimal("0.175")));
System.out.println("0.215;" + nf.format(0.215) + ";" + nf.format(Double.parseDouble("0.215")) + ";"
+ nf.format(new BigDecimal(0.215)) + ";" + nf.format(new BigDecimal("0.215")));
System.out.println("0.225;" + nf.format(0.225) + ";" + nf.format(Double.parseDouble("0.225")) + ";"
+ nf.format(new BigDecimal(0.225)) + ";" + nf.format(new BigDecimal("0.225")));
System.out.println("0.235;" + nf.format(0.235) + ";" + nf.format(Double.parseDouble("0.235")) + ";"
+ nf.format(new BigDecimal(0.235)) + ";" + nf.format(new BigDecimal("0.235")));
System.out.println("0.265;" + nf.format(0.265) + ";" + nf.format(Double.parseDouble("0.265")) + ";"
+ nf.format(new BigDecimal(0.265)) + ";" + nf.format(new BigDecimal("0.265")));
System.out.println("0.295;" + nf.format(0.295) + ";" + nf.format(Double.parseDouble("0.295")) + ";"
+ nf.format(new BigDecimal(0.295)) + ";" + nf.format(new BigDecimal("0.295")));
System.out.println("0.325;" + nf.format(0.325) + ";" + nf.format(Double.parseDouble("0.325")) + ";"
+ nf.format(new BigDecimal(0.325)) + ";" + nf.format(new BigDecimal("0.325")));
System.out.println("0.355;" + nf.format(0.355) + ";" + nf.format(Double.parseDouble("0.355")) + ";"
+ nf.format(new BigDecimal(0.355)) + ";" + nf.format(new BigDecimal("0.355")));
System.out.println("0.385;" + nf.format(0.385) + ";" + nf.format(Double.parseDouble("0.385")) + ";"
+ nf.format(new BigDecimal(0.385)) + ";" + nf.format(new BigDecimal("0.385")));
System.out.println("0.405;" + nf.format(0.405) + ";" + nf.format(Double.parseDouble("0.405")) + ";"
+ nf.format(new BigDecimal(0.405)) + ";" + nf.format(new BigDecimal("0.405")));
System.out.println("0.415;" + nf.format(0.415) + ";" + nf.format(Double.parseDouble("0.415")) + ";"
+ nf.format(new BigDecimal(0.415)) + ";" + nf.format(new BigDecimal("0.415")));
System.out.println("0.435;" + nf.format(0.435) + ";" + nf.format(Double.parseDouble("0.435")) + ";"
+ nf.format(new BigDecimal(0.435)) + ";" + nf.format(new BigDecimal("0.435")));
System.out.println("0.445;" + nf.format(0.445) + ";" + nf.format(Double.parseDouble("0.445")) + ";"
+ nf.format(new BigDecimal(0.445)) + ";" + nf.format(new BigDecimal("0.445")));
System.out.println("0.465;" + nf.format(0.465) + ";" + nf.format(Double.parseDouble("0.465")) + ";"
+ nf.format(new BigDecimal(0.465)) + ";" + nf.format(new BigDecimal("0.465")));
System.out.println("0.475;" + nf.format(0.475) + ";" + nf.format(Double.parseDouble("0.475")) + ";"
+ nf.format(new BigDecimal(0.475)) + ";" + nf.format(new BigDecimal("0.475")));
System.out.println("0.495;" + nf.format(0.495) + ";" + nf.format(Double.parseDouble("0.495")) + ";"
+ nf.format(new BigDecimal(0.495)) + ";" + nf.format(new BigDecimal("0.495")));
System.out.println("0.505;" + nf.format(0.505) + ";" + nf.format(Double.parseDouble("0.505")) + ";"
+ nf.format(new BigDecimal(0.505)) + ";" + nf.format(new BigDecimal("0.505")));
System.out.println("0.525;" + nf.format(0.525) + ";" + nf.format(Double.parseDouble("0.525")) + ";"
+ nf.format(new BigDecimal(0.525)) + ";" + nf.format(new BigDecimal("0.525")));
System.out.println("0.545;" + nf.format(0.545) + ";" + nf.format(Double.parseDouble("0.545")) + ";"
+ nf.format(new BigDecimal(0.545)) + ";" + nf.format(new BigDecimal("0.545")));
System.out.println("0.575;" + nf.format(0.575) + ";" + nf.format(Double.parseDouble("0.575")) + ";"
+ nf.format(new BigDecimal(0.575)) + ";" + nf.format(new BigDecimal("0.575")));
System.out.println("0.595;" + nf.format(0.595) + ";" + nf.format(Double.parseDouble("0.595")) + ";"
+ nf.format(new BigDecimal(0.595)) + ";" + nf.format(new BigDecimal("0.595")));
System.out.println("0.615;" + nf.format(0.615) + ";" + nf.format(Double.parseDouble("0.615")) + ";"
+ nf.format(new BigDecimal(0.615)) + ";" + nf.format(new BigDecimal("0.615")));
System.out.println("0.645;" + nf.format(0.645) + ";" + nf.format(Double.parseDouble("0.645")) + ";"
+ nf.format(new BigDecimal(0.645)) + ";" + nf.format(new BigDecimal("0.645")));
System.out.println("0.665;" + nf.format(0.665) + ";" + nf.format(Double.parseDouble("0.665")) + ";"
+ nf.format(new BigDecimal(0.665)) + ";" + nf.format(new BigDecimal("0.665")));
System.out.println("0.685;" + nf.format(0.685) + ";" + nf.format(Double.parseDouble("0.685")) + ";"
+ nf.format(new BigDecimal(0.685)) + ";" + nf.format(new BigDecimal("0.685")));
System.out.println("0.695;" + nf.format(0.695) + ";" + nf.format(Double.parseDouble("0.695")) + ";"
+ nf.format(new BigDecimal(0.695)) + ";" + nf.format(new BigDecimal("0.695")));
System.out.println("0.715;" + nf.format(0.715) + ";" + nf.format(Double.parseDouble("0.715")) + ";"
+ nf.format(new BigDecimal(0.715)) + ";" + nf.format(new BigDecimal("0.715")));
System.out.println("0.735;" + nf.format(0.735) + ";" + nf.format(Double.parseDouble("0.735")) + ";"
+ nf.format(new BigDecimal(0.735)) + ";" + nf.format(new BigDecimal("0.735")));
System.out.println("0.765;" + nf.format(0.765) + ";" + nf.format(Double.parseDouble("0.765")) + ";"
+ nf.format(new BigDecimal(0.765)) + ";" + nf.format(new BigDecimal("0.765")));
System.out.println("0.785;" + nf.format(0.785) + ";" + nf.format(Double.parseDouble("0.785")) + ";"
+ nf.format(new BigDecimal(0.785)) + ";" + nf.format(new BigDecimal("0.785")));
System.out.println("0.805;" + nf.format(0.805) + ";" + nf.format(Double.parseDouble("0.805")) + ";"
+ nf.format(new BigDecimal(0.805)) + ";" + nf.format(new BigDecimal("0.805")));
System.out.println("0.815;" + nf.format(0.815) + ";" + nf.format(Double.parseDouble("0.815")) + ";"
+ nf.format(new BigDecimal(0.815)) + ";" + nf.format(new BigDecimal("0.815")));
System.out.println("0.835;" + nf.format(0.835) + ";" + nf.format(Double.parseDouble("0.835")) + ";"
+ nf.format(new BigDecimal(0.835)) + ";" + nf.format(new BigDecimal("0.835")));
System.out.println("0.855;" + nf.format(0.855) + ";" + nf.format(Double.parseDouble("0.855")) + ";"
+ nf.format(new BigDecimal(0.855)) + ";" + nf.format(new BigDecimal("0.855")));
System.out.println("0.885;" + nf.format(0.885) + ";" + nf.format(Double.parseDouble("0.885")) + ";"
+ nf.format(new BigDecimal(0.885)) + ";" + nf.format(new BigDecimal("0.885")));
System.out.println("0.905;" + nf.format(0.905) + ";" + nf.format(Double.parseDouble("0.905")) + ";"
+ nf.format(new BigDecimal(0.905)) + ";" + nf.format(new BigDecimal("0.905")));
System.out.println("0.925;" + nf.format(0.925) + ";" + nf.format(Double.parseDouble("0.925")) + ";"
+ nf.format(new BigDecimal(0.925)) + ";" + nf.format(new BigDecimal("0.925")));
System.out.println("0.955;" + nf.format(0.955) + ";" + nf.format(Double.parseDouble("0.955")) + ";"
+ nf.format(new BigDecimal(0.955)) + ";" + nf.format(new BigDecimal("0.955")));
System.out.println("0.975;" + nf.format(0.975) + ";" + nf.format(Double.parseDouble("0.975")) + ";"
+ nf.format(new BigDecimal(0.975)) + ";" + nf.format(new BigDecimal("0.975")));
System.out.println("0.995;" + nf.format(0.995) + ";" + nf.format(Double.parseDouble("0.995")) + ";"
+ nf.format(new BigDecimal(0.995)) + ";" + nf.format(new BigDecimal("0.995")));
System.out.println("1.015;" + nf.format(1.015) + ";" + nf.format(Double.parseDouble("1.015")) + ";"
+ nf.format(new BigDecimal(1.015)) + ";" + nf.format(new BigDecimal("1.015")));

package comp7;

importstaticorg.junit.Assert.assertEquals;
importjava.math.BigDecimal;
importjava.math.RoundingMode;
importjava.text.NumberFormat;
importjava.util.Locale;
importorg.junit.Test;
publicclass JavaRoundTest {
    @Test
publicvoid testJavaNumberFormatRounding()throwsException{
int total = 1_000_000;
long time = System.currentTimeMillis();
for(boolean negate : newboolean[]{false, true}){
for(int decimals = 1; decimals <7; decimals++){
                performRun(total, negate, decimals);
}
}
System.out.println("Finished after " + (System.currentTimeMillis() - time) + " ms");
}
privatevoid performRun(int upperBound, boolean negate, int decimals){
NumberFormat testedNumberFormat = NumberFormat.getInstance(Locale.US);
        assertEquals(RoundingMode.HALF_EVEN, testedNumberFormat.getRoundingMode());
        testedNumberFormat.setGroupingUsed(false);
        testedNumberFormat.setMaximumFractionDigits(decimals - 1);
        testedNumberFormat.setMinimumFractionDigits(decimals - 1);
        DiffCountResult dcr = computeRun(testedNumberFormat, upperBound, negate, decimals);
double diffShare = ((double) dcr.diffCount / upperBound);
double diffAtTieShare = ((double) dcr.diffAtTieCount / upperBound);
NumberFormat percNumberFormat = NumberFormat.getPercentInstance(Locale.US);
        percNumberFormat.setMaximumFractionDigits(2);
        percNumberFormat.setMinimumFractionDigits(2);
        percNumberFormat.setGroupingUsed(true);
System.out.println("Range [" + makeDecimalString(0, negate, decimals) + ".."
                + makeDecimalString(upperBound, negate, decimals) + "), step=" + makeDecimalString(1, negate, decimals)
                + " -> diff=" + percNumberFormat.format(newBigDecimal(diffShare)) + ", diffAtTie="
                + percNumberFormat.format(newBigDecimal(diffAtTieShare)));
}
private DiffCountResult computeRun(NumberFormat nf, int upperBound, boolean negate, int decimals){
        DiffCountResult dcr = new DiffCountResult();
for(int i = 0; i < upperBound; i++){
String sNum = makeDecimalString(i, negate, decimals);
String sRoundedDouble = nf.format(Double.parseDouble(sNum));
String sRoundedBigDecimal = nf.format(newBigDecimal(sNum));
if(!sRoundedDouble.equals(sRoundedBigDecimal)){
                dcr.diffCount++;
if(sNum.charAt(sNum.length() - 1) == '5'){
                    dcr.diffAtTieCount++;
}
}
}
return dcr;
}
staticString makeDecimalString(int sourceValue, boolean negate, int decimals){
String res = "0.";
String sourceString = "" + sourceValue;
int sourceLen = sourceString.length();
if(sourceLen <= decimals){
int padLen = decimals - sourceLen;
for(int i = 0; i < padLen; i++){
                res = res + "0";
}
            res = res + sourceString;
}else{
int dotPos = sourceLen - decimals;
            res = sourceString.substring(0, dotPos) + "." + sourceString.substring(dotPos);
}
if(sourceValue >0&& negate){
            res = "-" + res;
}
return res;
}
staticclass DiffCountResult {
int diffCount = 0;
int diffAtTieCount = 0;
}
}

Added on Dec 11 2015

#java-8-questions

2 comments

2,360 views