Detailed explanation of the rounding accuracy problem of the toFixed() method in JS

The pitfalls

Recently, at work, when calculating the discount price of a product, there is always a difference of one cent in the price. Issues involving money are more sensitive. After investigation, it was finally discovered that it was a problem with the native toFixed method of JS.

My goodness, what kind of rules are these? . . (⊙o⊙)

Filling method

Don't rush to explore the problem. Since you have found the problem, fix the bug first. If the native method doesn't work, just write one yourself. It only takes a few minutes, hahaha!

/**
 * Keep the number of decimal places, automatically fill in zeros, round up* @param num: value* @param digit: number of decimal places* @returns string
 */ 
function myFixed(num, digit) {
  if(Object.is(parseFloat(num), NaN)) {
    return console.log(`Incoming value: ${num} is not a number`);
  }
  num = parseFloat(num);
  return (Math.round((num + Number.EPSILON) * Math.pow(10, digit)) / Math.pow(10, digit)).toFixed(digit);
}

What kind of pit?

OK, now that the bug is fixed, let’s explore the secrets of toFixed.

Uh... First of all, Em... Let's search on Baidu, targeting Baidu programming engineers. Sure enough, there are a lot of results. It seems to be a classic problem.

After some understanding, we found out that the rounding used by the toFixed method is not the literal rounding we understand. The toFixed method uses a strange method called "rounding to even numbers", also known as the banker's algorithm. What does this mean?

Complete statement: "Round up to the nearest five. If the number after five is not zero, add one. If the number after five is zero, consider whether it is odd or even. If the number before five is even, discard it. If the number before five is odd, add one."

The general meaning is: when the value of the discarded digit is ≤4, discard it; when it is ≥6, add it; when it =5, it is determined according to the number after 5; when there is a non-zero number after 5, round 5 to 1; when there is no valid number after 5, it is divided into two cases: if the number before 5 is an even number, round 5 and do not add it; if the number before 5 is an odd number, round 5 to 1.

Following this rule, I ran some more tests on the browser, but it still didn’t feel right.

// The number before five is an even number, so it is not discarded?
console.log(1.00000065.toFixed(7)); // 1.0000007 error console.log(1.000000065.toFixed(8)); // 1.00000007 error // The number before five is an odd number, so there is no increase by one?
console.log(1.00000015.toFixed(7)); // 1.0000001 error console.log(1.000000015.toFixed(8)); // 1.00000001 error

Why is this? It's really confusing. . . (︶︿︶)

After some further exploration, I finally got some results. Now let's take a look at the ECMAScript specification's definition of this method. Sometimes returning to the specification is the most reliable way.

The above picture is the definition of the entire toFixed method, but it is a translated version. There will be some differences but the difference is not big. You can also click the link above to view the original text. We mainly focus on the red box part in the figure and use the formula to calculate the discarded value.

Let's take the following two examples and test them.

console.log(1.0000005.toFixed(6)); // 1.000001 correct console.log(1.00000005.toFixed(7)); // 1.0000000 wrong

First, according to the conditions in the red box, x<10^21, 1.0000005 and 1.00000005 are both less than 10^21, so we can directly use the formula n / 10^ - x to play with.

Let's first substitute x=1.0000005 into the formula to see the situation:

// Assume n1
var n1 = 1000000;
var x = 1.0000005;
var f = 6;
console.log((n1 / Math.pow(10, f) - x)); // -5.00000000069889e-7

// Assume n2
var n2 = 1000001;
var x = 1.0000005;
var f = 6;
console.log((n2 / Math.pow(10, f) - x)); // 4.9999999998478444e-7

From the results, we can see that when n1=1000001, the result is the value closest to 0, so:

console.log(1.0000005.toFixed(6)); // 1.000001 is correct

Let's try again when x=1.00000005 is substituted into the formula:

// Assume n1
var n1 = 10000000;
var x = 1.00000005;
var f = 7;
console.log((n1 / Math.pow(10,f) - x)); // -4.99999999918171056e-8

// Assume n2
var n2 = 10000001;
var x = 1.00000005;
var f = 7;
console.log((n2 / Math.pow(10,f) - x)); // 5.000000014021566e-8

From the results, we can see that when n2=10000001, the result is the value closest to 0, so:

console.log(1.00000005.toFixed(7)); // 1.0000000 Error

Oh... I just realized I've dug a big hole for myself when I got here. Why do I have to use so many zeros? I'm dizzy from counting them. . .

In general, the above examples are just to teach you how to calculate the results using the formulas defined in the specifications. If you can understand the specifications, then there is no problem in directly substituting them in.

Summarize

This is the end of this article about the rounding precision problem of the toFixed() method in JS. For more information about the rounding precision problem of JS toFixed(), please search for previous articles on 123WORDPRESS.COM or continue to browse the related articles below. I hope you will support 123WORDPRESS.COM in the future!