Sunday, February 10, 2019

floating point - Dealing with float precision in Javascript




I have a large amount of numeric values y in javascript. I want to group them by rounding them down to the nearest multiple of x and convert the result to a string.



How do I get around the annoying floating point precision?



For example:



0.2 + 0.4 = 0.6000000000000001



Two things I have tried:



>>> y = 1.23456789 
>>> x = 0.2
>>> parseInt(Math.round(Math.floor(y/x))) * x;
1.2000000000000002


and:




>>> y = 1.23456789 
>>> x = 0.2
>>> y - (y % x)
1.2000000000000002

Answer



From this post: How to deal with floating point number precision in JavaScript?



You have a few options:





  • Use a special datatype for decimals, like decimal.js

  • Format your result to some fixed number of significant digits, like this:
    (Math.floor(y/x) * x).toFixed(2)

  • Convert all your numbers to integers


No comments:

Post a Comment

plot explanation - Why did Peaches' mom hang on the tree? - Movies & TV

In the middle of the movie Ice Age: Continental Drift Peaches' mom asked Peaches to go to sleep. Then, she hung on the tree. This parti...