I'm programming a calculator for flow in open channels where the user can input a 'roughness coefficient' that I'd like to round to two significant digits. But I'm getting inconsistent results.
console.log(0.00135.toPrecision(2))// -> 0.0014
console.log(0.0135.toPrecision(2)) // -> 0.013
console.log(0.135.toPrecision(2)) // -> 0.14
All I'm doing is shifting the decimal point (multiplying the number by 10) and would assume that the result should also be shifted by a factor of 10. But in some cases the 5 rounds up and in other cases the 5 rounds down. I realize there are 'odd-even' rules that can be used for rounding but they don't apply in this case when multiplying/dividing by 10.
Some other 'puzzling' (to me, at any rate) results:
console.log(0.0235.toPrecision(2))// -> 0.024
console.log(0.0245.toPrecision(2))// -> 0.025
console.log(0.0255.toPrecision(2))// -> 0.025
console.log(0.0265.toPrecision(2))// -> 0.026
console.log(0.0275.toPrecision(2))// -> 0.028
console.log(0.0285.toPrecision(2))// -> 0.029
Is it to do with floating point representation for small numbers? I'm getting the same behavior on FF and Brave. Is there an easy fix that anyone can suggest?