0
259
ViewsDecimal values are truncating with to_f
I have model called Item, where I am updating the unit_price (Data Type is Decimal) value, currently I am not putting any limit when storing the value, storing the value as it is. But now I can see this PG error PG::NumericValueOutOfRange, when the value exceeds the limit.
So I was just trying to limit the value and checking something in the console, Below is the data. (Here in the data I am not putting all the decimal values)
#<Item id: 167199, description: "192830139", category_id: 10327, unit_id: 5596, weight: 0.1e5, unit_price: 0.4083333333659917816764132553606237816656920077972709552126705653021442494641325536062378168e1
i = Item.find 167199
i.unit_price.to_f
=> 4.083333333659918
#<Item id: 167199, description: "192830139", category_id: 10327, unit_id: 5596, weight: 0.1e5, unit_price: 0.6511366980197836882065909262763993442019943880913510722934069011050182329156169820243980265070876781866034494363303661586489199452739290976143216266200531728395970406461889852558384421962422689303402903e-2
i.unit_price.to_f
=> 0.006511366980197837
Can I know what will be the reason the to_f automatically reduce the limit of the decimal? What will be best way to solve this issue, I was just thinking about some truncate with some limit.
2 answers
Answer question0
Can I know what will be the reason the to_f automatically reduce the limit of the decimal?
The reason is the to_f methods are used to convert objects to Floats, which are standard 64-bit double precision floating point numbers. The precision of these numbers is limited, therefore the precision of the original object must be automatically reduced during the conversion process in order to make it fit in a Float. All extra precision is lost.
It looks like you are using the BigDecimal class. The BigDecimal#to_f method will convert the arbitrary precision floating point decimal object into a Float. Naturally, information will be lost during this conversion should the big decimal be more precise than what Floats allow. This conversion can actually overflow or underflow if limits are exceeded.
I was just thinking about some truncate with some limit
There is a truncate method if you'd like explicit control over the precision of the result. No rounding of any kind will occur, there is a separate method for that.
BigDecimal#truncateDeletes the entire fractional part of the number, leaving only an integer.
BigDecimal('3.14159').truncate #=> 3BigDecimal#truncate(n)Keeps
ndigits of precision, deletes the rest.BigDecimal('3.14159').truncate(3) #=> 3.141
0
you can use ruby's built-in .truncate() method
for example:
floatNum = 1.222222222222222
truncatedNum = floatNum.truncate(3) #3 is the number of decimal places you want
puts floatNum #returns 1.222
another way is to use the .round() method
for example:
