wimpie3 Well-Known Member Licensed User Longtime User Oct 30, 2013 #1 Here is a little riddle for you. Given the following code, what does B4A return? B4X: Dim s As String s="12.340" Dim f As Float f=s Log(f) You'd expect 12.34, right? The conversion from the string to the floating point numeric value should have dropped the last 0. Well, it turns out the result that appears in the log is B4X: 12.34000015258789 Some kind of rounding error? What's going on here and how can I get the "12.34" I want?
Here is a little riddle for you. Given the following code, what does B4A return? B4X: Dim s As String s="12.340" Dim f As Float f=s Log(f) You'd expect 12.34, right? The conversion from the string to the floating point numeric value should have dropped the last 0. Well, it turns out the result that appears in the log is B4X: 12.34000015258789 Some kind of rounding error? What's going on here and how can I get the "12.34" I want?
M Mahares Expert Licensed User Longtime User Oct 30, 2013 #2 If you declare it as double, it will yield what you want. See example. I have seen a few posts discussing this with Erel's leadership: B4X: Dim s As String ="12.340" Dim f As Double f=s Msgbox(f,"") 'displays 12.34 Upvote 0
If you declare it as double, it will yield what you want. See example. I have seen a few posts discussing this with Erel's leadership: B4X: Dim s As String ="12.340" Dim f As Double f=s Msgbox(f,"") 'displays 12.34
Erel B4X founder Staff member Licensed User Longtime User Oct 30, 2013 #4 The correct way to convert a number to a string is to use NumberFormat or NumberFormat2. Upvote 0