My problem is a little bit hard to explain. I run into a problem when I try to get the integerpart of a double. Numberformat is useless in this case, in any case I have definitely to prevent rounding because this gives me huge trouble within the calculation. The result is needed for trigonometric functions (sin, cos, tan). Only chance I see, is to convert the double to a string and reconverting it then. Is there no other elegant way?
The following code will explain my problem (it's not real code, just put it in this form for better understanding)...
This is the formula I try to calculate: L0=2*PI*(0.374897+1325.55241*0.2167830253-INT(0.374897+1325.55241*0.2167830253))
The correct result of the formula is: 0.4139530972
So on top of the code window you see the complete formula "converted" to B4A.
Wrong result is: -1.68289693861034
Follow the next 4 lines of code with some numberformat examples, only using the right part of the formula. The reason for the error is the rounding when using numberformat.
Result of the 2nd line is: 288
Result of the 3rd line is: 287.732158633506
Result of the 4th line is: 288
Result of the 5th line is: 287.732158633506
So using numberformat in this case is completely nonsense, even if you try it "backwards". To set the formatting rule to "integers set to zero" delivers not zero, it delivers the complete integer. I would like to get the complete fraction part of the number. So this is impossible with numberformat, because of rounding.
The fraction part is then later used by trigonometrics. And sin(0.4139530972) is different from sin(-1.68289693861034). Got my problem?
Don't wonder if you calculate and get a different values. While I wrote this here, everything changed and I could not log everything and put it here fast enough. I am using variables and put fixed numbers into the formulas above.
Any ideas other than "stringcalculation"?
Cheers, BV
The following code will explain my problem (it's not real code, just put it in this form for better understanding)...
This is the formula I try to calculate: L0=2*PI*(0.374897+1325.55241*0.2167830253-INT(0.374897+1325.55241*0.2167830253))
The correct result of the formula is: 0.4139530972
B4X:
L0=2*cPI*(0.374897+1325.55241*0.2167830253-NumberFormat((0.374897+1325.55241*0.2167830253),1,0))
L0=NumberFormat((0.374897+1325.55241*0.2167830253),1,0)
L0=NumberFormat((0.374897+1325.55241*0.2167830253),0,12)
L0=NumberFormat2((0.374897+1325.55241*0.2167830253),1,0,0,False)
L0=NumberFormat2((0.374897+1325.55241*0.2167830253),0,12,12,False)So on top of the code window you see the complete formula "converted" to B4A.
Wrong result is: -1.68289693861034
Follow the next 4 lines of code with some numberformat examples, only using the right part of the formula. The reason for the error is the rounding when using numberformat.
Result of the 2nd line is: 288
Result of the 3rd line is: 287.732158633506
Result of the 4th line is: 288
Result of the 5th line is: 287.732158633506
So using numberformat in this case is completely nonsense, even if you try it "backwards". To set the formatting rule to "integers set to zero" delivers not zero, it delivers the complete integer. I would like to get the complete fraction part of the number. So this is impossible with numberformat, because of rounding.
The fraction part is then later used by trigonometrics. And sin(0.4139530972) is different from sin(-1.68289693861034). Got my problem?
Don't wonder if you calculate and get a different values. While I wrote this here, everything changed and I could not log everything and put it here fast enough. I am using variables and put fixed numbers into the formulas above.
Any ideas other than "stringcalculation"?
Cheers, BV