Android Question Problem with Float data type

toby

Well-Known Member
Licensed User
Longtime User
I want to store latitude and longitude data as Float data type values. When I moved mouse over flt1 variable, it showed 123.45679 while the Log() yielded another slightly different value.

Could someone tell me what's wrong, please?

Problem with float data type: 
Dim flt1 As Float=123.4567890123456
    'hoveover value: 123.45679 (rounded with 5 decimal digits)
    Log(flt1)      'output: 123.456787109375

TIA
 
Sort by date Sort by votes
E

emexes

Expert
Licensed User
A four-byte single-precision float has a 24 bit mantissa = accurate to within 0.85 metres worst case (at equator, opposite prime meridian ie Greenwich London)

An eight-byte double-precision float has an extra 29 bits of mantissa = accurate to within 1.6 nanometres worst case (in Australia aka within a bee's dick)
 
Upvote 0
You must log in or register to reply here.

Similar Threads

  • Question
Android Question How to write an array of floats to a File
Replies
6
Views
414
GiovanniPolese
E
Floating point accuracy and calculate Pi
Replies
1
Views
4K
Erel
  • Article
B4A Library Floating Action Menu (FAM) and Floating Action Button (FAB)
2 3
Replies
51
Views
30K
DonManfred
  • Question
Android Question Float type is wrong!
2
Replies
25
Views
5K
LucaMs
  • Article
Android Tutorial Data Type and Range
Replies
1
Views
30K
agraham
Menu
Log in
Register
Cookies are required to use this site. You must accept them to continue using the site. Learn more…