11-06-2022, 12:42 AM
(11-05-2022, 04:44 PM)Dimster Wrote: I'm damn sure you guys have explained this once before so if anyone could point me to the thread I'd appreciate it. Can't seem to come up with the one I kind of remember that addressed this, it could be on the other site. Anyway here's the simply code
Dim a As Double
Dim b As Single
a = .12345678
Print a
b = Abs(a * 10000000)
Print a; b
So my search is to find out what happened to the 7 in the difference between variable a and variable b
This topic is of a similar nature to an old thread: -
Possible error in INTEGER64 calculations
https://qb64forum.alephc.xyz/index.php?topic=4210.0
and my reply (last response to thread + sub-link (#183))'
Quote:Attached is the preliminary work in trying to establish the first transition point in going from Number Space Ordinary (NSO) where all positive integers are representable in _DOUBLE TO where the granularity is larger than 1 (i.e., resulting in "skipped" integers).
So, the above referred attachment (comparison table SINGLE vs DOUBLE vs FLOAT vs INTEGER64 etc.) among other things illustrates the "Moral": -
PRINT USING "may" give a more accurate representation of the compute result than PRINT only.
Hope this reply here is of help explaining/illustrating.