07-27-2022, 05:34 PM
(This post was last modified: 07-27-2022, 06:00 PM by Kernelpanic.)
(07-27-2022, 03:31 PM)SierraKen Wrote: Pete, yeah it's not exactly a scientific calculator and I believe QB64 rounds it off if you go too far in the decimal range. Thanks for showing me but I'm just going to leave it like it is. I don't know enough math to make such an algorithm and I don't think .000000000000001 will make much difference with anything, unless like I said, you are a scientist or mathematician. LOL
@Ken, there is an error in the calculations that appears to be related to the zero.
For example:
0.01 -> 1 -> x 3 = 3 Correct: 0.03
1.02 -> 12 -> / 3 = 4 Correct: 0.34
The problem is: ".0"