D notation bug
#5
(07-09-2023, 05:00 PM)bplus Wrote: As reported at the other forum:
Code: (Select All)
Print (1D+17) * (1D+17) ' <<< the buggy one!
Print (1E+17) * (1E+17) ' E gets it right

The D notation is off by 1 power of 10.
I'm pretty sure it's just a problem with rounding along with how the number is displayed. The floating point error for the double results in the value being slightly less than 1D+34, and because the value is slightly less it gets rounded to 1D+33 when displaying.

While I don't know what QBasic used to do. this seems like a bug since the number is still significantly closer to 1D+34 than 1D+33, so it should be displayed as that. I suspect it may be just rounding the mantissa without adjusting the exponent or something, I took a quick look at the code that renders the number but the issue wasn't clear to me. The way it works is pretty weird as it is, it might be easier to just redo it.
Reply


Messages In This Thread
D notation bug - by bplus - 07-09-2023, 05:00 PM
RE: D notation bug - by Jack - 07-09-2023, 05:53 PM
RE: D notation bug - by Jack - 07-09-2023, 06:10 PM
RE: D notation bug - by mnrvovrfc - 07-09-2023, 06:32 PM
RE: D notation bug - by mnrvovrfc - 07-14-2023, 09:53 PM
RE: D notation bug - by DSMan195276 - 07-09-2023, 06:39 PM
RE: D notation bug - by SMcNeill - 07-09-2023, 07:03 PM
RE: D notation bug - by bplus - 07-09-2023, 07:27 PM
RE: D notation bug - by Jack - 07-09-2023, 10:40 PM
RE: D notation bug - by bplus - 07-10-2023, 12:42 AM
RE: D notation bug - by Jack - 07-10-2023, 01:12 AM
RE: D notation bug - by bplus - 07-10-2023, 02:19 AM
RE: D notation bug - by bplus - 08-26-2023, 02:36 PM
RE: D notation bug - by Jack - 08-26-2023, 05:12 PM



Users browsing this thread: 14 Guest(s)