Precision
#22
(11-23-2022, 04:22 PM)Chris Wrote: Thanks for the hints. After looking at the formulas, I came to the conclusion that the calculations are basically on whole numbers. "Bplus" are not Easter formulas. It is recommended that the calculations are as precise as possible, because the results may be distorted. Someone told me (except Forum) that at least 32-bit precision must be used. I don't really understand what the 32-bit presidency means.

Regards Chris

Ah it is Integers! sorry I was having a little fun about Easter calculations.

You could then try at start

Code: (Select All)
_DEFINE A-Z AS _INTEGER64
leave off # or ## signs unless you know for sure they are supposed to be Doubles or _Floats (decimal point numbers).

You would have 64 bit Integer range which should be better than 32 bit, even more if knew all were positive even in sub calculations but probably not likely
 
Code: (Select All)
_DEFINE A-Z AS _UNSIGNED _INTEGER64

But your block of code is missing many variable values, not testable even if we knew what an answer looked like. We have to know the meaning of variables to assign the correct type.
b = b + ...
Reply


Messages In This Thread
Precision - by Chris - 11-22-2022, 08:43 PM
RE: Precision - by SMcNeill - 11-22-2022, 09:12 PM
RE: Precision - by mnrvovrfc - 11-22-2022, 09:14 PM
RE: Precision - by bplus - 11-22-2022, 09:15 PM
RE: Precision - by mnrvovrfc - 11-22-2022, 09:20 PM
RE: Precision - by Chris - 11-22-2022, 09:26 PM
RE: Precision - by Chris - 11-22-2022, 09:33 PM
RE: Precision - by Chris - 11-22-2022, 09:37 PM
RE: Precision - by Chris - 11-22-2022, 09:40 PM
RE: Precision - by mnrvovrfc - 11-22-2022, 09:41 PM
RE: Precision - by Kernelpanic - 11-23-2022, 12:58 AM
RE: Precision - by Chris - 11-23-2022, 04:22 PM
RE: Precision - by bplus - 11-23-2022, 04:30 PM
RE: Precision - by Chris - 11-23-2022, 05:28 PM
RE: Precision - by Kernelpanic - 11-23-2022, 07:45 PM
RE: Precision - by mnrvovrfc - 11-23-2022, 07:55 PM
RE: Precision - by Jack - 11-24-2022, 02:44 PM
RE: Precision - by Chris - 11-24-2022, 07:06 PM
RE: Precision - by Kernelpanic - 11-24-2022, 11:59 PM
RE: Precision - by Chris - 11-27-2022, 07:01 PM
RE: Precision - by Kernelpanic - 11-27-2022, 09:08 PM
RE: Precision - by Kernelpanic - 11-29-2022, 11:28 PM
RE: Precision - by bplus - 11-22-2022, 08:54 PM
RE: Precision - by Pete - 11-22-2022, 08:59 PM
Precision - by Chris - 11-22-2022, 09:00 PM
RE: Precision - by Chris - 11-22-2022, 09:04 PM
RE: Precision - by SMcNeill - 11-22-2022, 09:10 PM
RE: Precision - by Chris - 11-22-2022, 09:08 PM
RE: Precision - by Pete - 11-22-2022, 09:25 PM
RE: Precision - by bplus - 11-22-2022, 09:26 PM
RE: Precision - by bplus - 11-22-2022, 10:42 PM



Users browsing this thread: 11 Guest(s)