Large numbers - Printable Version +- QB64 Phoenix Edition (https://staging.qb64phoenix.com) +-- Forum: QB64 Rising (https://staging.qb64phoenix.com/forumdisplay.php?fid=1) +--- Forum: Code and Stuff (https://staging.qb64phoenix.com/forumdisplay.php?fid=3) +---- Forum: Help Me! (https://staging.qb64phoenix.com/forumdisplay.php?fid=10) +---- Thread: Large numbers (/showthread.php?tid=828) |
Large numbers - james2464 - 08-29-2022 I'm working on a (purely academic) math exercise that involves a 12 digit integer...I was planning to use 'long' integers but that seems to be limited to smaller numbers. I assumed long integers could handle such a number, but that apparently isn't the case. Is scientific notation the only format for integers at these values? RE: Large numbers - SMcNeill - 08-29-2022 12 digits works with _INTEGER64 types. DIM variable as _INTEGER64. https://qb64phoenix.com/qb64wiki/index.php/INTEGER64 RE: Large numbers - james2464 - 08-29-2022 Thank you! RE: Large numbers - james2464 - 08-30-2022 I'm still having trouble, as I don't fully understand the variable types. I'm wondering if this is something that can be done in QB64 or if this requires another language. I want to compare these two values, and it needs to be precise. I found that QB64 considered these equal (it appears to have rounded the results) and it's possible, even likely, that it's just my mistake. Value 1: int (600,851,475,143 / 347) Value 2: 600,851,475,143 / 347 I just want the integer version compared to the raw version. If it evenly divides then they are equal. If not, then they aren't. According to my phone calculator, the actual result of this is: 1,731,560,447.097983 So Value 1 isn't equal to Value 2. Can anyone advise on the correct or proper variable types to allow this to work? Or is this a limitation of QB64? The results I got in QB64 are 1,731,560,448 for both Value 1 and Value 2 Using this code Code: (Select All) checka& = Int(aa&& / p(s)) RE: Large numbers - SMcNeill - 08-30-2022 DIM p(s) AS DOUBLE and you avoid the trouble. This may be a glitch in QB64 when dividing by a SINGLE type here. It'll need to be looked into. Code: (Select All) Dim x As _Integer64, y As Double RE: Large numbers - DSMan195276 - 08-30-2022 The problem you're experiencing is that the SINGLEand DOUBLEdata types are floating point numbers with a limited precision. With SINGLEit's about 8 digits or so (23 bits of precision), meaning that it cannot accurately represent the answer 1,731,560,447.097983 and has to round it to the nearest value that can be represented. That value is the integer result you get, and it's an integer because the integer part of the original number was already too precise to represent (that's why it rounds in the wrong direction). When you use DOUBLEyou get more like 15 digits of precision, so the result will be a lot closer (but still not quite right) and more importantly there will still be a fractional component in the answer since the integer component isn't too large anymore. RE: Large numbers - mdijkens - 08-30-2022 Use MOD: if aa&& Mod p(s) = 0 then ... RE: Large numbers - james2464 - 08-30-2022 This code seems to work properly Code: (Select All) Dim A As _Integer64 RE: Large numbers - james2464 - 08-30-2022 Thank you RE: Large numbers - james2464 - 08-30-2022 (08-30-2022, 04:20 AM)DSMan195276 Wrote: The problem you're experiencing is that theSINGLEandDOUBLEdata types are floating point numbers with a limited precision. WithSINGLEit's about 8 digits or so (23 bits of precision), meaning that it cannot accurately represent the answer 1,731,560,447.097983 and has to round it to the nearest value that can be represented. That value is the integer result you get, and it's an integer because the integer part of the original number was already too precise to represent (that's why it rounds in the wrong direction). Thank you for this explanation. I wasn't sure about these types of variables, but this makes perfect sense. Much appreciated. |