08-30-2022, 01:00 PM
(08-30-2022, 04:20 AM)DSMan195276 Wrote: The problem you're experiencing is that theSINGLEandDOUBLEdata types are floating point numbers with a limited precision. WithSINGLEit's about 8 digits or so (23 bits of precision), meaning that it cannot accurately represent the answer 1,731,560,447.097983 and has to round it to the nearest value that can be represented. That value is the integer result you get, and it's an integer because the integer part of the original number was already too precise to represent (that's why it rounds in the wrong direction).
When you useDOUBLEyou get more like 15 digits of precision, so the result will be a lot closer (but still not quite right) and more importantly there will still be a fractional component in the answer since the integer component isn't too large anymore.
Thank you for this explanation. I wasn't sure about these types of variables, but this makes perfect sense. Much appreciated.