Large numbers

[james2464]

I'm working on a (purely academic) math exercise that involves a 12 digit integer...I was planning to use 'long' integers but that seems to be limited to smaller numbers.

I assumed long integers could handle such a number, but that apparently isn't the case.   Is scientific notation the only format for integers at these values?

[SMcNeill]

12 digits works with _INTEGER64 types.

DIM variable as _INTEGER64.

https://qb64phoenix.com/qb64wiki/index.php/INTEGER64

[james2464]

Thank you!

[james2464]

I'm still having trouble, as I don't fully understand the variable types.

I'm wondering if this is something that can be done in QB64 or if this requires another language.

I want to compare these two values, and it needs to be precise.  I found that QB64 considered these equal (it appears to have rounded the results) and it's possible, even likely, that it's just my mistake.

Value 1:  int (600,851,475,143 / 347)
Value 2:  600,851,475,143 / 347

I just want the integer version compared to the raw version.  If it evenly divides then they are equal.  If not, then they aren't.

According to my phone calculator, the actual result of this is:

1,731,560,447.097983

So Value 1 isn't  equal to Value 2. 

Can anyone advise on the correct or proper variable types to allow this to work?  Or is this a limitation of QB64?

The results I got in QB64 are 1,731,560,448 for both Value 1 and Value 2

Using this code

Code: (Select All)

checka& = Int(aa&& / p(s))
checkb# = aa&& / p(s)

[SMcNeill]

DIM p(s) AS DOUBLE and you avoid the trouble.

This may be a glitch in QB64 when dividing by a SINGLE type here. It'll need to be looked into.

Code: (Select All)

Dim x As _Integer64, y As Double

a&& = 600851475143
p! = 347

x = Int(a&& / p!)
y = a&& / p!

Print x
Print y

p# = 347

x = Int(a&& / p#)
y = a&& / p#

Print x
Print y

[DSMan195276]

The problem you're experiencing is that the

SINGLE

and

DOUBLE

data types are floating point numbers with a limited precision. With

SINGLE

it's about 8 digits or so (23 bits of precision), meaning that it cannot accurately represent the answer 1,731,560,447.097983 and has to round it to the nearest value that can be represented. That value is the integer result you get, and it's an integer because the integer part of the original number was already too precise to represent (that's why it rounds in the wrong direction).

When you use

DOUBLE

you get more like 15 digits of precision, so the result will be a lot closer (but still not quite right) and more importantly there will still be a fractional component in the answer since the integer component isn't too large anymore.

[mdijkens]

Use MOD:
if aa&& Mod p(s) = 0 then ...

[james2464]

This code seems to work properly

Code: (Select All)

Dim A As _Integer64
Dim B As Double
Dim C As Integer
Dim D As Long

A = 600851475143
C = 347
B = A / C
D = Int(A / C)

Print A
Print C
Print B
Print D

If B = D Then
    Print "equal"
Else
    Print "not equal"
End If

End

[james2464]

Thank you

[james2464]

The problem you're experiencing is that the

SINGLE

and

DOUBLE

data types are floating point numbers with a limited precision. With

SINGLE

it's about 8 digits or so (23 bits of precision), meaning that it cannot accurately represent the answer 1,731,560,447.097983 and has to round it to the nearest value that can be represented. That value is the integer result you get, and it's an integer because the integer part of the original number was already too precise to represent (that's why it rounds in the wrong direction).

When you use

DOUBLE

you get more like 15 digits of precision, so the result will be a lot closer (but still not quite right) and more importantly there will still be a fractional component in the answer since the integer component isn't too large anymore.

Thank you for this explanation.   I wasn't sure about these types of variables, but this makes perfect sense.   Much appreciated.