Compiler setting for accurate math?

[James D Jarvis]

Why does this happen?   



when this is the code?

Code: (Select All)

Dim x
For x = 1 To .05 Step -.05
    Print x
    _Delay x
Next x

Is there a complier setting to keep this from happening?

[justsomeguy]

You could try for the same effect.

Code: (Select All)

DIM x AS INTEGER
FOR x = 100 TO 5 STEP -5
  PRINT x / 100
  _DELAY x / 100
NEXT x

[bplus]

When you get desperate enough there is Steve's N2S$ and a rounding thing that uses it in my vs GUI.BM that I absolutely needed to get numbers right for Slider Boxes.

[Jack]

hello @James D Jarvis
if you feel like having a bit of fun try the following
in the file internal\c\libqb.cpp search for qbs *qbs_str(float value) {
and change l = sprintf((char *)&qbs_str_buffer, "% .6E", value);
to l = sprintf((char *)&qbs_str_buffer, "% .5G", value);
save and recompile your basic program
I only tested this with your code in first post, don't know how this would behave for other values, one thing for sure is that this will print 1 digit less

[Pete]

Use Jack's decfloat or my string math routine. Computer binary math is troublesome when the results we want are to be in base 10. Before I made a string math routine, I made a rounding routine that worked fine for my accounting apps, although we are talking about maximum transactions in the thousands. With Bideflation, I'd have to test it out to a few more zeros.

Pete

[bplus]

Code: (Select All)

Dim x
For x = 1 To -0.001 Step -.05
    Print Round2$(x, -2)
    _Limit 4
Next x


Function N2S$ (EXP$) 'remove scientific Notation to String (~40 LOC)
    'SMcNeill Jan 7, 2020 ref: https://www.qb64.org/forum/index.php?topic=1555.msg112989#msg112989
    'Last Function in code marked Best Answer (removed debug comments and blank lines added these 2 lines.)
    ReDim t$, sign$, l$, r$, r&&
    ReDim dp As Long, dm As Long, ep As Long, em As Long, check1 As Long, l As Long, i As Long
    t$ = LTrim$(RTrim$(EXP$))
    If Left$(t$, 1) = "-" Or Left$(t$, 1) = "N" Then sign$ = "-": t$ = Mid$(t$, 2)
    dp = InStr(t$, "D+"): dm = InStr(t$, "D-")
    ep = InStr(t$, "E+"): em = InStr(t$, "E-")
    check1 = Sgn(dp) + Sgn(dm) + Sgn(ep) + Sgn(em)
    If check1 < 1 Or check1 > 1 Then N2S = _Trim$(EXP$): Exit Function 'If no scientic notation is found, or if we find more than 1 type, it's not SN!
    Select Case l 'l now tells us where the SN starts at.
        Case Is < dp: l = dp
        Case Is < dm: l = dm
        Case Is < ep: l = ep
        Case Is < em: l = em
    End Select
    l$ = Left$(t$, l - 1) 'The left of the SN
    r$ = Mid$(t$, l + 1): r&& = Val(r$) 'The right of the SN, turned into a workable long
    If InStr(l$, ".") Then 'Location of the decimal, if any
        If r&& > 0 Then
            r&& = r&& - Len(l$) + 2
        Else
            r&& = r&& + 1
        End If
        l$ = Left$(l$, 1) + Mid$(l$, 3)
    End If
    Select Case r&&
        Case 0 'what the heck? We solved it already?
            'l$ = l$
        Case Is < 0
            For i = 1 To -r&&
                l$ = "0" + l$
            Next
            l$ = "." + l$
        Case Else
            For i = 1 To r&&
                l$ = l$ + "0"
            Next
            l$ = l$
    End Select
    N2S$ = sign$ + l$
End Function

Function Round2$ (anyNumber As _Float, dp As Long) ' uses N2S$
    ' 5 and up at decimal place dp+1 > +1 at decimal place  4 and down  > +0 at dp

    '2 1 0.-1 -2 -3 -4 ...  pick dp like this for this Round$ Function
    Dim sn$, dot, predot, postdot, rtn$

    sn$ = N2S$(Str$(anyNumber + .5 * 10 ^ dp)) 'get rid of sci notation, steve trims it so next find dot
    dot = InStr(sn$, ".")
    If dot Then
        predot = dot - 1
        postdot = Len(sn$) - (dot + 1)
    Else
        predot = Len(sn$)
        postdot = 0
    End If
    ' xxx.yyyyyy  dp = -2
    '      ^ dp
    If dp >= 0 Then
        rtn$ = Mid$(sn$, 1, predot - dp) + String$(dp, "0")
    Else
        rtn$ = Mid$(sn$, 1, predot) + "." + Mid$(sn$, dot + 1, -dp)
    End If
    If rtn$ = "" Then Round2$ = "0" Else Round2$ = rtn$
End Function

A few LOC less than string math and still QB64

[Pete]

As best as my memory serves me these days, I think this is or is close to the ledger formula I made 30 years ago...

I put is Steve's side by side to compare result.

Edit: I found some unacceptable results in both my function and Steve's on an expanded level. I'll post an example in a bit...

Code: (Select All)

$CONSOLE:ONLY
DIM x
FOR x = 1 TO -0.001 STEP -.01
    PRINT x;: LOCATE , 25: PRINT "Pete = "; pete$(x);: LOCATE , 45: PRINT "Steve = "; Round2$(x, -2)
NEXT x
'-----------------------------------------------------------------------
FUNCTION pete$ (x)
    tmp1$ = ".00"
    tmp2$ = LTRIM$(STR$(INT(x * 100 + .5) / 100))
    IF INSTR(tmp2$, ".") THEN
        MID$(tmp1$, 2, LEN(tmp2$)) = MID$(tmp2$, INSTR(tmp2$, ".") + 1)
        tmpint$ = MID$(tmp2$, 1, INSTR(tmp2$, ".") - 1)
    ELSE
        tmpint$ = tmp2$
    END IF
    pete$ = tmpint$ + tmp1$
END FUNCTION
'-----------------------------------------------------------------------
FUNCTION N2S$ (EXP$) 'remove scientific Notation to String (~40 LOC)
    'SMcNeill Jan 7, 2020 ref: https://www.qb64.org/forum/index.php?topic=1555.msg112989#msg112989
    'Last Function in code marked Best Answer (removed debug comments and blank lines added these 2 lines.)
    REDIM t$, sign$, l$, r$, r&&
    REDIM dp AS LONG, dm AS LONG, ep AS LONG, em AS LONG, check1 AS LONG, l AS LONG, i AS LONG
    t$ = LTRIM$(RTRIM$(EXP$))
    IF LEFT$(t$, 1) = "-" OR LEFT$(t$, 1) = "N" THEN sign$ = "-": t$ = MID$(t$, 2)
    dp = INSTR(t$, "D+"): dm = INSTR(t$, "D-")
    ep = INSTR(t$, "E+"): em = INSTR(t$, "E-")
    check1 = SGN(dp) + SGN(dm) + SGN(ep) + SGN(em)
    IF check1 < 1 OR check1 > 1 THEN N2S = _TRIM$(EXP$): EXIT FUNCTION 'If no scientic notation is found, or if we find more than 1 type, it's not SN!
    SELECT CASE l 'l now tells us where the SN starts at.
        CASE IS < dp: l = dp
        CASE IS < dm: l = dm
        CASE IS < ep: l = ep
        CASE IS < em: l = em
    END SELECT
    l$ = LEFT$(t$, l - 1) 'The left of the SN
    r$ = MID$(t$, l + 1): r&& = VAL(r$) 'The right of the SN, turned into a workable long
    IF INSTR(l$, ".") THEN 'Location of the decimal, if any
        IF r&& > 0 THEN
            r&& = r&& - LEN(l$) + 2
        ELSE
            r&& = r&& + 1
        END IF
        l$ = LEFT$(l$, 1) + MID$(l$, 3)
    END IF
    SELECT CASE r&&
        CASE 0 'what the heck? We solved it already?
            'l$ = l$
        CASE IS < 0
            FOR i = 1 TO -r&&
                l$ = "0" + l$
            NEXT
            l$ = "." + l$
        CASE ELSE
            FOR i = 1 TO r&&
                l$ = l$ + "0"
            NEXT
            l$ = l$
    END SELECT
    N2S$ = sign$ + l$
END FUNCTION

FUNCTION Round2$ (anyNumber AS _FLOAT, dp AS LONG) ' uses N2S$
    ' 5 and up at decimal place dp+1 > +1 at decimal place  4 and down  > +0 at dp

    '2 1 0.-1 -2 -3 -4 ...  pick dp like this for this Round$ Function
    DIM sn$, dot, predot, postdot, rtn$

    sn$ = N2S$(STR$(anyNumber + .5 * 10 ^ dp)) 'get rid of sci notation, steve trims it so next find dot
    dot = INSTR(sn$, ".")
    IF dot THEN
        predot = dot - 1
        postdot = LEN(sn$) - (dot + 1)
    ELSE
        predot = LEN(sn$)
        postdot = 0
    END IF
    ' xxx.yyyyyy  dp = -2
    '      ^ dp
    IF dp >= 0 THEN
        rtn$ = MID$(sn$, 1, predot - dp) + STRING$(dp, "0")
    ELSE
        rtn$ = MID$(sn$, 1, predot) + "." + MID$(sn$, dot + 1, -dp)
    END IF
    IF rtn$ = "" THEN Round2$ = "0" ELSE Round2$ = rtn$
END FUNCTION

Here's the glitch I found in Steve's function results Mark posted, using an extended testing range. Run it and it will beep and pause at the first error. Click the left mouse button to continue. It will beep and pause one more time. Note in the first case a sequence is skipped and in the second case a number is duplicated. I experienced similar errors at slightly different places in my function results.

Code: (Select All)

DIM x
cnt = 10000
FOR x = 100 TO -0.001 STEP -.01
    a$ = Round2$(x, -2)
    PRINT cnt, x;: LOCATE , 30: PRINT "Steve = "; a$
    REM IF cnt <> VAL(MID$(a$, 1, INSTR(a$, ".") - 1) + MID$(a$, INSTR(a$, ".") + 1)) THEN DO: WHILE _MOUSEINPUT: WEND: LOOP UNTIL _MOUSEBUTTON(1): _DELAY .1
    IF LEN(olda$) AND ABS(VAL(a$) - VAL(olda$)) <> .01 THEN BEEP: DO: WHILE _MOUSEINPUT: WEND: LOOP UNTIL _MOUSEBUTTON(1): _DELAY .1
    cnt = cnt - 1
    olda$ = a$
NEXT

FUNCTION N2S$ (EXP$) 'remove scientific Notation to String (~40 LOC)
    'SMcNeill Jan 7, 2020 ref: https://www.qb64.org/forum/index.php?topic=1555.msg112989#msg112989
    'Last Function in code marked Best Answer (removed debug comments and blank lines added these 2 lines.)
    REDIM t$, sign$, l$, r$, r&&
    REDIM dp AS LONG, dm AS LONG, ep AS LONG, em AS LONG, check1 AS LONG, l AS LONG, i AS LONG
    t$ = LTRIM$(RTRIM$(EXP$))
    IF LEFT$(t$, 1) = "-" OR LEFT$(t$, 1) = "N" THEN sign$ = "-": t$ = MID$(t$, 2)
    dp = INSTR(t$, "D+"): dm = INSTR(t$, "D-")
    ep = INSTR(t$, "E+"): em = INSTR(t$, "E-")
    check1 = SGN(dp) + SGN(dm) + SGN(ep) + SGN(em)
    IF check1 < 1 OR check1 > 1 THEN N2S = _TRIM$(EXP$): EXIT FUNCTION 'If no scientic notation is found, or if we find more than 1 type, it's not SN!
    SELECT CASE l 'l now tells us where the SN starts at.
        CASE IS < dp: l = dp
        CASE IS < dm: l = dm
        CASE IS < ep: l = ep
        CASE IS < em: l = em
    END SELECT
    l$ = LEFT$(t$, l - 1) 'The left of the SN
    r$ = MID$(t$, l + 1): r&& = VAL(r$) 'The right of the SN, turned into a workable long
    IF INSTR(l$, ".") THEN 'Location of the decimal, if any
        IF r&& > 0 THEN
            r&& = r&& - LEN(l$) + 2
        ELSE
            r&& = r&& + 1
        END IF
        l$ = LEFT$(l$, 1) + MID$(l$, 3)
    END IF
    SELECT CASE r&&
        CASE 0 'what the heck? We solved it already?
            'l$ = l$
        CASE IS < 0
            FOR i = 1 TO -r&&
                l$ = "0" + l$
            NEXT
            l$ = "." + l$
        CASE ELSE
            FOR i = 1 TO r&&
                l$ = l$ + "0"
            NEXT
            l$ = l$
    END SELECT
    N2S$ = sign$ + l$
END FUNCTION

FUNCTION Round2$ (anyNumber AS _FLOAT, dp AS LONG) ' uses N2S$
    ' 5 and up at decimal place dp+1 > +1 at decimal place  4 and down  > +0 at dp

    '2 1 0.-1 -2 -3 -4 ...  pick dp like this for this Round$ Function
    DIM sn$, dot, predot, postdot, rtn$

    sn$ = N2S$(STR$(anyNumber + .5 * 10 ^ dp)) 'get rid of sci notation, steve trims it so next find dot
    dot = INSTR(sn$, ".")
    IF dot THEN
        predot = dot - 1
        postdot = LEN(sn$) - (dot + 1)
    ELSE
        predot = LEN(sn$)
        postdot = 0
    END IF
    ' xxx.yyyyyy  dp = -2
    '      ^ dp
    IF dp >= 0 THEN
        rtn$ = MID$(sn$, 1, predot - dp) + STRING$(dp, "0")
    ELSE
        rtn$ = MID$(sn$, 1, predot) + "." + MID$(sn$, dot + 1, -dp)
    END IF
    IF rtn$ = "" THEN Round2$ = "0" ELSE Round2$ = rtn$
END FUNCTION

Now this could easily be a case of apples and oranges. If Steve's function is for rounding a single number instead of for use in adjusting the output sequence generated in a decimal loop, it isn't the right tool for this particular task.

I'll wait until some more eyes get into this. I've pulled an all-nighter (no, not 'highlighter' you dumb ass spell check...) working on some code and none code related stuff, and for all I know I'm imagining being on the forum typing this post. That's right, sleep coding again!

Pete

[bplus]

SOB code has me swearing, every time I think I have some accuracy in display settled, someone comes up with something to shake my faith.

@Pete 

Took me a long time to figure out what was going on, not pleasantly!

Here it is!

Code: (Select All)

For x = 100 To -.001 Step -.01

You (we) are (were) expecting x of a single type in the FOR loop to be better behaved than any other single type!

I have no idea why it decides not to round up at the 2 places in 10,000 numbers but look at those freak'n x values wandering as much as .005 off the expected value!

I did an equivalent test like yours dropping integers down: For cnt = 10000 to 1 step -1
only setting x = cnt/100 and x doesn't wander nearly as much as .005 and so roundDP$ returns expected results.

BTW Round2$ was old code that failed with negative numbers.

Code: (Select All)

$Console:Only
''test DP$ 2022-10-14  another Round Helper
'For n = 1 To 10000
'    If n = 7658 Then Beep: Flag = -1
'    a = n / 100
'    r$ = roundDP$(a, 2)
'    Print n, a, r$
'    If n Mod 20 = 0 Then
'        If Flag Then Sleep
'        Cls
'    End If
'Next


'Dim x ' ref Pete's test    https://staging.qb64phoenix.com/showthread.php?tid=1209&pid=10895#pid10895
'cnt = 10000
'For x = 100 To -.001 Step -.01
'    a$ = roundDP$(x, 2)
'    Print cnt, x;: Locate , 30: Print "Steve = "; a$
'    'Rem IF cnt <> VAL(MID$(a$, 1, INSTR(a$, ".") - 1) + MID$(a$, INSTR(a$, ".") + 1)) THEN DO: WHILE _MOUSEINPUT: WEND: LOOP UNTIL _MOUSEBUTTON(1): _DELAY .1
'    'If Len(olda$) And Abs(Val(a$) - Val(olda$)) <> .01 Then Beep: Do: While _MouseInput: Wend: Loop Until _MouseButton(1): _Delay .1
'    If Len(olda$) And Abs(Val(a$) - Val(olda$)) <> .01 Then Beep: Sleep ' <<< b= mod this is less confusing than above
'    cnt = cnt - 1
'    olda$ = a$

'    'If cnt Mod 10 = 0 Then ' check screens
'    '    Sleep
'    '    Cls
'    'End If

'Next

Dim cnt As Long, x As Single
For cnt = 10000 To 5000 Step -1
    x = cnt / 100
    a$ = roundDP$(x, 2)
    Print cnt, x;: Locate , 30: Print "Steve = "; a$
    'Rem IF cnt <> VAL(MID$(a$, 1, INSTR(a$, ".") - 1) + MID$(a$, INSTR(a$, ".") + 1)) THEN DO: WHILE _MOUSEINPUT: WEND: LOOP UNTIL _MOUSEBUTTON(1): _DELAY .1
    'If Len(olda$) And Abs(Val(a$) - Val(olda$)) <> .01 Then Beep: Do: While _MouseInput: Wend: Loop Until _MouseButton(1): _Delay .1
    If Len(olda$) And Abs(Val(a$) - Val(olda$)) <> .01 Then Beep: Sleep ' <<< b= mod this is less confusing than above
    olda$ = a$

    'If cnt Mod 20 = 0 Then   ' check screens
    '    Sleep
    '    Cls
    'End If

Next



Function roundDP$ (num, digits) 'flaw if expontential notation involved.
    Print "roundDP recd this num"; num
    s$ = N2S$(Str$(num + Sgn(num) * (.5 * (10 ^ -digits))))
    dot = InStr(s$, ".")
    If dot Then r$ = Mid$(s$, 1, dot + digits) Else r$ = s$
    roundDP$ = r$
End Function

Function N2S$ (EXP$) 'remove scientific Notation to String (~40 LOC)
    'SMcNeill Jan 7, 2020 ref: https://www.qb64.org/forum/index.php?topic=1555.msg112989#msg112989
    'Last Function in code marked Best Answer (removed debug comments and blank lines added these 2 lines.)
    ReDim t$, sign$, l$, r$, r&&
    ReDim dp As Long, dm As Long, ep As Long, em As Long, check1 As Long, l As Long, i As Long
    t$ = LTrim$(RTrim$(EXP$))
    If Left$(t$, 1) = "-" Or Left$(t$, 1) = "N" Then sign$ = "-": t$ = Mid$(t$, 2)
    dp = InStr(t$, "D+"): dm = InStr(t$, "D-")
    ep = InStr(t$, "E+"): em = InStr(t$, "E-")
    check1 = Sgn(dp) + Sgn(dm) + Sgn(ep) + Sgn(em)
    If check1 < 1 Or check1 > 1 Then N2S = _Trim$(EXP$): Exit Function 'If no scientic notation is found, or if we find more than 1 type, it's not SN!
    Select Case l 'l now tells us where the SN starts at.
        Case Is < dp: l = dp
        Case Is < dm: l = dm
        Case Is < ep: l = ep
        Case Is < em: l = em
    End Select
    l$ = Left$(t$, l - 1) 'The left of the SN
    r$ = Mid$(t$, l + 1): r&& = Val(r$) 'The right of the SN, turned into a workable long
    If InStr(l$, ".") Then 'Location of the decimal, if any
        If r&& > 0 Then
            r&& = r&& - Len(l$) + 2
        Else
            r&& = r&& + 1
        End If
        l$ = Left$(l$, 1) + Mid$(l$, 3)
    End If
    Select Case r&&
        Case 0 'what the heck? We solved it already?
            'l$ = l$
        Case Is < 0
            For i = 1 To -r&&
                l$ = "0" + l$
            Next
            l$ = "." + l$
        Case Else
            For i = 1 To r&&
                l$ = l$ + "0"
            Next
            l$ = l$
    End Select
    N2S$ = sign$ + l$
End Function

I am glad I did do this because I am updating GUI for new dialogs in v3.4.1 and I was using the buggy Round2$ code that failed with neg numbers.

BTW I get something short of 5000 lines in Console, good to know.

[bplus]

Why does this happen?   



when this is the code?

Code: (Select All)

Dim x
For x = 1 To .05 Step -.05
    Print x
    _Delay x
Next x

Is there a complier setting to keep this from happening?

The lesson is don't trust a single index in a FOR loop with < 1 steps. It gets off track by as much as half the step value.
Here is equivalent that works better:

Code: (Select All)

Dim x
For x = 100 To 5 Step -5
    i = i + 1: Print i, x / 100
    _Delay x / 100 ' <<< why ??  accuracy .001
Next x

[Pete]

Well don't tear your hair out over it, but if you do, save some for me. There isn't much going on under that big ol' hat of mine. Shut up Steve, I meant hair-wise!

Pete