MemPrint and MemInput

[SMcNeill]

Updated these to help showcase a bit of how useful they can be for us.

Code: (Select All)

Type Mem_File_Type
    inUse As Integer
    EOF_Marker As _Offset
    Current_Pos As _Offset
    Content As _MEM
End Type

Dim Shared MemFile(1 To 100) As Mem_File_Type
'BI HEADER INFO BEFORE THIS








'first, let's showcase how to print and input some information to and from memory
handle = MemFileOpen '                            This is a combination FREEFILE + OPEN statement
MemPrint handle, "Hello World", 1 '              PRINT #filehandle, whatever$ + CRLF
MemPrint handle, "My name is Steve", 1 '          PRINT #filehandle, whatever$ + CRLF
MemSeek handle, 0 '                              SEEK #filehandle, byte

Do Until MemEOF(handle) '                        DO UNTIL EOF(filehandle)
    MemLineInput handle, temp$ '                    LINE INPUT #filehandle, temp$
    Print temp$ '                                    PRINT temp$
Loop '                                            LOOP
MemFileClose handle '                            CLOSE filehandle
'I hope the above with the comments help to highlight how exactly similar the two syntaxes are here.
'With these mem routines, we're basically just using mem *exactly* as we'd do basic file access with PRINT and LINE INPUT


'And let's showcase a bit more of how this works, why don't we.
Dim As String wordlist(466544), wordlist2(466544), wordlist3(466544) 'arrays to hold the data

'MEM FILE INPUT
handle = MemFileLoad("466544 Word List.txt", 0) 'load a file directly into memory, and it's not compressed
t## = Timer '                                                        timer to see how long we take loading this data
Do Until MemEOF(handle) '                        Hopefully these lines will be intuitive enough.
    count = count + 1 '                          Especially when compared to the notes above
    MemLineInput handle, wordlist(count) '        and the preceeding lines after
Loop
Print count; Using " words loaded into memory from file, in ##.#### seconds."; Timer - t##
MemFileClose handle


'OPEN FILE FOR INPUT
Open "466544 Word List.txt" For Input As #1
t## = Timer
Do Until EOF(1)
    count2 = count2 + 1
    Line Input #1, wordlist2(count2)
Loop
Print count2; Using " words loaded from file OPEN FOR INPUT, in ##.#### seconds."; Timer - t##
Close handle

'OPEN FILE FOR BINARY
Open "466544 Word List.txt" For Binary As #1
t## = Timer
Do Until EOF(1)
    count3 = count3 + 1
    Line Input #1, wordlist3(count3)
Loop
Print count3; Using " words loaded from file OPEN FOR BINARY, in ##.#### seconds."; Timer - t##
Close handle

'and let's compare contents to be safe
For i = 1 To count
    If wordlist(i) <> wordlist2(i) Then Print "Wordlist does not match Wordlist2": failed = -1
    If wordlist(i) <> wordlist3(i) Then Print "Wordlist does not match Wordlist3": failed = -1
Next

If failed Then
    Print "Lists do not match"
Else
    Print "Lists match each other perfectly"
End If





'BM FOOTER AFTER THIS

Function MemFileLoad (file$, compressed)
    'Error codes for MemFileLoad
    '1: No mem files available.  (All 100 are in use!  Free some to use more!)
    For i = 1 To 100
        If MemFile(i).inUse = 0 Then Exit For
    Next
    If i > 100 Then MemFileLoad = 0: Exit Function 'can't open any more memfiles!
    If _FileExists(file$) = 0 Then Error 53: Exit Function 'file not found


    MemFileLoad = i
    temphandle = FreeFile
    Open file$ For Binary As #temphandle
    temp$ = Space$(LOF(temphandle))
    Get temphandle, 1, temp$
    Close temphandle
    If compressed Then temp$ = _Deflate$(temp$)
    length = Len(temp$)
    MemFile(i).Content = _MemNew(length)
    $Checking:Off
    _MemPut MemFile(i).Content, MemFile(i).Content.OFFSET, temp$
    $Checking:On
    MemFile(i).inUse = -1 'TRUE
    MemFile(i).EOF_Marker = length - 1 'the end of the file is the length of the file to begin with
    MemFile(i).Current_Pos = 0 'and we're at the start of our nothing in the file
End Function


Sub MemFileClose (memfile)
    If memfile < 1 Or memfile > 100 Then Error 5: Exit Sub 'ILLEGAL FUNCTION CALL
    MemFile(memfile).inUse = 0 'no longer in sue
    MemFile(memfile).EOF_Marker = 0 'nothing is written in the file to begin with
    MemFile(memfile).Current_Pos = 0 'and we're at the start of our nothing in the file
    _MemFree MemFile(memfile).Content 'free the memory we were using
End Sub


Function MemFileOpen
    'Error codes for MemFileOpen
    '1: No mem files available.  (All 100 are in use!  Free some to use more!)
    For i = 1 To 100
        If MemFile(i).inUse = 0 Then Exit For
    Next
    If i > 100 Then MemFileOpen = 0: Exit Function 'can't open any more memfiles!
    MemFileOpen = i
    MemFile(i).inUse = -1 'TRUE
    MemFile(i).EOF_Marker = 0 'nothing is written in the file to begin with
    MemFile(i).Current_Pos = 0 'and we're at the start of our nothing in the file
    MemFile(i).Content = _MemNew(1000000) '1mb memfile by default
    $Checking:Off
    _MemFill MemFile(i).Content, MemFile(i).Content.OFFSET, MemFile(i).Content.SIZE, 0 As _UNSIGNED _BYTE
    'make certain to blank the file when opening it for the first time so we don't have unwanted characters in it.
    $Checking:On
End Function

Function MemEOF (memfile)
    If MemFile(memfile).inUse = 0 Then Error 53: Exit Function 'File Not Found Error message
    If MemFile(memfile).Current_Pos >= MemFile(memfile).EOF_Marker Then MemEOF = -1
End Function


Sub MemSeek (memfile, position As _Offset)
    If MemFile(memfile).inUse = 0 Then Error 53: Exit Sub 'File Not Found Error message
    If position < 0 Then Error 5: Exit Sub 'Invalid Function Call
    If position > MemFile(memfile).EOF_Marker Then Error 5: Exit Sub 'Invalid Function Call
    MemFile(memfile).Current_Pos = position
End Sub

Sub MemLineInput (memfile, what$)
    'only valid line endings here are CHR$(10), chr$(13), and chr$(13) + chr$(10)

    If MemFile(memfile).inUse = 0 Then Error 53: Exit Sub 'File Not Found Error message

    Dim As _Offset CP, EP, Size, L
    Dim tempM As _MEM, a1 As _Unsigned _Byte
    tempM = MemFile(memfile).Content 'it's just much shorter to type!

    CP = MemFile(memfile).Current_Pos
    EP = MemFile(memfile).EOF_Marker
    If CP >= EP Then Error 62: Exit Sub 'INPUT PAST END OF FILE error
    Size = tempM.SIZE
    Do
        a$ = _MemGet(tempM, tempM.OFFSET + CP, String * 1)
        Select Case a$
            Case Chr$(13)
                _MemGet tempM, tempM.OFFSET + CP + 1, a1
                If a1 = 10 Then CP = CP + 1 'move the Current Pointer past the 2nd character in a windows CRLF ending
                finished = -1
            Case Chr$(10)
                finished = -1
            Case Else
                temp$ = temp$ + a$
        End Select
        CP = CP + 1
        If CP >= EP Then finished = -1
    Loop Until finished
    MemFile(memfile).Current_Pos = CP
    what$ = temp$
End Sub



Sub MemPrint (memfile, what$, EOL_Type As Integer)
    'memfile is the memfile handle to print to
    'what$ is what we want to print
    'EOL_Type is the type of line ending we want after this print statement
    '1: This is a CHR$(10) line ending                  (Linux style line ending)
    '2: This is a CHR$(13) line ending                  (Old Mac style line ending)
    '3: This is a CHR$(13) + CHR$(10) line ending      (Old Windows style line ending)
    '4: This is a COMMA line ending.  Use this if writing continous CSV fields.
    '      (Think PRINT #1, stuff$, <-- see the comma there at the end of the print statement??)


    If MemFile(memfile).inUse = 0 Then Error 53: Exit Sub 'File Not Found Error message

    Dim CRLF As String
    Dim As _Offset CP, EP, Size, L

    Select Case EOL_Type
        Case 1: CRLF = Chr$(10)
        Case 2: CRLF = Chr$(13)
        Case 3: CRLF = Chr$(13) + Chr$(10)
        Case 4: CRLF = ","
    End Select
    CP = MemFile(memfile).Current_Pos
    EP = MemFile(memfile).EOF_Marker
    Size = MemFile(memfile).Content.SIZE
    L = Len(what$) + Len(CRLF)
    If CP + L > Size Then 'we're writing beyond the bounds of our reserved memory!
        Dim tempM As _MEM
        recheck:
        If Size <= 100000000 Then 'resize our memblock (to the limit) to save our data
            tempM = _MemNew(Size * 10)
            _MemCopy MemFile(memfile).Content, MemFile(memfile).Content.OFFSET, Size To tempM, tempM.OFFSET
            _MemFree MemFile(memfile).Content
            MemFile(memfile).Content = tempM
            Size = Size * 10
            GoTo recheck 'just to make certain that our reserved memory is now large enough to hold our data
        Else
            Error 61 'DISK FULL ERROR MESSAGE
            Exit Sub 'I'm coding a hard size limit of 1GB for each memfile opened!
            '        Anything larger than that, and I'm tossing a Disk Full Error
        End If
    End If
    _MemPut MemFile(memfile).Content, MemFile(memfile).Content.OFFSET + CP, what$ + CRLF
    MemFile(memfile).Current_Pos = CP + L
    If CP + L > EP Then MemFile(memfile).EOF_Marker = CP + L
End Sub

As you can see, we now have a MemFileClose routine which we can use to close and free our old mem blocks and file handles.  We also now have a nice little MemFileOpen which we can use to directly load a file into memory so we can work with it with the MemPrint and MemInput functions.

Note that you'll need the word list below so you can run the code and test it.  Just download the attached 7z and extract the contents into your QB64 folder (or the folder where you save this BAS file at).

   

With the image above, you can see where I'm using 3 different methods to load my dictionary from the file and put it into an array so I could use it in a program later.  Even without the use of $CHECKING:OFF and such (which I'll add *after* excessive testing and before I finalize this project), you can see which of the three methods is the fastest on my machine.

Not only does making use of MEM reduce the wear and tear on our hard drives, but it's also simply *faster* to boot!

---

And for those with a discerning eye, you may have noticed that our MemFileOpen holds a parameter for "Compressed"...   That's not really useful at the moment, but once I get the MemFileSave routine added which basically dumps the entire file back to disk all at once, just as MemFileOpen loads it all at once, then it should be a nice feature for us.  With one flip of that toggle, we'll be able to save compressed files to the disk and then load them back and uncompress them just as easily!