DAY 007: _PRESERVE

[SMcNeill]

This is another one of those commands that, unfortunately, we see people using wrong all the time.  Let's see if we can help showcase a bit of a better way to use it than what many are doing for their standard practice. 

First, let's start with the same basic questions as every day.

What is it?  _PRESERVE is a command which is used when REDIM-ing an array and one wishes to preserve the existing data within it.  It mainly only works properly with single dimensional arrays, so I'm not going to talk about the issues it faces with multi-dimensional arrays here.  If someone is truly curious about those problems, go watch my movie-length video on REDIM and how it interacts with _MEM commands and memory.  https://staging.qb64phoenix.com/showthre...707#pid707

How's it used?  Place your _PRESERVE statement after REDIM and then set your array to the new size that you need it to be.  The syntax here is rather simple for folks to grasp. 

So how's it used wrongly??  Let me share a BAD example of the command to begin with -- we see this type of code all the time:

Code: (Select All)

OPEN "myfile.txt" FOR INPUT AS #1
DO UNTIL EOF(1)
    count = count + 1
    IF count > UBOUND(array) THEN REDIM _PRESERVE array(count) AS STRING
    LINE INPUT array(count)
LOOP

A file is opened where one might not know the length of the file contents, but they want to read each line into an array.  How do many people do this?  They end up reading the file one line at a time and growing the array with REDIM and _PRESERVE until it's large enough to hold all their data.

As I've said many times regarding this practice:  Yuck!  Yucky!  YUCK!!

Let me showcase why this is a bad practice:

Code: (Select All)

Limit = 5000000 '5,000,000  -- only a limit of five million

count = 0: t## = Timer
ReDim Array(0) As Long 'a standard redimable array, with a starting index of zero
Do
    count = count + 1
    If count > UBound(Array) Then ReDim _Preserve Array(count) As Long
Loop Until count >= Limit
Print Using " ##.##### seconds to redim and preserve our array as we go one increment at a time."; Timer - t##

Print "Phew!  That took a wee bit to just redim and count, now didn't it?"
Print "IS there a better way??"
Print
Print "How about:"

count = 0: t## = Timer
ReDim _Preserve Array(100000) As Long 'start with an arbitary large number to begin with...

Do
    count = count + 1
    If count > UBound(Array) Then ReDim _Preserve Array(count + 100000) As Long 'add a large number of elements all at once, instead of 1 at a time
Loop Until count >= Limit
ReDim _Preserve Array(count) As Long 'resize the array to the max size AFTER the loop is finished.
Print Using " ##.##### seconds to redim and preserve our array as we go in large chunks."; Timer - t##

Now there's two REDIM and _PRESERVE loops inside the above code.  Let's explain them both a little:

The first loop does nothing but counts from 0 to 5,000,000 and redims and preserves as we see above -- WITHOUT any calls to load data from the drive or anything else.  All I want to show here is *how long* it takes for REDIM and PRESERVE to resize our array in this manner.

Our second loop works a wee bit different than the first -- instead of resizing itself one element at a time, it resizes in *large chunks*.  At the end of our loops, BOTH arrays are the exact same size -- but there's a slight difference in the speed and performance between the two routines as show in the image below.



12.2 seconds for the first loop to run and resize.  0.1 seconds for the second loop.   AND REMEMBER -- This isn't actually reading data or assigning data to the array, or anything else with these loops.  This is the simple speed difference in how long it takes them both to resize and count to five million!

Use REDIM and _PRESERVE properly in your code, and you may be able to cut down load/processing times from multiple seconds/minutes to just fractions of a second.  _PRESERVE is an important command for any programmer's tool box, but it's definitely one which needs to be used properly so that it doesn't bog down your programs unnecessarily.