Dirwalker - Simplistic and Ergonomic Directory Browser

[eoredson]

Thanks for reporting this issue.
Is this 32bit compiler, and what version of QB64 is in use?

QB64PE 32-bit v3.5.0

[Sanmayce]

Thanks to @eoredson here comes the 32bit variant as well:

  Dirwalker_r7++.zip (Size: 6.25 MB / Downloads: 21)

I did manual casting, in three places, for ctime(), now we have 32bit and 64bit code running. An 32bit executable is included as well.

Also, the wildcard search is boosted 4x, simply by using 128bit SIMD for converting lowercase to uppercase and for converting NULL to SPC, the latter being problematic in Tcheburaschka - since it uses ASCIIZ strings.

Now, the poor (due to invoking wildcard matcher for each line) search performance is less poor, about 9,100KB/s or 150,000 lines-per-second (of fixed 64bytes width, with spanning hits on 2 lines):

Releasing 'Enter' over Wikipedia:
   

Pressing 'Ctrl+End', showing 1.4+ billion 64bytes long lines:
   

Releasing '/' in order to search all occurences of 'sanmayce':
   

[eoredson]

Now crashes with this error:

Code: (Select All)

internal\c\c_compiler\bin\c++.exe  -w -std=gnu++11 -DGLEW_STATIC -DFREEGLUT_STATIC -Iinternal\c\libqb/include -Iinternal\c/parts/core/src/ -Iinternal\c/parts/core/glew/include/ -DDEPENDENCY_NO_SOCKETS -DDEPENDENCY_NO_PRINTER -DDEPENDENCY_ICON -DDEPENDENCY_NO_SCREENIMAGE -DDEPENDENCY_LOADFONT -DDEPENDENCY_AUDIO_MINIAUDIO internal\c/qbx.cpp -c -o internal\c/qbx.o
internal\c\c_compiler\bin\windres.exe -i internal\temp\icon.rc -o internal\temp\icon.o
In file included from C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/xmmintrin.h:1324,
                from C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/immintrin.h:31,
                from C:/qb64pe/Dirwalker_r7++/manatarka.h:58,
                from internal\c/../temp/regsf.txt:47,
                from internal\c/qbx.cpp:1164:
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h: In function 'void UCASE_XMM(uint64_t*, uint64_t*, int64_t)':
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:654:1: error: inlining failed in call to 'always_inline' '__m128i _mm_set1_epi8(char)': target specific option mismatch
  654 | _mm_set1_epi8 (char __A)
      | ^~~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:74:31: note: called from here
  74 |        mask32 = _mm_set1_epi8( 32 );
      |                  ~~~~~~~~~~~~~^~~~~~
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:654:1: error: inlining failed in call to 'always_inline' '__m128i _mm_set1_epi8(char)': target specific option mismatch
  654 | _mm_set1_epi8 (char __A)
      | ^~~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:73:30: note: called from here
  73 |        maskz = _mm_set1_epi8( 'z' );
      |                ~~~~~~~~~~~~~^~~~~~~
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:654:1: error: inlining failed in call to 'always_inline' '__m128i _mm_set1_epi8(char)': target specific option mismatch
  654 | _mm_set1_epi8 (char __A)
      | ^~~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:72:30: note: called from here
  72 |        maska = _mm_set1_epi8( 'a' );
      |                ~~~~~~~~~~~~~^~~~~~~
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:737:1: error: inlining failed in call to 'always_inline' 'void _mm_storeu_si128(__m128i_u*, __m128i)': target specific option mismatch
  737 | _mm_storeu_si128 (__m128i_u *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:85:33: note: called from here
  85 |                _mm_storeu_si128( ( __m128i * )&QWORDSd[ i ], r0 );
      |                ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:1315:1: error: inlining failed in call to 'always_inline' '__m128i _mm_andnot_si128(__m128i, __m128i)': target specific option mismatch
1315 | _mm_andnot_si128 (__m128i __A, __m128i __B)
      | ^~~~~~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:84:38: note: called from here
  84 |                r0 = _mm_andnot_si128( _mm_andnot_si128( maskaz, mask32 ), r0 );
      |                      ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:1315:1: error: inlining failed in call to 'always_inline' '__m128i _mm_andnot_si128(__m128i, __m128i)': target specific option mismatch
1315 | _mm_andnot_si128 (__m128i __A, __m128i __B)
      | ^~~~~~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:84:38: note: called from here
  84 |                r0 = _mm_andnot_si128( _mm_andnot_si128( maskaz, mask32 ), r0 );
      |                      ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:1321:1: error: inlining failed in call to 'always_inline' '__m128i _mm_or_si128(__m128i, __m128i)': target specific option mismatch
1321 | _mm_or_si128 (__m128i __A, __m128i __B)
      | ^~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:81:38: note: called from here
  81 |                maskaz = _mm_or_si128( _mm_cmplt_epi8( r0, maska ), _mm_cmpgt_epi8( r0, maskz ) );
      |                          ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:1351:1: error: inlining failed in call to 'always_inline' '__m128i _mm_cmplt_epi8(__m128i, __m128i)': target specific option mismatch
1351 | _mm_cmplt_epi8 (__m128i __A, __m128i __B)
      | ^~~~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:81:38: note: called from here
  81 |                maskaz = _mm_or_si128( _mm_cmplt_epi8( r0, maska ), _mm_cmpgt_epi8( r0, maskz ) );
      |                          ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:1369:1: error: inlining failed in call to 'always_inline' '__m128i _mm_cmpgt_epi8(__m128i, __m128i)': target specific option mismatch
1369 | _mm_cmpgt_epi8 (__m128i __A, __m128i __B)
      | ^~~~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:81:38: note: called from here
  81 |                maskaz = _mm_or_si128( _mm_cmplt_epi8( r0, maska ), _mm_cmpgt_epi8( r0, maskz ) );
      |                          ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:/qb64pe/internal/c/c_compiler/lib/gcc/i686-w64-mingw32/12.2.0/include/emmintrin.h:701:1: error: inlining failed in call to 'always_inline' '__m128i _mm_loadu_si128(const __m128i_u*)': target specific option mismatch
  701 | _mm_loadu_si128 (__m128i_u const *__P)
      | ^~~~~~~~~~~~~~~
C:/qb64pe/Dirwalker_r7++/manatarka.h:78:37: note: called from here
  78 |                r0 = _mm_loadu_si128( ( __m128i * )&QWORDSs[ i ] );
      |                      ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mingw32-make: *** [Makefile:417: internal\c/qbx.o] Error 1

[eoredson]

Also got this error when starting:

[Sanmayce]

@eoredson

These errors are due to missing compile options, the line 3 is:

Code: (Select All)

'Note1: Compile with '-O3 -ffast-math -mavx -maes'.

I use QB64 v2.0.2, and the way I compile is described in first lines of Dirwalker.bas, however I hear the new QB64PE now makes compilation much easier by allowing supplying above compile options from the IDE.

Code: (Select All)

745 PoolToHousePathS_MAXsize&& = 1&& ' make it QB64 32bit friendly by default
746 If Line4$ <> "" Then
747    PoolToHousePathS_MAXsize&& = Val(Line4$)
748 End If
749 Print Time$; " Allocating"; Str$(PoolToHousePathS_MAXsize&&); "GB for minimum"; Str$(PoolToHousePathS_MAXsize&&); "GB/4096 =~"; Str$(PoolToHousePathS_MAXsize&& * 1024 / 4096); "+ Million MaxLength filepaths (or more likely"; Str$(PoolToHousePathS_MAXsize&&); "GB/256 =~"; Str$(PoolToHousePathS_MAXsize&& * 1024 / 256); "+ Million)..."
750 PoolToHousePathS_MAXsize&& = PoolToHousePathS_MAXsize&& * 1024 * 1024 * 1024
751 MhandleXMM = _MemNew(PoolToHousePathS_MAXsize&&)

As you see the line where it breaks (751) is the allocating of memory, why so, because the 1GB is overridden by the 'Dirwalker.ini' value 6 - it means 6GB:

Code: (Select All)

[empty line]
224
60
6
Above 4 lines being 'MAX' for 1920x1080, 8x16 Toshiba, 8GB RAM

If your computer is an old one (on top of that running 32bit OS), then try:

Code: (Select All)

v
208
54
1
Above 4 lines being 'MAX' for 1366x768, Japanese DOSV 6x12, 4GB RAM

[eoredson]

Ok, I got the compiler switches in Alt-Options and reduced the GB Ram setting to 1 and it does compile/starts..

btw: when I click on a file and get the full screen contents display how do I back out to dir list? I tried <esc> and <backspace>..

Erik.

nice code!

[Sanmayce]

btw: when I click on a file and get the full screen contents display how do I back out to dir list? I tried <esc> and <backspace>..

You are right, the intuitive 'Esc', 'Backspace' are not yet implemented, I'm sorry. Currently, only Double_LeftShift toggles between Content/FileTree/FileCurrent modes.
Sometimes, as in this case, the functionality is not as it should - not intuitive enough, yet, implementing ideas was the priority - not the seamless tailoring. Will make 'Esc', 'Backspace' returning to the FileCurrent mode, in next revision...

Basically, we have 4 modes, at the moment:
- FileCurrent mode - the "dir" like;
- DirTree mode - listing all the folders, recursively;
- FileTree mode - listing all the files, recursively;
- FileContent mode - all the 64 bytes chunks of the file.

Many things are to be added...

Please, share what features/enhancements you wanna have/see in the future.

[eoredson]

Please, share what features/enhancements you wanna have/see in the future.

Double-Left Click should open that directory or load a file.

<delete> could delete a file or recursively delete a directory w/ dialog box to prompt to do so.

After running your program the \qb64pe\internal\temp\recent.bin is getting deleted..

Erik.

Here is one for you..

When displaying a file contents there appear to be 2 panes, the left one in ascii and the right one in hex.
This reminds me of a hexeditor!? Maybe you could allow the bytes to be edited and then the modified file to be written...

[Sanmayce]

Here comes the most stable revision, the only one I recommend:

  Dirwalker_r7+++.zip (Size: 6.26 MB / Downloads: 22)

A buggy usage (when filtering with /) of Wildcard function was fixed, simply forgot to update the old one.
Now, returning to the DIR mode is possible with 'Backspace', also, few small tunings were made.

> Double-Left Click should open that directory or load a file.

Ugh, it is occupied, Double-Left Click is equivalent to PgUp.
These functions are available via Third Mouse Button, Enter, [E] from the ScrollRibbon, 'Enter - enters the DIRECTORY' button at the FATTY panel.

> <delete> could delete a file or recursively delete a directory w/ dialog box to prompt to do so.

Will keep it in my TODO list, however first I wanna cover the READ-ONLY functionality, modifying/deleting is not the focus of Dirwalker.

> This reminds me of a hexeditor!? Maybe you could allow the bytes to be edited and then the modified file to be written...

Sure, sometimes modifying some huge file (bypassing the File Editors) is needed, will implement it after finishing browsing the LINE-BY-LINE mode, the physical lines, not logical as at the moment.

Wanna make a Smooth_Scroll_Mode in which the lines to be scrolled with one pixel granularity (not one character), as a first step, I introduced a flag for TEXT/GRAPHIC toggling:

Code: (Select All)

'Note2: Try setting 'GraphicOrTextFlag' (see below) to 1 to go fully graphical.

Many many things are needed, hope in the future to implement them one by one.

[TDarcos]

I am very impressed by this program! I was thinking of doing something similar for handling archives and nested subdirectories. I'll have to look at how this works. I am very impressed at how fast the listing process is or directory scan. You've really done something special.

In case no one else has said so, thank you for having created this, and for making it publicly available.

If I may offer a few suggestions, one about the program:

1.  Have F1 open a pop-up window showing what keys do what things, i.e. a cheat sheet. If F1 is in use, pick a different button and have the program say which one it is. 
   Now, I'd like to offer some workflow suggestions. If you're already doing/using these things, please excuse me.
   
2. If you aren't already using source control, obtain a copy of GIT / GIT for Windows (if using it) or both.
   
3. Use git to create an empty local repository; you can rename your current work directory, then give the repository that name, or do the opposite, rename after copying
   
4. Copy your files to this repository
   
5.  'GIT add' all files in this repository
   
6. From now on, after every change where the program isn't broken, do a 'GIT commit'.
   
7. Get an account on Github, create a repository for this project, then copy it over. This can be done at no charge. Repositories may be private. Do frequent 'GIT push' of commits in order to have an external copy of your work.
   
   The reason for doing all this is Git saves every change once committed, and if you make a mistake, you can roll back changes to any earlier "known good version." I couldn't even guess how many times it has saved my bacon. Having a copy of your work on Github protects you if something goes wrong.
   Since I didn't see any .git files in the zip file or the work directory once extracted, I thought you might not be using it. Source control makes work easier and less stressful.
   
8. (If using Windows) Get a copy of Winmerge. It allows you to scan two directories against each other and scan files in those directories and show differences. It can also be used to make changes. The first time you try to figure out what you've just changed against another file, you'll appreciate this tool.
   

If I think of other suggestions, I'll offer them, if you don't mind.