the input file is now read via mORMot's type 'TMemoryMap', the hash code is now computed via mORMot's crc32c-function, several small improvements

Author: hg747

entries/hgrosser/README.md

@@ -3,42 +3,33 @@
 **1 billion row Challenge entry**
 
 ## Version
-Version 2.00 (first version with threads)
+Version 2.10 (version with threads, does NOT yet work for 400 stations or 5 billion rows)
 
 ## How to compile
 The program was developed with FPC 3.2.2 and Lazarus 2.2.4
 
-1brc.lpi + 1brc.pas = version without threads
+It uses package **mORMot 2** to compile, which you can download from https://github.com/synopse/mORMot2
 
-1brc_th.lpi + 1brc_th.pas = version with threads
+Files to compile: **1brc_th.lpi** and **1brc_th.pas** = this version with threads
 
 ## How to start
 ```
-Usage: <path to input file> <thread count> [<bit-width for hash-list> [buffer size in KB]]
- - thread count: allowed range = [1..32]
+Usage: <path to input file> <thread count> [bit-width for hash-list]
+ - thread count: allowed range = [1..64]
  - bit-width for hash-list: sets the size of the hash list, e.g. '16' => 65536 entries,
-   allowed range = [16..28], Default=18
- - buffer size in KB: allowed range = [1..2,000,000 KB], Default=128 KB
-Example: hgrosser measurements.txt 32 18 128
+   allowed range = [16..28], Default=16
+Example: hgrosser measurements.txt 32 16
 ```
-There are no switches like `-i` etc, only 2..4 values.
+There are no switches like `-i` etc, only 2 or 3 values.
 
-### Optimizing the 2nd command line parameter
-This parameter sets the thread count. With my own old CPU I could only test 1..4 threads. Although I expect 32 threads to be the fastest, I would be very interested to see the results for 1, 4, 8, 16 and 32 threads. Please use for this test the defaults for the 3rd and 4th command line parameters.
-
-### Optimizing the 3rd command line parameter
-In theory the program should run faster with greater bit-widths for the hash-list (because of less collisions). On the computer of Gus - without threads - 18 bits was the fastest. Please try the values from 16 to 20 again and use '32' for the 2nd command line parameter and the default for the 4th command line parameter.
-
-### Optimizing the 4th command line parameter
-After the 2nd and 3rd command line parameters had been optimized, please try with them 64, 96, 128, 192 and 256 KB as the 4th command line parameter. Thanks a lot!
+There is no more need for further optimizing. Please use '32' as 2nd command line parameter and '16' as 3rd command line parameter.
 
 ## How the program works
-The Program works with multi threads.
+The Program works with threads.
 
 To speed things up:
 
-- the input file is read via procedure 'blockread' ...
-- into an AnsiString, so that function 'PosEx' can be used to parse it
+- the input file is read via **mORMot**'s type 'TMemoryMap'
 - to manage the city names, a self made hash-list is used
 - temperatures are stored as integers (multiplied by 10)
 
@@ -50,4 +41,4 @@ To speed things up:
 - Version 1.60: hash-list optimized, some minor improvements, Conditional "noCR" added
 - Version 1.61: Conditional "noCR" constantely enabled => input files must NOT have CR's
 - Version 2.00: 1st version with threads
-
+- Version 2.10: the input file is now read via **mORMot**'s type 'TMemoryMap', the hash code is now computed via **mORMot**'s crc32c-function, several small improvements

entries/hgrosser/src/1brc.lpi renamed to entries/hgrosser/src/1brc_old.lpi

@@ -11,7 +11,7 @@
         <CompatibilityMode Value="True"/>
       </Flags>
       <SessionStorage Value="InIDEConfig"/>
-      <Title Value="1brc"/>
+      <Title Value="1brc_old"/>
       <UseAppBundle Value="False"/>
       <ResourceType Value="res"/>
     </General>
@@ -24,7 +24,7 @@
         <CompilerOptions>
           <Version Value="11"/>
           <Target>
-            <Filename Value="../../../bin/hgrosser"/>
+            <Filename Value="../../../bin/hgrosser_old"/>
           </Target>
           <SearchPaths>
             <IncludeFiles Value="$(ProjOutDir)"/>
@@ -59,7 +59,7 @@
         <CompilerOptions>
           <Version Value="11"/>
           <Target>
-            <Filename Value="../../../bin/hgrosser"/>
+            <Filename Value="../../../bin/hgrosser_old"/>
           </Target>
           <SearchPaths>
             <IncludeFiles Value="$(ProjOutDir)"/>
@@ -104,15 +104,15 @@
     </RunParams>
     <Units Count="1">
       <Unit0>
-        <Filename Value="1brc.pas"/>
+        <Filename Value="1brc_old.pas"/>
         <IsPartOfProject Value="True"/>
       </Unit0>
     </Units>
   </ProjectOptions>
   <CompilerOptions>
     <Version Value="11"/>
     <Target>
-      <Filename Value="../../../bin/hgrosser"/>
+      <Filename Value="../../../bin/hgrosser_old"/>
     </Target>
     <SearchPaths>
       <IncludeFiles Value="$(ProjOutDir)"/>

entries/hgrosser/src/1brc.pas renamed to entries/hgrosser/src/1brc_old.pas

@@ -101,6 +101,11 @@
         </Mode0>
       </Modes>
     </RunParams>
+    <RequiredPackages Count="1">
+      <Item1>
+        <PackageName Value="mormot2"/>
+      </Item1>
+    </RequiredPackages>
     <Units Count="1">
       <Unit0>
         <Filename Value="1brc_th.pas"/>
@@ -115,6 +120,7 @@
     </Target>
     <SearchPaths>
       <IncludeFiles Value="$(ProjOutDir)"/>
+      <OtherUnitFiles Value="../../../baseline/Common"/>
       <UnitOutputDirectory Value="../../../bin/lib/$(TargetCPU)-$(TargetOS)"/>
     </SearchPaths>
     <Parsing>