@@ -782,104 +782,7 @@ You can also force a specific repository test from running through the following
782782
783783## 9. Tuning OpenGrok for large code bases
784784
785- ### 9.1 Almost atomic index flip using ZFS
786-
787- While indexing big source repos you might consider using ZFS filesystem to give
788- you advantage of datasets which can be flipped over or cloned when needed.
789- If the machine is strong enough it will also give you an option to
790- incrementally index in parallel to having the current sources and index in sync.
791- (So Tomcat sees certain zfs datasets, then you just stop it, flip datasets to
792- the ones that were updated by SCM/index and start tomcat again - outage is
793- minimal, sources+indexes are ** always** in sync, users see the truth)
794-
795- ### 9.2 JVM tuning
796-
797- OpenGrok script by default uses 2GB of heap and 16MB per thread for flush size of
798- lucene docs indexing(when to flush to disk).
799- It also uses default 32bit JRE.
800- This ** might not** be enough. You might need to consider this:
801- Lucene 4.x sets indexer defaults:
802-
803- ```
804- DEFAULT_RAM_PER_THREAD_HARD_LIMIT_MB = 1945;
805- DEFAULT_MAX_THREAD_STATES = 8;
806- DEFAULT_RAM_BUFFER_SIZE_MB = 16.0;
807- ```
808-
809- * which might grow as big as 16GB (though ` DEFAULT_RAM_BUFFER_SIZE_MB ` shouldn't
810- really allow it, but keep it around 1-2GB)
811-
812- * the lucenes ` RAM_BUFFER_SIZE_MB ` can be tuned now using the parameter ` -m ` , so
813- running a 8GB 64 bit server JDK indexer with tuned docs flushing(on Solaris 11):
814-
815- ```
816- # export JAVA=/usr/java/bin/`isainfo -k`/java
817- (or use /usr/java/bin/amd64/java )
818- # export JAVA_OPTS="-Xmx8192m -server"
819- # OPENGROK_FLUSH_RAM_BUFFER_SIZE="-m 256" ./OpenGrok index /source
820- ```
821-
822- Tomcat by default also supports only small deployments. For bigger ones you
823- ** might** need to increase its heap which might necessitate the switch to 64-bit
824- Java. It will most probably be the same for other containers as well.
825- For tomcat you can easily get this done by creating ` conf/setenv.sh ` :
826-
827- ``` bash
828- # cat conf/setenv.sh
829- # 64-bit Java
830- JAVA_OPTS=" $JAVA_OPTS -d64 -server"
831-
832- # OpenGrok memory boost to cover all-project searches
833- # (7 MB * 247 projects + 300 MB for cache should be enough)
834- # 64-bit Java allows for more so let's use 8GB to be on the safe side.
835- # We might need to allow more for concurrent all-project searches.
836- JAVA_OPTS=" $JAVA_OPTS -Xmx8g"
837-
838- export JAVA_OPTS
839- ```
840-
841- ### 9.3 Tomcat/Apache tuning
842-
843- For tomcat you might also hit a limit for http header size (we use it to send
844- the project list when requesting search results):
845-
846- * increase(add) in ` conf/server.xml `
847-
848- ``` xml
849- maxHttpHeaderSize
850- connectionTimeout="20000"
851- maxHttpHeaderSize="65536"
852- redirectPort="8443" />
853- ```
854-
855- Refer to docs of other containers for more info on how to achieve the same.
856-
857- The same tuning to Apache can be done with the ` LimitRequestLine ` directive:
858-
859- ```
860- LimitRequestLine 65536
861- LimitRequestFieldSize 65536
862- ```
863-
864- ### 9.4 Open File and processes hard and soft limits
865-
866- The initial index creation process is resource intensive and often the error
867- ` java.io.IOException: error=24, Too many open files ` appears in the logs. To
868- avoid this increase the ` ulimit ` value to a higher number.
869-
870- It is noted that the hard and soft limit for open files of 10240 works for mid
871- sized repositories and so the recommendation is to start with 10240.
872-
873- If you get a similar error, but for threads:
874- ` java.lang.OutOfMemoryError: unable to create new native thread `
875- it might be due to strict security limits and you need to increase the limits.
876-
877- ### 9.5 Multi-project search speed tip
878-
879- If multi-project search is performed frequently, it might be good to warm
880- up file system cache after each reindex. This can be done e.g. with
881- < https://github.com/hoytech/vmtouch >
882-
785+ See https://github.com/oracle/opengrok/wiki/Tuning-for-large-code-bases
883786
884787## 10. Authors
885788
0 commit comments