Our Goal: Use less Memory
As our Solr application runs in client machine, so it's important for us to use less memory as possible as we can.
In Solr, we have fields which are used only for search and one way to reduce the index size and memory usage is to remove the term that is larger than the threshold: for example: 50.
The reason is that the user is very unlikely to search on these large terms, so why keep them in the index.
The Definition for content field
Other Tricks
Reduce fetch size when retrieve data from remote server.
Commit frequently.
Reduce the cache size
Monitor memory usage and run gc when needed.
As our Solr application runs in client machine, so it's important for us to use less memory as possible as we can.
In Solr, we have fields which are used only for search and one way to reduce the index size and memory usage is to remove the term that is larger than the threshold: for example: 50.
The reason is that the user is very unlikely to search on these large terms, so why keep them in the index.
The Definition for content field
<fieldType name="text_rev_trucated" class="solr.TextField" positionIncrementGap="100" >
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0" preserveOriginal="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishMinimalStemFilterFactory"/>
<filter class="solr.LengthFilterFactory" min="1" max="50"/>
<filter class="solr.ReversedWildcardFilterFactory" withOriginal="true" maxPosAsterisk="3" maxPosQuestion="2" maxFractionAsterisk="0.33"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="0" preserveOriginal="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishMinimalStemFilterFactory"/>
</analyzer>
</fieldType>
Other Tricks
Reduce fetch size when retrieve data from remote server.
Commit frequently.
Reduce the cache size
Monitor memory usage and run gc when needed.
