Keep User's Change when Update solr.xml

Problem
In our product, user is able to create additional core: Solr will update solr.xml to add extra core information. But at some time, we need update solr.xml to add our extra cores. 
This is a common requirement: we need upgrade solr config files in future release, but at same time keep customer's change.

For solrconfig.xml or solrschema.xml, we can use xi:include to put our change in one xml(for example solrconfig-upgrade.xml), user should only change solrconfig.xml either manually or via GUI.
Please refer to http://wiki.apache.org/solr/SolrConfigXml#XInclude

But this doesn't work for solr.xml, as if we define xi:include in solr.xml:
<solr persistent="true" sharedLib="lib" shareSchema="true">
 <cores adminPath="/admin/cores" defaultCoreName="collection1" 
host="${host:}" hostPort="${jetty.port:}" hostContext="${hostContext:}" zkClientTimeout="${zkClientTimeout:15000}">
<core name="collection1" instanceDir="collection1"/>
  <!-- this doesn't work -->
   <xi:include href="solr_extracores.xml"/>
 </cores>
</solr>

later if user creates another solr core, solr will update solr.xml: it will delete <xi:include> and overwrite cores section to include all cores.
Solution
We can define extra cores in a property file: cvanalytics_solr_extra.propertie like below:
# support corename_name, corename_instanceDir, corename_schemaName, corename_configName
extra_cores=extracore1, extracore2
extracore1_name=extracore1
extracore1_instanceDir=extracore1
extracore1_schemaName=extracore1_schema.xml
extracore1_configName=extracore1_config.xml
....
We will iterate all core names in extra_cores, if it's already loaded: CoreContainer.getCoreFromAnyList(core_name) returns not null, we ignore, Otherwise, we create a SolrCore and register it.
Implementation Code
org.apache.solr.core.CoreContainer.Initializer
private void initExtraCores(CoreContainer cores) throws RuntimeException {
  try {
 Properties cvProps = cores.getCVProperties();
 String solrxml_extra = cvProps.getProperty("solrxml_extra");
 if (!StringUtils.isBlank(solrxml_extra)) {
   File solrxml_extraFile = new File(cores.getSolrHome(), solrxml_extra);
   Properties solrxml_extraProps = readProperties(solrxml_extraFile);
   
   String extra_cores = solrxml_extraProps.getProperty("extra_cores");
   if (!StringUtils.isBlank(extra_cores)) {
  String[] extra_coresArr = extra_cores.split(",");
  for (String extra_core : extra_coresArr) {
    extra_core = extra_core.trim();
    if (cores.getCoreFromAnyList(extra_core) == null) {
   String coreName = solrxml_extraProps.getProperty(extra_core
    + "_name");
   if (StringUtils.isBlank(coreName)) {
     coreName = extra_core;
   }
   String instanceDir = solrxml_extraProps.getProperty(extra_core
    + "_instanceDir");
   if (StringUtils.isBlank(instanceDir)) {
     throw new RuntimeException("No instanceDir defined for "
      + coreName + "in " + solrxml_extraFile);
   }
   CoreDescriptor cd = new CoreDescriptor(cores, coreName,
    instanceDir);
   String schemaName = solrxml_extraProps.getProperty(extra_core
    + "_schemaName");
   if (!StringUtils.isBlank(schemaName)) {
     cd.setSchemaName(schemaName);
   }
   String configName = solrxml_extraProps.getProperty(extra_core
    + "_configName");
   if (!StringUtils.isBlank(configName)) {
     cd.setConfigName(configName);
   }
   SolrCore core = cores.create(cd);
   cores.register(core, false);
    }
  }
   }
 }
  } catch (Exception e) {
 throw new RuntimeException(e);
  }
}
public CoreContainer initialize() {
  CoreContainer cores = null;
  String solrHome = SolrResourceLoader.locateSolrHome();
  cores = new CoreContainer(solrHome);
  
  if (fconf.exists()) {
 cores.load(solrHome, fconf);
  } else {
 log.info("no solr.xml file found - using default");
 try {
   cores.load(solrHome, new InputSource(new ByteArrayInputStream(DEF_SOLR_XML.getBytes("UTF-8"))));
 } catch (Exception e) {
   throw new SolrException(ErrorCode.SERVER_ERROR,
    "CoreContainer.Initialize failed when trying to load default solr.xml file", e);
 }
 cores.configFile = fconf;
  }
  // call initExtraCores
  initExtraCores(cores);
  return cores;
}
Post a Comment

Labels

Java (159) Lucene-Solr (110) All (60) Interview (59) J2SE (53) Algorithm (37) Eclipse (35) Soft Skills (35) Code Example (31) Linux (26) JavaScript (23) Spring (22) Windows (22) Web Development (20) Tools (19) Nutch2 (18) Bugs (17) Debug (15) Defects (14) Text Mining (14) J2EE (13) Network (13) PowerShell (11) Chrome (9) Continuous Integration (9) How to (9) Learning code (9) Performance (9) UIMA (9) html (9) Design (8) Dynamic Languages (8) Http Client (8) Maven (8) Security (8) Trouble Shooting (8) bat (8) blogger (8) Big Data (7) Google (7) Guava (7) JSON (7) Problem Solving (7) ANT (6) Coding Skills (6) Database (6) Scala (6) Shell (6) css (6) Algorithm Series (5) Cache (5) IDE (5) Lesson Learned (5) Miscs (5) Programmer Skills (5) System Design (5) Tips (5) adsense (5) xml (5) AIX (4) Code Quality (4) GAE (4) Git (4) Good Programming Practices (4) Jackson (4) Memory Usage (4) OpenNLP (4) Project Managment (4) Python (4) Spark (4) Testing (4) ads (4) regular-expression (4) Android (3) Apache Spark (3) Become a Better You (3) Concurrency (3) Eclipse RCP (3) English (3) Firefox (3) Happy Hacking (3) IBM (3) J2SE Knowledge Series (3) JAX-RS (3) Jetty (3) Restful Web Service (3) Script (3) regex (3) seo (3) .Net (2) Android Studio (2) Apache (2) Apache Procrun (2) Architecture (2) Batch (2) Build (2) Building Scalable Web Sites (2) C# (2) C/C++ (2) CSV (2) Career (2) Cassandra (2) Distributed (2) Fiddler (2) Google Drive (2) Gson (2) Html Parser (2) Http (2) Image Tools (2) JQuery (2) Jersey (2) LDAP (2) Life (2) Logging (2) Software Issues (2) Storage (2) Text Search (2) xml parser (2) AOP (1) Application Design (1) AspectJ (1) Bit Operation (1) Chrome DevTools (1) Cloud (1) Codility (1) Data Mining (1) Data Structure (1) ExceptionUtils (1) Exif (1) Feature Request (1) FindBugs (1) Greasemonkey (1) HTML5 (1) Httpd (1) I18N (1) IBM Java Thread Dump Analyzer (1) JDK Source Code (1) JDK8 (1) JMX (1) Lazy Developer (1) Mac (1) Machine Learning (1) Mobile (1) My Plan for 2010 (1) Netbeans (1) Notes (1) Operating System (1) Perl (1) Problems (1) Product Architecture (1) Programming Life (1) Quality (1) Redhat (1) Redis (1) Review (1) RxJava (1) Solutions logs (1) Team Management (1) Thread Dump Analyzer (1) Troubleshooting (1) Visualization (1) boilerpipe (1) htm (1) ongoing (1) procrun (1) rss (1)

Popular Posts