Solr: Use UpdateRequestProcessorChain to Execute Processor Chain Before Import Data

Scenario
I am sending query to a middle proxy layer which talks with remote Solr server, import the received documents into Local Solr server.

There are different ways to put data into Local Solr server.
1. Use core.getUpdateHandler(), we can add SolrInputDocuemt, delete by id or query.
2. We can use SolrRequestHandler like below:
SolrRequestHandler updateHandler = core.getRequestHandler("/update");
SolrQueryRequest req = new LocalSolrQueryRequest(core,solrParams);
updateHandler.handleRequest(req, solrQueryRsp);
Be sure to close the created SolrQueryRequest. Please refer to.

Compared with method 2, method 1 is better, as this pass the phrases- 1: Create a XML or JSON documents playload. 2: Parse the playload. 3: Create a SolrInputDocuemt from playload.
But with method 1, we can 't tell it to execute one processor chain.
Solution
If we want to tell Solr to execute one processor chain, before import data, we can get one UpdateRequestProcessorChain, create a processor, and use the processor to add a new SolrInputDocuemt. 
UpdateRequestProcessorChain updateChian = request.getCore().getUpdateProcessingChain("dedup");
UpdateRequestProcessor processor = updateChian.createProcessor(request,response);
ddUpdateCommand command = new AddUpdateCommand(request);
command.solrDoc = toSolrServerSolrDoc;
processor.processAdd(command);
The complete code looks like below

private static boolean importDocs(String remoteDocsStr,
      SolrQueryRequest request) throws IOException {
    Object obj = ObjectBuilder.fromJSON(remoteDocsStr);
    HashMap map = (HashMap) obj;
    HashMap responseMap = (HashMap) map.get("response");
    List lists = (List) responseMap.get("docs");
    
    UpdateRequestProcessorChain updateChian = request.getCore()
        .getUpdateProcessingChain("dedup");
    for (HashMap docJsonMap : lists) {
      SolrInputDocument toSolrServerSolrDoc = convertToSolrDoc(docJsonMap);
      AddUpdateCommand command = new AddUpdateCommand(request);
      command.solrDoc = toSolrServerSolrDoc;
      SolrQueryResponse response = new SolrQueryResponse();
      UpdateRequestProcessor processor = updateChian.createProcessor(request,
          response);
      processor.processAdd(command);
    }
}
Post a Comment

Labels

Java (159) Lucene-Solr (110) Interview (61) All (58) J2SE (53) Algorithm (45) Soft Skills (37) Eclipse (33) Code Example (31) Linux (24) JavaScript (23) Spring (22) Windows (22) Web Development (20) Nutch2 (18) Tools (18) Bugs (17) Debug (16) Defects (14) Text Mining (14) J2EE (13) Network (13) Troubleshooting (13) PowerShell (11) Chrome (9) Design (9) How to (9) Learning code (9) Performance (9) Problem Solving (9) UIMA (9) html (9) Http Client (8) Maven (8) Security (8) bat (8) blogger (8) Big Data (7) Continuous Integration (7) Google (7) Guava (7) JSON (7) ANT (6) Coding Skills (6) Database (6) Scala (6) Shell (6) css (6) Algorithm Series (5) Cache (5) Dynamic Languages (5) IDE (5) Lesson Learned (5) Programmer Skills (5) System Design (5) Tips (5) adsense (5) xml (5) AIX (4) Code Quality (4) GAE (4) Git (4) Good Programming Practices (4) Jackson (4) Memory Usage (4) Miscs (4) OpenNLP (4) Project Managment (4) Spark (4) Testing (4) ads (4) regular-expression (4) Android (3) Apache Spark (3) Become a Better You (3) Concurrency (3) Eclipse RCP (3) English (3) Happy Hacking (3) IBM (3) J2SE Knowledge Series (3) JAX-RS (3) Jetty (3) Restful Web Service (3) Script (3) regex (3) seo (3) .Net (2) Android Studio (2) Apache (2) Apache Procrun (2) Architecture (2) Batch (2) Bit Operation (2) Build (2) Building Scalable Web Sites (2) C# (2) C/C++ (2) CSV (2) Career (2) Cassandra (2) Distributed (2) Fiddler (2) Firefox (2) Google Drive (2) Gson (2) How to Interview (2) Html Parser (2) Http (2) Image Tools (2) JQuery (2) Jersey (2) LDAP (2) Life (2) Logging (2) Python (2) Software Issues (2) Storage (2) Text Search (2) xml parser (2) AOP (1) Application Design (1) AspectJ (1) Chrome DevTools (1) Cloud (1) Codility (1) Data Mining (1) Data Structure (1) ExceptionUtils (1) Exif (1) Feature Request (1) FindBugs (1) Greasemonkey (1) HTML5 (1) Httpd (1) I18N (1) IBM Java Thread Dump Analyzer (1) JDK Source Code (1) JDK8 (1) JMX (1) Lazy Developer (1) Mac (1) Machine Learning (1) Mobile (1) My Plan for 2010 (1) Netbeans (1) Notes (1) Operating System (1) Perl (1) Problems (1) Product Architecture (1) Programming Life (1) Quality (1) Redhat (1) Redis (1) Review (1) RxJava (1) Solutions logs (1) Team Management (1) Thread Dump Analyzer (1) Visualization (1) boilerpipe (1) htm (1) ongoing (1) procrun (1) rss (1)

Popular Posts