How Solr Executes a Request

How Solr Executes Search Request to a Single Server
SolrDispatchFilter.doFilter
Find the RequestHandler corresponding to the path.
core.getRequestHandler( path )
SolrDispatchFilter.execute
sreq.getContext().put( "webapp", req.getContextPath() );

SolrCore.execute will call handler.handleRequest.
RequestHandlerBase.handleRequest define basic skeleton, which calls handleRequestBody implemented in subclass.
How Solr parses query string and content stream
SolrDispatchFilter will call solrReq = (SolrRequestParsers)parser.parse( core, path, req );

SolrRequestParsers.parse will put the data into SolrParams and contentStream of request.
SolrParams params = parser.parseParamsAndFillStreams( req, streams );
SolrQueryRequest sreq = buildRequestFrom( core, params, streams );
sreq.getContext().put( "path", path );

SolrRequestParsers wraps multiple SolrRequestParser, in parseParamsAndFillStreams, it will call corresponding SolrRequestParser based on request method(Get or Post) and contentType - formdata: "application/x-www-form-urlencoded", or multipart content.

SolrRequestParsers.init
  parsers.put( MULTIPART, multi );
  parsers.put( FORMDATA, formdata );
  parsers.put( RAW, raw );
  parsers.put( SIMPLE, new SimpleRequestParser() );
  parsers.put( STANDARD, standard );
  parsers.put( "", standard );
RequestHandlerBase.init will parse the request hander definition in solrconfig.xml, and put them into defaults, appends, invariants accordingly.
There is only one instance of request hander for a core, be careful of its thread safety.

SolrCore constructor calls loadSearchComponents, then initPlugins to load all components defined, will also put all classed defined in solrconfig.xml which implement SolrCoreAware int waitingForCore.

Then later it will cal SolrResourceLoader.inform(SolrCore), to call inform(SolrCore) defined in each class.

For a non-shard query, SearchHandler.handleRequestBody will call prepare on all components, and then call process on all components.

QueryComponent.prepare user QParser to prepare parameter to get query, sort.
FacetComponent.prepare check whether facet it enabled: facet=true
MoreLikeThisComponent.prepare does nothing
HighlightComponent.prepare check whether highlight it enabled: hl=true
StatsComponent.prepare check whether stats it enabled: stats=true
DebugComponent.prepare does nothing

QueryComponent.process do the real search stuff, grouping
SolrIndexSearcher searcher = req.getSearcher();
searcher.search(QueryResult qr, QueryCommand cmd)

SearchHandler.inform will create SearchComponent, shardHandlerFactory.
Solr.postDecorateResponse will put status, QTime, and possibly request handler and params, in the response header.
Then SolrDispatchFilter will choose QueryResponseWriter and write response.
Classes: XMLResponseWriter, XMLWriter; CSVResponseWriter, CSVWriter.
Classes
QParser: Parse query, sort
QParserPlugin.DEFAULT_QTYPE=lucene
QueryParsing.LOCALPARAM_START
How to parse Local Param
QParser.getParser(String, String, SolrQueryRequest)
if(qstr.startsWith(QueryParsing.LOCALPARAM_START))
localParamsEnd = QueryParsing.parseLocalParams(qstr, 0, localMap, globalParams);

ThreadLocal: SolrRequestInfo
protected final static ThreadLocal<SolrRequestInfo> threadLocal = new ThreadLocal<SolrRequestInfo>();
It is cleared in finally bock of SolrDispatchFilter.doFilter.
How Solr Executes Shard Requests
Same as non-shard request, through SolrDispatchFilter to SearchHandler.handleRequestBody.
It will call prepare on all components, it defines different stages for distributed request. In method distributedProcess of each component, it checks the current stages and responds accordingly.
If needed, the component may create ShardRequest, and call modifyRequest on each components in ResponseBuilder.addRequest(SearchComponent, ShardRequest).
Then it will remove parameter like shards, set distrib false, use completionService and Callable task to send request to all shards using ShardHandler, later will get ShardResponse back, which wraps SolrResponse. Then call handleResponses on all components, which usually merges response from multiple servers.
Then call finishStage on all components.

ShardHandler shardHandler1 = shardHandlerFactory.getShardHandler();
shardHandler1.checkDistributed(rb);

for (String shard : sreq.actualShards) {
shardHandler1.submit(sreq, shard, params);
}

handler.component.HttpShardHandler.submit will use completionService to submit a Callable task to call HttpSolrServer to send the request,
SolrServer server = new HttpSolrServer(url, httpClient);
ssr.nl = server.request(req);

In the shards parameter, for each shard, we can use | to specify multiple solr server to balance request in multiple servers.
handler.component.HttpShardHandler.getURLs(String)
urls = StrUtils.splitSmart(shard, "|", true);
Resources
http://wiki.apache.org/solr/DistributedSearch
Post a Comment

Labels

Java (159) Lucene-Solr (110) Interview (61) All (58) J2SE (53) Algorithm (45) Soft Skills (36) Eclipse (34) Code Example (31) Linux (24) JavaScript (23) Spring (22) Windows (22) Web Development (20) Nutch2 (18) Tools (18) Bugs (17) Debug (16) Defects (14) Text Mining (14) J2EE (13) Network (13) Troubleshooting (12) PowerShell (11) Chrome (9) Design (9) How to (9) Learning code (9) Performance (9) UIMA (9) html (9) Http Client (8) Maven (8) Problem Solving (8) Security (8) bat (8) blogger (8) Big Data (7) Continuous Integration (7) Google (7) Guava (7) JSON (7) ANT (6) Coding Skills (6) Database (6) Scala (6) Shell (6) css (6) Algorithm Series (5) Cache (5) Dynamic Languages (5) IDE (5) Lesson Learned (5) Programmer Skills (5) Tips (5) adsense (5) xml (5) AIX (4) Code Quality (4) GAE (4) Git (4) Good Programming Practices (4) Jackson (4) Memory Usage (4) Miscs (4) OpenNLP (4) Project Managment (4) Spark (4) System Design (4) Testing (4) ads (4) regular-expression (4) Android (3) Apache Spark (3) Become a Better You (3) Concurrency (3) Eclipse RCP (3) English (3) Happy Hacking (3) IBM (3) J2SE Knowledge Series (3) JAX-RS (3) Jetty (3) Restful Web Service (3) Script (3) regex (3) seo (3) .Net (2) Android Studio (2) Apache (2) Apache Procrun (2) Architecture (2) Batch (2) Bit Operation (2) Build (2) Building Scalable Web Sites (2) C# (2) C/C++ (2) CSV (2) Career (2) Cassandra (2) Distributed (2) Fiddler (2) Firefox (2) Google Drive (2) Gson (2) How to Interview (2) Html Parser (2) Http (2) Image Tools (2) JQuery (2) Jersey (2) LDAP (2) Life (2) Logging (2) Python (2) Software Issues (2) Storage (2) Text Search (2) xml parser (2) AOP (1) Application Design (1) AspectJ (1) Chrome DevTools (1) Cloud (1) Codility (1) Data Mining (1) Data Structure (1) ExceptionUtils (1) Exif (1) Feature Request (1) FindBugs (1) Greasemonkey (1) HTML5 (1) Httpd (1) I18N (1) IBM Java Thread Dump Analyzer (1) JDK Source Code (1) JDK8 (1) JMX (1) Lazy Developer (1) Mac (1) Machine Learning (1) Mobile (1) My Plan for 2010 (1) Netbeans (1) Notes (1) Operating System (1) Perl (1) Problems (1) Product Architecture (1) Programming Life (1) Quality (1) Redhat (1) Redis (1) Review (1) RxJava (1) Solutions logs (1) Team Management (1) Thread Dump Analyzer (1) Visualization (1) boilerpipe (1) htm (1) ongoing (1) procrun (1) rss (1)

Popular Posts