How Solr Executes a Request


How Solr Executes Search Request to a Single Server
SolrDispatchFilter.doFilter
Find the RequestHandler corresponding to the path.
core.getRequestHandler( path )
SolrDispatchFilter.execute
sreq.getContext().put( "webapp", req.getContextPath() );

SolrCore.execute will call handler.handleRequest.
RequestHandlerBase.handleRequest define basic skeleton, which calls handleRequestBody implemented in subclass.
How Solr parses query string and content stream
SolrDispatchFilter will call solrReq = (SolrRequestParsers)parser.parse( core, path, req );

SolrRequestParsers.parse will put the data into SolrParams and contentStream of request.
SolrParams params = parser.parseParamsAndFillStreams( req, streams );
SolrQueryRequest sreq = buildRequestFrom( core, params, streams );
sreq.getContext().put( "path", path );

SolrRequestParsers wraps multiple SolrRequestParser, in parseParamsAndFillStreams, it will call corresponding SolrRequestParser based on request method(Get or Post) and contentType - formdata: "application/x-www-form-urlencoded", or multipart content.

SolrRequestParsers.init
  parsers.put( MULTIPART, multi );
  parsers.put( FORMDATA, formdata );
  parsers.put( RAW, raw );
  parsers.put( SIMPLE, new SimpleRequestParser() );
  parsers.put( STANDARD, standard );
  parsers.put( "", standard );
RequestHandlerBase.init will parse the request hander definition in solrconfig.xml, and put them into defaults, appends, invariants accordingly.
There is only one instance of request hander for a core, be careful of its thread safety.

SolrCore constructor calls loadSearchComponents, then initPlugins to load all components defined, will also put all classed defined in solrconfig.xml which implement SolrCoreAware int waitingForCore.

Then later it will cal SolrResourceLoader.inform(SolrCore), to call inform(SolrCore) defined in each class.

For a non-shard query, SearchHandler.handleRequestBody will call prepare on all components, and then call process on all components.

QueryComponent.prepare user QParser to prepare parameter to get query, sort.
FacetComponent.prepare check whether facet it enabled: facet=true
MoreLikeThisComponent.prepare does nothing
HighlightComponent.prepare check whether highlight it enabled: hl=true
StatsComponent.prepare check whether stats it enabled: stats=true
DebugComponent.prepare does nothing

QueryComponent.process do the real search stuff, grouping
SolrIndexSearcher searcher = req.getSearcher();
searcher.search(QueryResult qr, QueryCommand cmd)

SearchHandler.inform will create SearchComponent, shardHandlerFactory.
Solr.postDecorateResponse will put status, QTime, and possibly request handler and params, in the response header.
Then SolrDispatchFilter will choose QueryResponseWriter and write response.
Classes: XMLResponseWriter, XMLWriter; CSVResponseWriter, CSVWriter.
Classes
QParser: Parse query, sort
QParserPlugin.DEFAULT_QTYPE=lucene
QueryParsing.LOCALPARAM_START
How to parse Local Param
QParser.getParser(String, String, SolrQueryRequest)
if(qstr.startsWith(QueryParsing.LOCALPARAM_START))
localParamsEnd = QueryParsing.parseLocalParams(qstr, 0, localMap, globalParams);

ThreadLocal: SolrRequestInfo
protected final static ThreadLocal<SolrRequestInfo> threadLocal = new ThreadLocal<SolrRequestInfo>();
It is cleared in finally bock of SolrDispatchFilter.doFilter.
How Solr Executes Shard Requests
Same as non-shard request, through SolrDispatchFilter to SearchHandler.handleRequestBody.
It will call prepare on all components, it defines different stages for distributed request. In method distributedProcess of each component, it checks the current stages and responds accordingly.
If needed, the component may create ShardRequest, and call modifyRequest on each components in ResponseBuilder.addRequest(SearchComponent, ShardRequest).
Then it will remove parameter like shards, set distrib false, use completionService and Callable task to send request to all shards using ShardHandler, later will get ShardResponse back, which wraps SolrResponse. Then call handleResponses on all components, which usually merges response from multiple servers.
Then call finishStage on all components.

ShardHandler shardHandler1 = shardHandlerFactory.getShardHandler();
shardHandler1.checkDistributed(rb);

for (String shard : sreq.actualShards) {
shardHandler1.submit(sreq, shard, params);
}

handler.component.HttpShardHandler.submit will use completionService to submit a Callable task to call HttpSolrServer to send the request,
SolrServer server = new HttpSolrServer(url, httpClient);
ssr.nl = server.request(req);

In the shards parameter, for each shard, we can use | to specify multiple solr server to balance request in multiple servers.
handler.component.HttpShardHandler.getURLs(String)
urls = StrUtils.splitSmart(shard, "|", true);
Resources
http://wiki.apache.org/solr/DistributedSearch

Labels

adsense (5) Algorithm (69) Algorithm Series (35) Android (7) ANT (6) bat (8) Big Data (7) Blogger (14) Bugs (6) Cache (5) Chrome (19) Code Example (29) Code Quality (7) Coding Skills (5) Database (7) Debug (16) Design (5) Dev Tips (63) Eclipse (32) Git (5) Google (33) Guava (7) How to (9) Http Client (8) IDE (7) Interview (88) J2EE (13) J2SE (49) Java (186) JavaScript (27) JSON (7) Learning code (9) Lesson Learned (6) Linux (26) Lucene-Solr (112) Mac (10) Maven (8) Network (9) Nutch2 (18) Performance (9) PowerShell (11) Problem Solving (11) Programmer Skills (6) regex (5) Scala (6) Security (9) Soft Skills (38) Spring (22) System Design (11) Testing (7) Text Mining (14) Tips (17) Tools (24) Troubleshooting (29) UIMA (9) Web Development (19) Windows (21) xml (5)