How Solr Join Works

QParserPlugin
Exaple query: http://localhost:8080/solr/select?q={!join from=classid to=classid}classname:"Grade 1"
QueryComponent.prepare call QParser.getParser which will 
QParser parser = QParser.getParser(rb.getQueryString(), defType, req);

QParser.getParser
default parser is lucene. It will parse local parameter to get the parser type, in this case it is join: {!join from=classid to=classid}. Put the query classname:"Grade 1" into localMap, key=QueryParsing.V.

if (qstr != null && qstr.startsWith(QueryParsing.LOCALPARAM_START)) {
{
 localParamsEnd = QueryParsing.parseLocalParams(qstr, 0, localMap, globalParams);
 //It first tries to get query string from local param
 String val = localMap.get(QueryParsing.V);
 if (val == null) {
  val = qstr.substring(localParamsEnd);
  localMap.put(QueryParsing.V, val);
 }
}
For the join query, this will get JoinQParserPlugin.
QParserPlugin qplug = req.getCore().getQueryPlugin(parserName);
QParser parser =  qplug.createParser(qstr, localParams, req.getParams(), req);

QueryComponent.prepare then calls the parser to get the query. 
 Query q = (QParser)parser.getQuery();, 
This will call the abstract parse method to pear the query string -the parse is only called once.
In this case, it is the anonymous parser class returned by JoinQParserPlugin.createParser.

In the anonymous parser class, 
if fromIndex parameter is not null, 
Then it creates subQuery from the query string:classname:"Grade 1". This will again determine/ create the parser for the query string via Parser.getParser, and get the query via LuceneQParser.parse.
QParser fromQueryParser = subQuery(v, null);
fromQuery = fromQueryParser.getQuery();

Then create one join query, and return it.
JoinQuery jq = new JoinQuery(fromField, toField, fromIndex, fromQuery);
jq.fromCoreOpenTime = fromCoreOpenTime;

Classes:
LuceneQParserPlugin
createParser creates LuceneQParser.
QParser
QParser is an abstract class, main methods include:
static getParser like a factory which creats concrete parser class based on local parameter. 
abstract parse method: create and return the Query object represented by qstr.
LuceneQParser extends QParser and implements parse method. In the parse method, LuceneQParser creates a QueryParser, then call QueryParser.parse to parse the qstr and return a Query.
SolrQueryParserBase
QueryParser extends SolrQueryParserBase
SolrQueryParser extends QueryParser
Query
JoinQuery, TermQuery,BooleanQuery.

QueryComponent.prepare then parse fq(filer query), save it to ResponseBuilder.
QParser fqp = QParser.getParser(fq, null, req);
filters.add(fqp.getQuery());
rb.setFilters( filters )

QueryComponent.process
this will create a QueryCommand: Query:JoinQuery.
SolrIndexSearcher.QueryCommand cmd = rb.getQueryCommand();
searcher.search(result,cmd);
filterList:[studentname:*], query:JoinQuery{!join from=classid to=classid}classname:Grade 1

Main task is done in SolrIndexSearcher.search
SolrIndexSearcher extends lucene.search.IndexSearcher.

SolrIndexSearcher.getDocListC(QueryResult, QueryCommand)
Query q = cmd.getQuery() --> JoinQuery
It will first try to get it from cache, the key is comprised of the Query object, sort, nflags. 
key = new QueryResultKey(q, cmd.getFilterList(), cmd.getSort(), flags);
superset = (SolrCache<QueryResultKey,DocList> queryResultCache)queryResultCache.get(key);
if (maxDocRequested < queryResultWindowSize) {
  supersetMaxDoc=queryResultWindowSize;
}
cmd.setSupersetMaxDoc(supersetMaxDoc);
At the first time, superset is null, then it will call getDocListNC(qr,cmd);

[getDocListNC]
ProcessedFilter pf = getProcessedFilter(cmd.getFilter(), cmd.getFilterList());
This will search the fq, and save the returned docset in pf.
cmd.getFilterList() is List<Query>: [studentname:*]
sets[end] = getPositiveDocSet(posQuery);
answer is DocSet, in this case BitDocSet which includs OpenBitSet.
answer = getDocSetNC(q,null);
if (filterCache != null) filterCache.put(q,answer);
pf.answer = answer;

getDocSetNC calls (lucene.search.IndexSearcher)super.search(query,null,collector);

[getDocListNC]
topCollector = TopScoreDocCollector.create(len, true);
Query is {!join from=classid to=classid}classname:Grade 1. luceneFilter is the the result which fq search returns
(lucene.search.IndexSearcher)super.search(query, luceneFilter, collector);

Here
(lucene.search.IndexSearcher)search(leafContexts, createNormalizedWeight(wrapFilter(query, filter)), results);

createNormalizedWeight will call Weight weight = query.createWeight(this); This will go to JoinQuery.createWeight.

[JoinQueryWeight]
Threadlocal:
SolrRequestInfo info = SolrRequestInfo.getRequestInfo();

[QueryComponent.process]
searcher.search(result,cmd);
rb.setResult( result );
doFieldSortValues(rb, searcher);
doPrefetch(rb);

Classes
lucene.search.IndexSearcher.search(List<AtomicReaderContext>, Weight, Collector)
Post a Comment

Labels

Java (159) Lucene-Solr (110) All (60) Interview (59) J2SE (53) Algorithm (37) Eclipse (35) Soft Skills (35) Code Example (31) Linux (26) JavaScript (23) Spring (22) Windows (22) Web Development (20) Tools (19) Nutch2 (18) Bugs (17) Debug (15) Defects (14) Text Mining (14) J2EE (13) Network (13) PowerShell (11) Chrome (9) Continuous Integration (9) How to (9) Learning code (9) Performance (9) UIMA (9) html (9) Design (8) Dynamic Languages (8) Http Client (8) Maven (8) Security (8) Trouble Shooting (8) bat (8) blogger (8) Big Data (7) Google (7) Guava (7) JSON (7) Problem Solving (7) ANT (6) Coding Skills (6) Database (6) Scala (6) Shell (6) css (6) Algorithm Series (5) Cache (5) IDE (5) Lesson Learned (5) Miscs (5) Programmer Skills (5) System Design (5) Tips (5) adsense (5) xml (5) AIX (4) Code Quality (4) GAE (4) Git (4) Good Programming Practices (4) Jackson (4) Memory Usage (4) OpenNLP (4) Project Managment (4) Python (4) Spark (4) Testing (4) ads (4) regular-expression (4) Android (3) Apache Spark (3) Become a Better You (3) Concurrency (3) Eclipse RCP (3) English (3) Firefox (3) Happy Hacking (3) IBM (3) J2SE Knowledge Series (3) JAX-RS (3) Jetty (3) Restful Web Service (3) Script (3) regex (3) seo (3) .Net (2) Android Studio (2) Apache (2) Apache Procrun (2) Architecture (2) Batch (2) Build (2) Building Scalable Web Sites (2) C# (2) C/C++ (2) CSV (2) Career (2) Cassandra (2) Distributed (2) Fiddler (2) Google Drive (2) Gson (2) Html Parser (2) Http (2) Image Tools (2) JQuery (2) Jersey (2) LDAP (2) Life (2) Logging (2) Software Issues (2) Storage (2) Text Search (2) xml parser (2) AOP (1) Application Design (1) AspectJ (1) Bit Operation (1) Chrome DevTools (1) Cloud (1) Codility (1) Data Mining (1) Data Structure (1) ExceptionUtils (1) Exif (1) Feature Request (1) FindBugs (1) Greasemonkey (1) HTML5 (1) Httpd (1) I18N (1) IBM Java Thread Dump Analyzer (1) JDK Source Code (1) JDK8 (1) JMX (1) Lazy Developer (1) Mac (1) Machine Learning (1) Mobile (1) My Plan for 2010 (1) Netbeans (1) Notes (1) Operating System (1) Perl (1) Problems (1) Product Architecture (1) Programming Life (1) Quality (1) Redhat (1) Redis (1) Review (1) RxJava (1) Solutions logs (1) Team Management (1) Thread Dump Analyzer (1) Troubleshooting (1) Visualization (1) boilerpipe (1) htm (1) ongoing (1) procrun (1) rss (1)

Popular Posts