Making Child Documents Working with Spring-data-solr

The Problem
We use spring-data-solr in our project - as we like its conversion feature which can convert string to enum, entity to json data and etc, and vice versa, and recently we need use Solr's nested documents feature which spring-data-solr doesn't support.

Issues in Spring-data-solr
SolrInputDocument class contains a Map _fields AND List _childDocuments.

Spring-data-solr converts java entity class to SolrDocument. It provides two converters: MappingSolrConverter and SolrJConverter.

MappingSolrConverter converts the entity to a Map: MappingSolrConverter.write(Object, Map, SolrPersistentEntity)

SolrJConverter uses solr's DocumentObjectBinder to convert entity to SolrInputDocument,
it will convert field that is annotated with @Field(child = true) to child documents.
- This also means that spring-data-solt's convert features will not work with SolrJConverter

BUT SolrJConverter still just thinks SolrInputDocument is a map and add all into the destination: Map sink
- SolrJConverter.write(Object, Map)

After this, the child documents is discarded.

The Fix
We still want to use spring-data-solr's conversion functions - partly because we don't want to rewrite everything to use SolrJ directly.

So when save to solr: we uses spring-data-solr's MappingSolrConverter to convert parent entity as solrInputDocument, then convert child entities as solrInputDocuments and add them into parent's solrInputDocument.

When read from solr, we read the SolrDocument as parent entity, then read its child documents as child entities and add them into parent entity.
public class ParentEntity {
  @Field(child = true)
  private List<ChildEntity> children;
}
@Autowired
protected SolrClient solrClient;

// we add our own converters into MappingSolrConverter
// for more, please check 
// http://lifelongprogrammer.blogspot.com/2015/09/mix-spring-data-solr-and-solrj-in-solr.html
@Autowired
protected MyMappingSolrConverter solrConverter;

public void save(@Nonnull final ParentEntity parentEntity) {
    final SolrInputDocument solrInputDocument = solrConverter.createAndWrite(parentEntity);
    daddChildDocuemnts(parentEntity, solrInputDocument);
    try {
        solrClient.add(getCollection(), solrInputDocument);
        solrClient.commit(getCollection());
    } catch (SolrServerException | IOException e) {
        throw new BusinessException(e, "failed to save " + parentEntity);
    }
}

protected void daddChildDocuemnts(@Nonnull final ParentEntity parentEntity,
        @Nonnull final SolrInputDocument solrInputDocument) {
    solrInputDocument.addChildDocuments(parentEntity.getChildren().stream()
            .map(child -> solrConverter.createAndWrite(child)).collect(Collectors.toList()));
}

public List<T> querySolr(final SolrParams query) {
    try {
        final QueryResponse response = solrClient.query(getCollection(), query);
        return convertFromSolrDocs(response.getResults());
    } catch (final Exception e) {
        throw new BusinessException("data retrieve failed." + query);
    }
}
/*
 * Also return child documents in solr response as ChildEntity if it exists
 */
protected List<ParentEntity> convertFromSolrDocs(final SolrDocumentList docList) {
    List<ParentEntity> result = new ArrayList<>();
    if (docList != null) {
        result = docList.stream().map(solrDoc -> {
            final ParentEntity parentEntity = solrConverter.read(ParentEntity.class, solrDoc);
            final List<SolrDocument> childDocs = solrDoc.getChildDocuments();
            if (childDocs != null) {
                ParentEntity.setChildren(
                        childDocs.stream().map(childSolrDoc -> solrConverter.read(ChildEntity.class, solrDoc))
                                .collect(Collectors.toList()));
            }

            return parentEntity;
        }).collect(Collectors.toList());
    }

    return result;
}
Related
Mix Spring Data Solr and SolrJ in Solr Cloud 5
SolrJ: Support Converter and make it easier to extend DocumentObjectBinder
Post a Comment

Labels

Java (159) Lucene-Solr (112) Interview (61) All (58) J2SE (53) Algorithm (45) Soft Skills (38) Eclipse (33) Code Example (31) Linux (25) JavaScript (23) Spring (22) Windows (22) Web Development (20) Tools (19) Nutch2 (18) Bugs (17) Debug (16) Defects (14) Text Mining (14) J2EE (13) Network (13) Troubleshooting (13) PowerShell (11) Chrome (9) Design (9) How to (9) Learning code (9) Performance (9) Problem Solving (9) UIMA (9) html (9) Http Client (8) Maven (8) Security (8) bat (8) blogger (8) Big Data (7) Continuous Integration (7) Google (7) Guava (7) JSON (7) Shell (7) ANT (6) Coding Skills (6) Database (6) Lesson Learned (6) Programmer Skills (6) Scala (6) Tips (6) css (6) Algorithm Series (5) Cache (5) Dynamic Languages (5) IDE (5) System Design (5) adsense (5) xml (5) AIX (4) Code Quality (4) GAE (4) Git (4) Good Programming Practices (4) Jackson (4) Memory Usage (4) Miscs (4) OpenNLP (4) Project Managment (4) Spark (4) Testing (4) ads (4) regular-expression (4) Android (3) Apache Spark (3) Become a Better You (3) Concurrency (3) Eclipse RCP (3) English (3) Happy Hacking (3) IBM (3) J2SE Knowledge Series (3) JAX-RS (3) Jetty (3) Restful Web Service (3) Script (3) regex (3) seo (3) .Net (2) Android Studio (2) Apache (2) Apache Procrun (2) Architecture (2) Batch (2) Bit Operation (2) Build (2) Building Scalable Web Sites (2) C# (2) C/C++ (2) CSV (2) Career (2) Cassandra (2) Distributed (2) Fiddler (2) Firefox (2) Google Drive (2) Gson (2) How to Interview (2) Html Parser (2) Http (2) Image Tools (2) JQuery (2) Jersey (2) LDAP (2) Life (2) Logging (2) Python (2) Software Issues (2) Storage (2) Text Search (2) xml parser (2) AOP (1) Application Design (1) AspectJ (1) Chrome DevTools (1) Cloud (1) Codility (1) Data Mining (1) Data Structure (1) ExceptionUtils (1) Exif (1) Feature Request (1) FindBugs (1) Greasemonkey (1) HTML5 (1) Httpd (1) I18N (1) IBM Java Thread Dump Analyzer (1) JDK Source Code (1) JDK8 (1) JMX (1) Lazy Developer (1) Mac (1) Machine Learning (1) Mobile (1) My Plan for 2010 (1) Netbeans (1) Notes (1) Operating System (1) Perl (1) Problems (1) Product Architecture (1) Programming Life (1) Quality (1) Redhat (1) Redis (1) Review (1) RxJava (1) Solutions logs (1) Team Management (1) Thread Dump Analyzer (1) Visualization (1) boilerpipe (1) htm (1) ongoing (1) procrun (1) rss (1)

Popular Posts