DataStax 4.0.5 bug: DeleteById from Solr not delete from Cassandra


Just found one issue in DataStax 4.0 today: if we delete the data by ID from Solr, DataStax will remove all fields, but leave its ID field.

For example, we have ID:1 in Solr and Cassandra table, then run reuqest:
http://localhost:8983/solr/ckeyspace.tablename/update?commit=true&stream.body=1

Datastax will delete the data from Solr, but remove all fields from Cassandra(all fields has default value, null or false).

But if we deleteByQuery in Solr:id:1
, Datastax will delete data from Cassandra.
Hacking
Trying to figure out why this happens. so I use Java Decompiler: JD-GUI to decompile DataStax code, create eclipse project from it, then change cassandra script in dse-4.0.5/resources/cassandra/bin:

JVM_OPTS="$JVM_OPTS -agentlib:jdwp=transport=dt_socket,address=7777,server=y,suspend=n"
exec $NUMACTL "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" $props "$class"

Then restart DataStax: ./dse cassandra -s -f
This will start DataStax in remote debug mode.

Then run Solr deleteById command, add break point at DirectUpdateHandler2 and CassandraDirectUpdateHandler.

Follow the code, I find out that DataStax will call the following delete command in deleteById case: delete field1, fieldn from table where id=1, this will just delete all fields(except id field).

But in case of delete by query, DataStax will call: delete from table where id=1 which will delete the whole data.

The root cause of the issue is at: Cql3CassandraRowWriter.buildDeleteStatement: it does something unnecessary: get all columns from Casssdran and then build the delete field1... fieldn commands.

DataStax fixed this problem in 4.7:


4.0.5 code from com.datastax.bdp.search.solr.Cql3CassandraRowWriter:
  public void deleteById(SolrQueryRequest request, String key)
    throws IOException
  {
    String cqlDeleteStatement = buildDeleteStatement(key);
    doDeletes(request, Arrays.asList(new String[] { cqlDeleteStatement }));
  }

  private String buildDeleteStatement(String key)
    throws IOException
  {
    CFMetaData cfMetaData = this.columnFamilyStore.metadata;
    String compositeKeyClause = Cql3Utils.createKeyClauseFromSolrKey(cfMetaData.getKeyValidator(), cfMetaData.getCfDef(), key);
    
    List columnNameArray = new ArrayList();
    for (CFDefinition.Name name : cfMetaData.getCfDef().regularColumns()) {
      columnNameArray.add("\"" + name.toString() + "\"");
    }
    String delete = "DELETE %s FROM \"%s\".\"%s\" WHERE %s";
    return String.format(delete, new Object[] { commaJoiner.join(columnNameArray), this.coreInfo.keySpace, this.coreInfo.columnFamily, compositeKeyClause });
  }

4.7 code from com.datastax.bdp.search.solr.Cql3CassandraRowWriter:
  private String Cql3CassandraRowWriter.buildDeleteStatement(String key)
    throws IOException
  {
    CFMetaData cfMetaData = this.columnFamilyStore.metadata;
    String compositeKeyClause = Cql3Utils.createKeyClauseFromSolrKey(cfMetaData, key);
    
    String delete = "DELETE FROM \"%s\".\"%s\" WHERE %s";
    return String.format(delete, new Object[] { this.coreInfo.keySpace, this.coreInfo.columnFamily, compositeKeyClause });
  }
So now, we may choose to upgrade to DataStax 4.7, or have to change the code to use deleteByQuery instead of deleteById.

Happy hacking...

Labels

adsense (5) Algorithm (69) Algorithm Series (35) Android (7) ANT (6) bat (8) Big Data (7) Blogger (14) Bugs (6) Cache (5) Chrome (19) Code Example (29) Code Quality (7) Coding Skills (5) Database (7) Debug (16) Design (5) Dev Tips (63) Eclipse (32) Git (5) Google (33) Guava (7) How to (9) Http Client (8) IDE (7) Interview (88) J2EE (13) J2SE (49) Java (186) JavaScript (27) JSON (7) Learning code (9) Lesson Learned (6) Linux (26) Lucene-Solr (112) Mac (10) Maven (8) Network (9) Nutch2 (18) Performance (9) PowerShell (11) Problem Solving (11) Programmer Skills (6) regex (5) Scala (6) Security (9) Soft Skills (38) Spring (22) System Design (11) Testing (7) Text Mining (14) Tips (17) Tools (24) Troubleshooting (29) UIMA (9) Web Development (19) Windows (21) xml (5)