Script to Setup SolrCloud Environment

Senario
Here is the script how to create SolrCloud environment in local dev setup: a collection with numShards=2&replicationFactor=3 on 3 separate (local) nodes.

vms_solr_init creates folders: example/cloud/node{1,2,3}/solr, copy solr.xml and zoo.cfg to these folders, starts the server and creates the collection using admin collection apis.

Other scripts which start/stop are easier to implement.
The implementation
function solr_init()
{
  cd $SOLR_HOME
  mkdir -p $SOLR_NODE1_REL_HOME
  mkdir -p $SOLR_NODE2_REL_HOME
  mkdir -p $SOLR_NODE3_REL_HOME

  cp $SOLR_HOME/server/solr/solr.xml $SOLR_HOME/server/solr/zoo.cfg $SOLR_NODE1_REL_HOME
  cp $SOLR_HOME/server/solr/solr.xml $SOLR_HOME/server/solr/zoo.cfg $SOLR_NODE2_REL_HOME
  cp $SOLR_HOME/server/solr/solr.xml $SOLR_HOME/server/solr/zoo.cfg $SOLR_NODE3_REL_HOME

  solr_start
  data_solr_create
}

function solr_start() {
  if [[ `solr_pid` ]]
  then
    echo "solr is already running...";
  else
    echo "Starting solr-cloud... $SOLR_NODE1_PORT, $SOLR_NODE2_PORT, $SOLR_NODE3_PORT";
    $SOLR_HOME/bin/solr start -cloud -Dsolr.ltr.enabled=true -s "$SOLR_NODE1_REL_HOME" -p $SOLR_NODE1_PORT -h $SOLR_HOSTNAME;
    $SOLR_HOME/bin/solr start -cloud -Dsolr.ltr.enabled=true -s "$SOLR_NODE2_REL_HOME" -p $SOLR_NODE2_PORT -z $SOLR_ZKHOST -h $SOLR_HOSTNAME;
    $SOLR_HOME/bin/solr start -cloud -Dsolr.ltr.enabled=true -s "$SOLR_NODE3_REL_HOME" -p $SOLR_NODE3_PORT -z $SOLR_ZKHOST -h $SOLR_HOSTNAME;
  fi
}

function solr_stop() {
  echo "Stopping solr-cloud...";
  $SOLR_HOME/bin/solr stop -all;
}

function solr_restart() {
  echo "Restarting solr-cloud...";
  solr_stop && solr_start
}

function solr_pid() {
  pgrep -f "solr-6.4.0/server";
}

function data_solr_create() {
  # Go to the solr config directory
  currdir=`pwd`;
  cd "$WS/resource/solr";

  # Retrieve list of collections
  collections_list=`curl -s -v -X GET  -H 'Content-type:application/json' "$SOLR_NODE1_PORT/admin/collections?action=LIST&wt=json" | jq '.collections | join(" ")' `;

  # Create/update schema
  mv solrconfig solrconfig.old.`datetimestamp`;
  unzip -d solrconfig solr-core-config.zip;

  # create myCollection
  cp myCollection_solrconfig.xml solrconfig/conf/solrconfig.xml;
  cp myCollection_schema.xml solrconfig/conf/schema.xml;
  $SOLR_HOME/server/scripts/cloud-scripts/zkcli.sh -zkhost "$SOLR_ZKHOST" -cmd upconfig -confname myCollection -confdir "solrconfig/conf/";
  if grep -q "myCollection" <<< $collections_list; then
    curl -s -v -X GET "$SOLR_NODE1_ENDPOINT/admin/collections?action=RELOAD&name=myCollection";
    echo "Updated myCollection";
  else
    curl -s -v -X GET "$SOLR_NODE1_ENDPOINT/admin/collections?action=CREATE&name=myCollection&numShards=2&collection.configName=myCollection&replicationFactor=3&maxShardsPerNode=2";
    echo "Created myCollection";
  fi

  rm -rf solrconfig;
  cd $currdir;
}

Post a Comment

Labels

Java (159) Lucene-Solr (110) All (60) Interview (59) J2SE (53) Algorithm (37) Eclipse (35) Soft Skills (35) Code Example (31) Linux (26) JavaScript (23) Spring (22) Windows (22) Web Development (20) Tools (19) Nutch2 (18) Bugs (17) Debug (15) Defects (14) Text Mining (14) J2EE (13) Network (13) PowerShell (11) Chrome (9) Continuous Integration (9) How to (9) Learning code (9) Performance (9) UIMA (9) html (9) Design (8) Dynamic Languages (8) Http Client (8) Maven (8) Security (8) Trouble Shooting (8) bat (8) blogger (8) Big Data (7) Google (7) Guava (7) JSON (7) Problem Solving (7) ANT (6) Coding Skills (6) Database (6) Scala (6) Shell (6) css (6) Algorithm Series (5) Cache (5) IDE (5) Lesson Learned (5) Miscs (5) Programmer Skills (5) System Design (5) Tips (5) adsense (5) xml (5) AIX (4) Code Quality (4) GAE (4) Git (4) Good Programming Practices (4) Jackson (4) Memory Usage (4) OpenNLP (4) Project Managment (4) Python (4) Spark (4) Testing (4) ads (4) regular-expression (4) Android (3) Apache Spark (3) Become a Better You (3) Concurrency (3) Eclipse RCP (3) English (3) Firefox (3) Happy Hacking (3) IBM (3) J2SE Knowledge Series (3) JAX-RS (3) Jetty (3) Restful Web Service (3) Script (3) regex (3) seo (3) .Net (2) Android Studio (2) Apache (2) Apache Procrun (2) Architecture (2) Batch (2) Build (2) Building Scalable Web Sites (2) C# (2) C/C++ (2) CSV (2) Career (2) Cassandra (2) Distributed (2) Fiddler (2) Google Drive (2) Gson (2) Html Parser (2) Http (2) Image Tools (2) JQuery (2) Jersey (2) LDAP (2) Life (2) Logging (2) Software Issues (2) Storage (2) Text Search (2) xml parser (2) AOP (1) Application Design (1) AspectJ (1) Bit Operation (1) Chrome DevTools (1) Cloud (1) Codility (1) Data Mining (1) Data Structure (1) ExceptionUtils (1) Exif (1) Feature Request (1) FindBugs (1) Greasemonkey (1) HTML5 (1) Httpd (1) I18N (1) IBM Java Thread Dump Analyzer (1) JDK Source Code (1) JDK8 (1) JMX (1) Lazy Developer (1) Mac (1) Machine Learning (1) Mobile (1) My Plan for 2010 (1) Netbeans (1) Notes (1) Operating System (1) Perl (1) Problems (1) Product Architecture (1) Programming Life (1) Quality (1) Redhat (1) Redis (1) Review (1) RxJava (1) Solutions logs (1) Team Management (1) Thread Dump Analyzer (1) Troubleshooting (1) Visualization (1) boilerpipe (1) htm (1) ongoing (1) procrun (1) rss (1)

Popular Posts