Solr 4: Run Solr in SolrCloud Mode on your local file system.

S

Running Solr in SolrCloud

Mode

Local File System

version 4.10.3

German A Gutierrez - SolrCloud

Agenda

S Introduction

S SolrCloud High-Level Overview

S Live SolrCloud Demo

S Solr Directory Structure

S SolrCloud Parameters

S Solr Logging

S Sample SolrCloud Start-up script

S Resources


SolrCloud High-Level

Overview

S Scalable

S Fault Tolerant

S Data Redundancy

S Advance

analysis/tokenization

capabilities

S Automatic Load Balancing

S NoSQL Database


S Easy to integrate with new and existing systems.

S Requires very little effort to setup a small SolrCloudcluster.

S Runs on Linus, OS X and Windows

S Local File System

S Cloudera HDFS

S Hortonworks HDFS

SolrCloud High-Level

Overview


S Leaders

S Replicas

S Collection

S Solr Node

S Shard

S Core

SolrCloud Overview

S solrconfig.xml

S schema.xml

S solr.xml

S core.properties


Instructs Solr How to behave

Downloads

S Download Solr version 4.10.3

http://lucene.apache.org/solr/

S Download Zookeeper version 3.4.6

http://zookeeper.apache.org/

S Download sample SolrCloud scriptshttps://github.com/gutierrezga00/SolrCloud_LocalFileSystem

S Unzip Files preferably to your Desktop


http://lucene.apache.org/solr/

http://zookeeper.apache.org/

https://github.com/gutierrezga00/SolrCloud_LocalFileSystem

Live Demo


Zookeeper Configurations

S Create a “zookeeper” folder under your /tmp directory.

S Create a “data” sub-folder under your /tmp/zookeeper directory.

S Make a copy of the zoo_sample.cfg file and name it zoo.cfgLocation: zookeeper-3.4.6/conf/zoo_sample.cfg

S Modified the zoo.cfg file and set the following parameter names:

dataDir=/tmp/zookeeper/data

clientPort=9181


Start Zookeeper

S cd zookeeper-3.4.6/bin

S To Start Zookeeper enter the following command:

zkServer start

S To Stop Zookeeper enter the following command:

zkServer stop

S To clean/reset Zookeeper enter the following commands:

zkServer stoprm –rf /tmp/zookeeper/data/*zkServer start


Solr Directory Structure


Solr 4.10.3 DEFAULT

Solr 4.10.3 AFTER


German A Gutierrez - SolrCloudSolr 4.10.3 DEFAULT

Solr 4.10.3 AFTER



FINAL DIRECTORY STRUCTURE

SolrCloud Parameters

S ZOOKEEPER - zkRun vs. zkHost

zkRun: Solr Internal Zookeeper (Development Setup)

zkHost: External Zookeeper (Production Setup)



S numShards – Determines the number of shard the SolrCloudindex will be comprise across multiple machines (servers).

S bootstrap_confdir – SolrCloud cluster configuration files (e.g. schema.xml, solrconfig.xml, etc).

S collection.configName – SolrCloud collection name.

S solr.solr.home – SolrCloud binaries location (e.g. examplefolder).

S solr.data.dir – Shard location (HDFS or Local File System)



S host – Solr defaults to using the machine IP address. Using the “host” parameter you can overwrite the solr.xmlhost value within the <solrcloud> element.<str name="host">${host:}</str>

S jetty.port – port number a shard will be listening


SolrCloud Logging

S Solr controls all logging via the log4j.properties file.

solr4.10.3/example/resources/log4j.properties

S The property name “solr.log” by default is set to use the “logs/” directory within the Solrhome directory. We purposely removed the logs/ directory.

S Set the “solr.log” property name to make use of a passed in parameter value (e.g. solr.solr.logging)

Before:solr.log=logs/

After:solr.log=${solr.solr.logging}


SolrCloud Bash Script

S SolrCloud Parameters

java ${SOLR_JAVA_OPTS} \-Dbootstrap_confdir=${BOOTSTRAP_CONFDIR} \-DnumShards=${NUM_SHARDS} \-Dsolr.data.dir=${DATA_DIR}/${SHARD_DIR_NAME}$c \-Djetty.port=${JETTY_PORT} \-DSTOP.PORT=${JETTY_STOP_PORT} \-DSTOP.KEY=${JETTY_STOP_PORT}${JETTY_STOP_KEYWORD} \-Dsolr.solr.logging=${LOGS_DIR}/${LOG_DIR_NAME}$c/ \-jar start.jar &

java ${SOLR_JAVA_OPTS} \-Dsolr.data.dir=${DATA_DIR}/${SHARD_DIR_NAME}$c \-Djetty.port=${JETTY_PORT} \-DSTOP.PORT=${JETTY_STOP_PORT} \-DSTOP.KEY=${JETTY_STOP_PORT}${JETTY_STOP_KEYWORD} \-Dsolr.solr.logging=${LOGS_DIR}/${LOG_DIR_NAME}$c/ \-jar start.jar &


Resources

S Download presentation materialhttps://github.com/gutierrezga00/SolrCloud_LocalFileSystem

S Zookeeper:http://zookeeper.apache.org

S Solr:http://lucene.apache.org/solr

S Documentation:Solr Reference Guidehttps://www.apache.org/dyn/closer.cgi/lucene/solr/ref-guide


Q&A

S German Gutierrez

email: [email protected]


Solr 4: Run Solr in SolrCloud Mode on your local file system.

Technology