Configuring Apache Solr Multi-core With Drupal and Tomcat on Ubuntu 9.10
One of the things I've had to do recently is set up a development server that has multiple sites with Apache Solr so that each site has its own index in Solr. After searching the internets for information on how to do this, I found two sources that contained part of the process, but not put together in a way that worked for me.
- Nick Veenhof's blog post on setting up Solr on Ubuntu 9.10
- The "Apache Solr Multi-core Setup" handbook page on drupal.org (adapted from a post by Chris Rikli)
Nick's post got me through the Tomcat setup with Solr for one site, but his step for multi-site involved making copies of the solr.war file. I noticed Peter Wolanin's comment that it would be better to use the solr multi core feature to serve the 2 indexes at different paths, so I decided to try and implement it that way. I found the handbook page, but it was not written for Tomcat on Ubuntu, so I had to combine them. And then of course, I knew I had to blog about it.
All of these steps (except for a couple minor modifications) are copied from the two sources listed above. I am doing this on Ubuntu 9.10, the same as Nick.
So here we go. Please feel free to raise your hand if you have any questions.
Step 1. Installing Tomcat
sudo apt-get update sudo apt-get upgrade sudo apt-get install tomcat6 tomcat6-admin tomcat6-common tomcat6-user tomcat6-docs tomcat6-examples
This command downloads and install the following packages
tomcat6 : Servlet and JSP engine tomcat6-admin : Admin web applications tomcat6-common : Common files tomcat6-user : Tools to create user instances tomcat6-docs : Example web applications tomcat6-examples : Example web applications
Start tomcat by typing
/etc/init.d/tomcat6 start
Step 2. Security
If you are using IP tables and installing Apache Solr on an external server, modify or add the following line to accept the port 8080:
-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
After installation go to http://localhost:8080 or http://serverip:8080 in your browser. Now you should see the Tomcat welcome page.
PS: As noticed by Robert Douglass, be careful when opening ports in public servers. Apache Solr Does NOT provide security by default so anyone is able to post data towards your system. Best thing to do is to add another rule where you only accept 8080 connections from a certain IP address.
Step 3. Downloading Solr
Download apachesolr to your home directory for temporary reasons. Get it from one of the mirrors at http://www.apache.org/dyn/closer.cgi/lucene/solr/. To use the recommended mirror:
cd ~/ wget <a href="http://www.motorlogy.com/apachemirror/lucene/solr/1.4.0/apache-solr-1.4.0.tgz tar" title="http://www.motorlogy.com/apachemirror/lucene/solr/1.4.0/apache-solr-1.4.0.tgz tar">http://www.motorlogy.com/apachemirror/lucene/solr/1.4.0/apache-solr-1.4....</a> -zxvf apache-solr-1.4.0.tgz
Step 4. Linking Tomcat6 with Apache Solr application
This should give you an idea on where your distribution installed tomcat6.
Attention: If your path is different, do not forget to also adjust this in the next steps. whereis tomcat6 should show you tomcat6:
/etc/tomcat6 /usr/share/tomcat6
Copy the solr.war file to the webapps directory:
sudo cp apache-solr-1.4.0/dist/apache-solr-1.4.0.war /usr/share/tomcat6/webapps/solr.war
Copy the example solr application to a new directory called solr. We will change this example solr application later on to be viable for Drupal 6:
sudo cp -R apache-solr-1.4.0/example/solr/ /usr/share/tomcat6/solr/
Create our config file
sudo nano /etc/tomcat6/Catalina/localhost/solr.xml
And fill it with the following configuration :
<Context docBase="/usr/share/tomcat6/webapps/solr.war" debug="0" privileged="true" allowLinking="true" crossContext="true"> <Environment name="solr/home" type="java.lang.String" value="/usr/share/tomcat6/solr" override="true" /> </Context>
Step 5. Managing Tomcat6 application
We want to be able to see how and/or if our Solr application is running, and we can do this by using the manager application. By default you don't have access to this application so we have to modify the permissions:
sudo nano /etc/tomcat6/tomcat-users.xml
And modify it so it more or less reflects the same information as shown here.
<tomcat-users> <role rolename="admin"/> <role rolename="manager"/> <user username="nick" password="ateneatech" roles="admin,manager"/> </tomcat-users>
Drop Tomcat security so Solr can access /usr/share/tomcat6/solr:
sudo nano /etc/default/tomcat6
And modify it so our security is disabled. Be careful if you are running on a server which you do not control 100%!
TOMCAT6_SECURITY=no
We'll restart Tomcat after we got Solr multi-core installed.
Step 6. Linking Drupal 6 with a running Apache Solr
I assume you have Drush installed, so we continue with downloading the apachesolr module. Execute this command in the designated website:
drush dl apachesolr
You will also need to get the Solr PHP Client library (available at http://code.google.com/p/solr-php-client/downloads/list) and unpack it so it is under the apachesolr module directory as SolrPhpCLient (i.e. sites/all/modules/apachesolr/SolrPHPClient)
Let's copy our schema files that will customize our Apache Solr instance so it fits the "Drupal" bill.
sudo cp ~/apachesolr/schema.xml /usr/share/tomcat6/solr/conf/schema.xml sudo cp ~/apachesolr/solrconfig.xml /usr/share/tomcat6/solr/conf/solrconfig.xml
Step 7. Set up up the multi-core functionality
First, we copy the solr.xml that is configured for mutli-core to the /solr directory:
sudo cp ~/apache-solr-1.4.0/example/multicore/solr.xml /usr/share/tomcat6/solr/solr.xml
Now create a directory within the /usr/share/tomcat6/solr directory for each site you want to use Solr with. For example, if I had two sites "site1.com" and "site2.com" and I wanted to use Solr with them both, I might create the following directories:
/usr/share/tomcat6/solr/site1 /usr/share/tomcat6/solr/site2
Copy the /usr/share/tomcat6/solr/conf directory into each directory you just created. When you're done, each site's directory should have a copy of /conf in them:
sudo cp /usr/share/tomcat6/solr/conf /usr/share/tomcat6/solr/site1/conf sudo cp /usr/share/tomcat6/solr/conf /usr/share/tomcat6/solr/site2/conf
Note: DO NOT delete the /usr/share/tomcat6/solr/conf directory. Even though there is a copy for each site, it is still needed by the Solr core. For stock Solr, this directory can be deleted, but it is needed when running it in Drupal.
Now alter the solr.xml file you copied over to list both sites. Your solr.xml should look like this:
<?xml version="1.0" encoding="UTF-8" ?> <solr persistent="false"> <cores adminPath="/admin/cores"> <core name="site1" instanceDir="site1" /> <core name="site2" instanceDir="site2" /> </cores> </solr>
OK, let's restart our Tomcat service:
sudo /etc/init.d/tomcat6 restart
Surf to http://localhost:8080/manager/html and log in with your username and password from above to check if the Solr instance is started. If not, start and it and check wether or not you receive an error code. If your application is started, surf to http://localhost:8080/solr/admin and you should see a nice screen!
Step 8. Configure the Drupal modules
Enable the "Apache Solr framework" and "Apache Solr search" modules. Also, enable the core Drupal Search module if you haven't already. Do this for both sites.
Navigate to the Solr Settings page for each site (admin/settings/apachesolr/settings):
- Solr host name: localhost
- Solr port: 8080
- Solr path for site1.com: /solr/site1
- Solr path for site2.com: /solr/site2
Under "Advanced Configuration", enabled the "Make Apache Solr Search the default" option if you want to replace the default search with Solr (and I assume you do).
Click "Save Configuration." The first time around it'll probably tell you it can't reach the server, but if you refresh the page you'll be good to go.
Now, whenever you add a new site, you will just need to do the following steps to enable Solr search:
- Copy /usr/share/tomcat6/solr/conf to /usr/share/tomcat6/solr/mysite
- Add the site to /usr/share/tomcat6/solr/solr.xml
- Configure Solr settings in Drupal at admin/settings/apachesolr
And that's it. Happy searching!






Yay, happy to see somebody
Yay, happy to see somebody writing a good post about this - took me some time figuring this out by myself and (sadly) I never got to writing a post like this.
Something some people will need to know: in the last step, the "solr hostname" should be "nick:ateneatech@localhost"
Cheers for this tutorial! I'm sure it will help a lot of people!
Hmmm, not in my experience. I
Hmmm, not in my experience. I have this running on my local laptop as well as an Ubuntu server, and in each case, the value for "Solr host name" is just "localhost."
Thanks for writing this up! I
Thanks for writing this up! I was willing to do this but had some time shortage!
Also I included your post in the multiple solr part of my tutorial.
Thanks again!
I`ll try this on my weekend.
I`ll try this on my weekend. Thanks for tutorial.
Great write up Steve. It
Great write up Steve. It inspired me to finally blog my notes on multi core solr with Ubuntu 10.04 (aka Lucid Lynx).
While I was at it I wrote a custom core admin handler so I can have Solr automagically create cores on the fly. The post is available at http://davehall.com.au/blog/dave/2010/06/26/multi-core-apache-solr-ubunt...
Execelent post. There is
Execelent post.
There is only one thing: I'm missing the part about the config file for solr (/etc/tomcat6/Catalina/localhost/solr.xm).
Oops, good catch. I updated
Oops, good catch. I updated the post.
I can't see the update in the
I can't see the update in the text body: did you put it in?
For other readers' reference, the XML file needs to live at /etc/tomcat6/Catalina/localhost/solr.xml (not solr.xm) and here's information about what should go in it: http://www.nickveenhof.be/blog/setup-drupal-6-apache-solr-tomcat6-and-ub...
There's also some typos of tomcast6 and /mysite1, /mysite2 that people might stumble across.
Finally there.
Finally there.
Thank you so much for the
Thank you so much for the great write up. It really helped!
great Post, but I also miss
great Post, but I also miss the configuration part with the Catalina path (/etc/tomcat6/Catalina/localhost/solr.xml). Is this not necessary or why did you not mentioned it above?
If you get an error 500 and
If you get an error 500 and something like "can´t find solrconfig.xml in classpath", you´ll have to uncomment the setting for the environment variable in tomcat/webapps/solr/WEB-INF/web.xml like:
solr/home
/put/your/solr/home/here
java.lang.String
I tried all of the steps
I tried all of the steps above, but still get an error 404 - The requested resource (/solr/admin) is not available when I try to access localhost:8080/solr/admin.
Tomcat is working fine (I can access the manager), but am having no luck getting solr to appear. Any troubleshooting tips?
it is showing me the sites
it is showing me the sites which i have coded in solr.xml
<?xml version="1.0" encoding="UTF-8" ?>
only,
- Solr host name: localhost
- Solr port: 8080
not display the lines below there
- Solr path for site1.com: /solr/site1
- Solr path for site2.com: /solr/site2
- Solr path for site3.com: /solr/site3
If you've gone through this
If you've gone through this tutorial and Drupal is telling you that it can't connect to Solr, you may have a permissions issue. Sniff around in /var/logs/tomcat6, particularly catalina.out and look for issues. Tomcat may not have the write permissions necessary to create the index within each core.
Another giveaway is if you're seeing the message "missing core name in path" when you try to view the Solr instances from the Tomcat Manager (http://localhost:8080/solr)
I had to execute the following to get things working on Ubuntu 10.04LTS:
Set the solr directory to be part of the tomcat6 group
sudo chgrp -R tomcat6 /usr/share/tomcat6/solrChange the read/write perms in the solr directory
sudo chmod -R 2750 /usr/share/tomcat6/solrMake each site's folder writable
sudo chmod -R 2770 /usr/share/tomcat6/solr/site1Thanks for the helpful tips,
Thanks for the helpful tips, Chris. The last time I did this setup, I ran into some permissions problems just like that, but didn't think to update this post.
Nice how to and very detail
Nice how to and very detail articles.
still get some error in my end but i try to do review it from above
thanks :)
Thanks for great detailed
Thanks for great detailed post.
I got up and running tomcat but not SOLR, i think there are some issues with solrconfig.xml, but dont know how to fix this. I am getting following error. Can you please help
HTTP Status 500 - Severe errors in solr configuration. Check your log files for more detailed information on what may be wrong. If you want solr to continue after configuration errors, change: false in solr.xml ------------------------------------------------------------- java.lang.RuntimeException: org.apache.lucene.index.IndexNotFoundException: no segments* file found in org.apache.lucene.store.MMapDirectory@/usr/share/tomcat6/solr/site1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a4133b0: files: [] at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1099) at org.apache.solr.core.SolrCore.(SolrCore.java:585) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:463) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:316) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:207) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:130) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:94) at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:295) at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(Application
and 10-15 more lines
Attribution, please. It's
Attribution, please. It's kind of obvious that much of this was paraphrased or plagiarized from the article I published on the Four Kitchens wiki.
Specifically, from the Four Kitchens page:
"On Ubuntu, drop Tomcat security so Solr can access /var/solr."
And on here:
"Drop Tomcat security so Solr can access /usr/share/tomcat6/solr:"
You just don't get such similar phrasing by coincidence.
Thanks,
David
David, I made it very clear
David,
I made it very clear at the beginning of the article where I got my information, and even linked to them. If you go back and read the two articles, you will see that everything was taken from there. I'm not sure how much clearer I could have been. In particular if you go back and look at Nick's blog post, step 5 has the (trademarked?) phrase. And, if you look at the "Sources" section at the bottom of Nick's post, you will see that he specifically lists your wiki as one of his sources.
So, am I supposed to provide attributions for Nick's post, too?
Steve
Post new comment