Full text indexing with Rails 3, Solr and Sunspot

Searching using SQL LIKE seems very outdated these days. We have many full-text indexing solutions: lucene, sphinx, solr and others. Since Heroku supports only solr let’s see how to use it. Luckily it has a pretty good documentation. I’ll just cover the basic though, for more nice features such as facets, highlighting and more refer to the official wiki and documentation.

Setup

Add the required gem to your Gemfile:

gem 'sunspot_rails', '~> 1.2.rc4'

Create the necessary files:

$ rails g sunspot_rails:install

Add solr/data to your .gitignore file. Launch the server, which happens to be packaged with the gem:

$ rake sunspot:solr:start

Model configuration

You can define the indexed fields with the searchable method. Simple example (taken, as most of this writing, from the sunspot wiki):

class Post < ActiveRecord::Base
  searchable do
    text :title, :default_boost => 2
    text :body
  end
end

Sometimes you might want to have some custom logic to define an attribute value. Blocks can then be useful:

text :categories do |post|
  post.categories.map {|cat| cat.name }
end

Field types

text: the most common field type, the one you search on with keywords
string: similar to text, but it isn’t tokenized so you cannot run full-text searches on it
integer, long, float, double: you can guess them
time, date
trie: particular numeric type. Basicly it splits a number in its factors to make range searching faster

Options

default_boost: relevancy multiplier. Search results are ordered by a computed relevancy value. With this option you can increase the importance of single attributes search-wise
using: sunspot gets the data to be indexing by calling the method itself of the words. If the method has actually a different name you can specify it with

Reindexing

The plugin is smart enough to update the index when the model is saved, updated or destroyed. If you want to force reindexing run

$ rake sunspot:reindex

You will have to reindex whenever you’re changing the searchable definition, though.

Performing a search

Let’s start with a simple keyword search:

@search = Post.search do
  keywords 'foobar'
end

You can access the results objects in many ways:

@search.hits: return an array of Hit objects - the searched object plus some search metadata such as score
@search.results: returns directly the searched objects
@search.each_hit_with_result{|hit| ...}

Ordering

By default it is on relevancy, but you can specify other orderings:

Sunspot.search(Post) do
  order_by(:blog_id, :asc)
  order_by(:created_at, :desc)
end

Pagination

Similar to will_paginate’s pagination. You can have something like this:

def search_with_solr(limit = 50)
   s = Product.search do
     keywords self.q
     paginate :page => 1, :per_page => limit
   end
   s.results
 end

Testing

To stub out RSOL add this line in a sunspot.rb initializer file:

Sunspot.session = Sunspot::Rails::StubSessionProxy.new(Sunspot.session)