Effective search is an essential component of a successful e-Commerce website. The latest research shows that people who use search are more likely to purchase products than those who are just browsing the products catalog. That's why I have been dealing with full-text search for years. If you live in the .NET ecosystem, then the obvious choice is to use Solr or Elasticsearch. Both of them are excellent scalable open-source options, but sometimes their usage is an overkill.

Few times a year, I need to build a full-text search functionality. While Solr or Elasticsearch are the right choices, bringing a standalone server is too much hassle. Another option is to use Amazon Elasticsearch Service, but sometimes I need a free solution. If the project uses SQL Server or MongoDB, you can use built-in full-text search indexes. That's an excellent option for small or medium-size projects, and I use it quite often.

Sometimes you can face the situation when the above options are not available. In such cases, I ended up building a full-text search microservice using Node.js and Lunr. Designed to be small, yet full-featured, Lunr enables you to provide a great search experience without external, server-side search services. I had been using it till the last month when I found the LunrCore project.

It is a port of Lunr to .NET Core. LunrCore is a small, full-text search library for use in small and medium-size applications. It indexes documents and provides a simple search interface for retrieving documents that best match text queries. Let's create a simple console application that filters out product names from the nopCommerce database. NopCommerce is one of the most popular shopping carts for ASP.NET that I am using quite a lot for e-Commerce clients. I will use Dapper to retrieve the data. I think this is still the fastest way if you like writing SQL.

First, let's add LunrCore and Dapper to our project.

dotnet add package LunrCore --version 2.3.8.5
dotnet add package Dapper --version 2.0.35

Now, let's create a Product model that we will get from the database and index.

class Product
{
    public int Id { get; set; }

    public string Name { get; set; }
}

The next step is loading all products from the database and indexing them. You can do it by using the Index class from LunrCore.

private static async Task<Index> IndexProducts()
{
    using var connection = new SqlConnection(ConnectionString);
    var products = await connection.QueryAsync<Product>("select Id, Name from dbo.Product");
    var index = await Index.Build(async builder =>
    {
        builder.AddField(new Field<string>("name"));

        foreach (var product in products)
        {
            await builder.Add(new Document
            {
                ["id"] = product.Id,
                ["name"] = product.Name
            });
        }
    });

    return index;
}

I want to add a few comments about this code. First, use the AddField method to create an index structure. My index has only one field called name. Second, the Add method accepts a document to index. A Document is an object implementing IDictionary<string, object> interface. The document must have a field called id. This field contains an entity identifier returned from the index if the document matches the search criteria.

Let's move on and implement search functionality. The index object has the Search method that returns a collection of matched documents.

var results = index.Search(query);

Each match object has DocumentReference, Score, and MatchData properties. MatchData contains the information about what term was found wherein the document. The Score property contains the document's relevance, and the DocumentReference includes the document's identifier. Keep in mind that DocumentReference is a string. In my example, it should be converted to an integer to be used in the SQL query.

var ids = new List<int>();
await foreach(var item in results)
{
    ids.Add(int.Parse(item.DocumentReference));
}

var products = await FindProducts(ids);

private static async Task<IEnumerable<Product>> FindProducts(IEnumerable<int> ids)
{
    using var connection = new SqlConnection(ConnectionString);
    var products = await connection.QueryAsync<Product>(
        "select Id, Name from dbo.Product where Id in @ids", 
        new { ids });
    return products;
}

LunrCore Query Syntax

• 'red' - find all documents that have a 'red' word in it.
• 'red apple' - find all documents with either a 'red' or an 'apple' word in it. The search terms are combined with OR.
• 'name:red' - find all documents with a 'red' word in its name field. The field names are defined in the AddField method's parameter.
• 'bla*' - the wildcards search. A wildcard is represented as an asterisk (*) and can appear anywhere in a search term. In this example, the term might be blank, blanket, or black.
• '+red +apple -green' - To indicate that a term must be present in matching documents, the term should be prefixed with a plus (+), and to indicate that a term must be absent, the term should be prefixed with a minus (-). In our example, the search algorithm will return all red apples, but won't give you green apples.

Index Persistence

Last but not least feature is index persistence. LunrCore can store the index in JSON format to a stream and load it back from the stream.

using var productsStream = File.OpenWrite(IDX);
await index.SaveToJsonStream(productsStream);
productsStream.Close();

using var stream = File.OpenRead(IDX);
index = await Index.LoadFromJsonStream(stream);
stream.Close();

Unfortunately, the only supported format is JSON, but there is an issue on GitHub to add additional serialization mechanisms later.

That's all I wanted to say about my experience suing LunrCore. Give it a try!