📘 Semantic Search

We can't have a technical event without talking about AI, ML, and LLMs these days. So, let's talk about semantic search.

What is semantic search?

Semantic search is a search technique that uses the meaning of words to find relevant results. It's what powers large language models that we see nowadays.

Using semantic search, we can find relevant results even if the search terms don't appear in the results. For example, if we search for "How to make a cake," we can find results that contain the words "How to bake a cake" or "How to make a pie."

This is done with vectors. Vectors are mathematical representations of words. They are used to find the similarity between words. For example, the word "cake" is similar to the word "pie" because they are both desserts.

How to create a semantic search engine

In our library application, let's change how our search bar works. Let's try to change the search bar behavior to find books based on their meaning, not just the words themselves.

This will help us find books such as "The Stand" when searching for "plague apocalypse."

Even if those words don't appear in the book title or description, we want to be able to find them.

You could use something similar to provide your customers with items that are similar to the ones they are looking at. For example, if a customer is looking at a pair of hiking boots, you could show them other equipment they might need.

Where does MongoDB come in?

With its document model, MongoDB is a great fit for storing vectors. You can store vectors as arrays of numbers in a document.

When the time comes to search for the relevant results, we can leverage the power of Lucene, just like we did for the full-text search.