Say Hello To Precision How Rerankers And Embeddings Boost Search

North Mini Code. Cohere's first model for developers.

Learn more

Mar 25, 2024

12 minutes read

Say hello to precision with AI search

Explore how the latest generation of language models is powering search and retrieval for AI applications.

!Blog Post Featured Image

Perhaps you're using AI to transform your customer support capabilities, enabling users to effortlessly self-serve their questions. Or creating an innovative knowledge assistant designed to seamlessly answer queries about your company's confidential documents. However you may be using AI, it’s likely that access to accurate and relevant information and cost control are your top priorities. Central to initiatives like these is the power of search and retrieval — the ability to efficiently unearth the right information and promptly deliver answers quickly and at affordable costs.

Many of the world’s business applications rely on having powerful search capabilities. Now, the latest generation of language models is bringing new life to legacy search systems and delivering unparalleled performance, even for complex domains and multilingual executions.

In this article, we reveal the ins and outs of how models like Rerank and Embed can improve accuracy, relevance, and speed for legacy search tools. We also explore how these models are becoming the secret sauce that’s powering enterprise AI applications with retrieval-augmented generation (RAG).

Let’s start by looking at the various elements that make up a search and retrieval system that’s boosted with AI.

The Elements of Great Search

Enterprise search has traditionally relied on keyword-based search methods (e.g., BM25). Many companies have also adopted semantic search, which is a powerful way to search large databases using the semantic meaning of a query. It differs from traditional keyword search in that it focuses on understanding the meaning behind the words used in a search query, rather than just matching keywords. Using embeddings for semantic search can enhance the relevancy and precision of results.

Comparison of common search methods

Today, it’s never been easier to boost a legacy search system by combining it or replacing it with semantic search using new state-of-the-art language models. This article will focus on two of the approaches that many of our customers take:

1. Reranking: Most customers start by adding a reranker to the last stage of their search system. A reranker is a language model that computes relevance scores for retrieved documents. Using a reranker lifts search performance with minimal interventions or costs. It is a simple plug-in model that can dramatically improve the accuracy of legacy search systems and downstream AI applications. 2. Dense retrieval: Some customers choose to evolve their search systems to incorporate dense retrieval, which requires computing and storing embeddings for all of the documents in their corpus. This is a larger lift than solely implementing a reranker, but it can lead to better upstream retrieval.

These two methods, separately or combined, are driving vast improvements in search for all sorts of applications, including powering the search and retrieval steps for RAG applications for enterprise. Let’s dive into each one separately.

Workflow of semantic search with embeddings followed by a reranker

Boosting Search with Rerank

One of the fastest and easiest ways to boost search is by using a reranker. A reranker is a type of language model that computes a relevance score between a document and a search query. Rerankers can be applied to keyword, vector, or hybrid search systems. In all cases, adding a reranker tends to lead to improved performance. Rerankers can also be very quick to implement, with minimal interventions and costs. For example, Cohere Rerank can be added to a legacy search system with just a couple lines of code, improving results by as much as 50% based on academic benchmarks.

A reranker works as follows: for each query-response pairing, the model computes a relevance score, and these pairs are then ordered in descending order of their score. As the name hints, relevance scores are high for pairs in which the response is relevant to the query, and low otherwise. To implement, rerankers can be used in a variety of architectures and setups with or without a vector database.

Example of how Cohere Rerank works

Most applications with a search component will likely see performance improvements with a reranker. For example, a SaaS business that delivers workforce collaboration and productivity tools came to us because, like many search applications, their solution was frustrating customers by giving poor results and taking too much time to find the right answers. By implementing Rerank, they saw immediate improvements to search results which led to higher customer satisfaction.

Beyond improving legacy search, rerankers are also the fastest and easiest way to make a RAG pipeline better. RAG is a method of augmenting a generative model's natural language capabilities with specific and current information by connecting it to a knowledge base or proprietary datastore. Implementing RAG, though, can be complicated. For optimal performance, RAG requires powerful search capabilities that can handle analyzing large volumes of data across different sources quickly, efficiently, and reliably.

In most...