Understanding Retrieval-Augmented Generation (RAG)

Before diving into implementation, let's understand what RAG is and why it's become such an important pattern in AI applications.

What is RAG?

Retrieval-Augmented Generation (RAG) is an AI architecture pattern that enhances large language models (LLMs) by retrieving relevant information from external knowledge sources before generating responses.

RAG Prompt Builder

User Query

Retrieved Documents

Source: mongodb-atlas.mdScore: 0.92

MongoDB Atlas provides multiple layers of security for your database: Network isolation with VPC peering, IP whitelisting, Advanced authentication, Field-level encryption, RBAC (Role-Based Access Control), and LDAP integration.

Source: security-overview.mdScore: 0.89

MongoDB Atlas Security Features include advanced authentication methods, network isolation, and encryption at rest and in transit.

Prompt Format:

BasicChat Messages

Include relevance scoresInstruct to use only provided contextInstruct to cite sourcesInstruct to admit when information is missing

The core workflow consists of:

1. Retrieval: Finding relevant information from a knowledge base
2. Augmentation: Incorporating that information into the context
3. Generation: Using an LLM to generate a response based on the augmented context

How RAG Works

1

User Query

The user submits a question, such as 'How does MongoDB Atlas Vector Search work?'

Why Use RAG?

RAG addresses several limitations of standalone LLMs:

1. Up-to-date information: LLMs are trained on historical data and don't know about recent events
2. Domain-specific knowledge: Standard LLMs lack deep expertise in specialized domains
3. Hallucination reduction: By grounding responses in retrieved facts, RAG reduces fabricated answers
4. Data privacy: Your proprietary data stays in your control rather than being sent to the LLM provider
5. Cost efficiency: Retrieval can reduce the context length needed for complex questions

Core Components of RAG

A typical RAG system has these key components:

1. Vector Database

Stores document embeddings and enables semantic search. MongoDB Atlas Vector Search provides this functionality with advanced features like:

• High-dimensional vector storage
• Approximate nearest neighbor (ANN) search algorithms
• Hybrid filtering (combining vector similarity with traditional queries)
• Horizontal scaling for large collections

Interactive Vector Search

Query

Apply Metadata Filter

// Basic vector search pipeline
const pipeline = [
  {
    $vectorSearch: {
      index: "vector_search_index",
      queryVector: embedding,
      path: "embedding",
      numCandidates: 100,
      limit: 5
    }
  },
  {
    $project: {
      _id: 0,
      content: 1,
      metadata: 1,
      score: { $meta: "vectorSearchScore" }
    }
  }
];

2. Document Processing Pipeline

Transforms raw documents into searchable chunks and embeddings:

• Document loading from various sources
• Text chunking strategies
• Embedding generation
• Metadata extraction

Interactive Document Chunking

Chunking Strategy

Max Chunk Size (chars)

Chunk Overlap (chars)

Sample Document{ "id": "sample-doc", "content": "# MongoDB Atlas Vector Search\n\nMongoDB Atlas Vector Search is a fully managed service that helps developers build vector search into their applications.\n\n## Key Features\n\nVector search in MongoDB Atlas provides several key capabilities:\n\n1. Integration with Atlas: Seamlessly works with your existing MongoDB Atlas deployment.\n2. Multiple similarity metrics: Supports cosine similarity, dot product, and Euclidean distance.\n3. Approximate nearest neighbor algorithms: Uses efficient indexing for fast searches.\n4. Hybrid search capabilities: Combine vector search with traditional MongoDB queries.", "metadata": { "source": "documentation", "type": "markdown" } }

Let's try implementing your own document chunking function:

Implement a Document Chunker

Complete the chunkDocument function to split the document into chunks using a sliding window approach

Your Code:async function chunkDocument(document, maxChunkSize = 500, overlap = 50) { const chunks = []; const content = document.content; // CODE_BLOCK_1: Your chunking logic here // Implement sliding window chunking return chunks; }

3. Retrieval Mechanism

Finds the most relevant information given a query:

• Vector similarity search
• Re-ranking for relevance
• Metadata filtering
• Results fusion (combining multiple retrieval methods)

Let's explore how vector search works with this interactive demo:

Implement Vector Search

Complete the vectorSearch function to perform a semantic search using MongoDB's $vectorSearch stage

Your Code:async function vectorSearch(query) { // Generate embedding for query const queryEmbedding = await getEmbedding(query); // CODE_BLOCK_2: Create the aggregation pipeline const pipeline = [ // Add the $vectorSearch stage here { $project: { _id: 0, content: 1, metadata: 1, score: { $meta: "vectorSearchScore" } } }, { $limit: 5 } ]; // Execute the pipeline const results = await collection.aggregate(pipeline).toArray(); return results; }

4. Augmentation Strategy

How retrieved content is added to the prompt:

• Document concatenation
• Structured formatting
• Relevance scoring
• Dynamic prompt construction

5. LLM Interface

The generation component that produces the final output:

• Prompt engineering
• Response generation
• Output formatting
• Fallback handling

The RAG Architecture with MongoDB Atlas

When building RAG with MongoDB Atlas, the architecture typically looks like this:

1. MongoDB Atlas serves as the vector database
2. Embedding models (like OpenAI's text-embedding-3-small) create vector representations
3. mongodb-rag library handles document processing and retrieval
4. LLM providers (OpenAI, Anthropic, etc.) generate the final responses

In the following sections, you'll implement each piece of this architecture to build a complete RAG system.

Vector Search Fundamentals

Before moving on, it's important to understand some key concepts about vector search:

Embeddings

Embeddings are numerical representations of text, images, or other data that capture semantic meaning. Similar concepts have similar vector representations, enabling "similarity search."

For example, these sentences would have similar embeddings:

• "The dog chased the ball"
• "A canine pursued a round toy"

Embedding Generator

Embedding Model

Text Inputs

Embedding Code

// Function to generate embeddings
async function getEmbedding(text) {
  const response = await embeddingModel.embed(text);
  return response.embedding;
}

// Function to get embeddings for multiple texts
async function getEmbeddings(texts) {
  const embeddings = [];
  for (const text of texts) {
    const embedding = await getEmbedding(text);
    embeddings.push(embedding);
  }
  return embeddings;
}

Vector Similarity Metrics

Different distance functions measure similarity between vectors:

• Cosine similarity: Measures the angle between vectors (1.0 = identical direction)
• Euclidean distance: Measures straight-line distance between points
• Dot product: Simple multiplication of vector components

Approximate Nearest Neighbor (ANN) Search

For large vector collections, exact search is inefficient. ANN algorithms like HNSW (Hierarchical Navigable Small Worlds) provide faster results with minimal accuracy trade-offs.

Let's Check Your Understanding

RAG Concepts Check

Question 1: What are the three main stages in the RAG workflow?

Vectorization, Augmentation, Generation

Retrieval, Augmentation, Generation

Retrieval, Analysis, Generation

Research, Augmentation, Generation

Try It Yourself

Now that you've learned about the fundamentals of RAG, try building your own prompt construction function:

Implement a RAG Prompt Builder

Complete the function to create a prompt that includes context from retrieved documents

Your Code:function createRagPrompt(query, retrievedDocuments) { // CODE_BLOCK_3: Format the retrieved documents into context // Create the prompt with the formatted context const prompt = `Answer the question based on the following context. If the context doesn't contain relevant information, say "I don't have enough information to answer that question." Context: ${context} Question: ${query}`; return prompt; }

Moving Forward

Now that you understand the core concepts behind RAG, let's set up your MongoDB Atlas environment to support vector search capabilities.