Skip to content
ShipWithIan
RAG Q&A over Your Docs

Module 09 · ai · advanced

Lesson 09 · RAG Q&A over Your Docs

Goal

Build retrieval-augmented generation (RAG): instead of hoping the model knows your company's policies, you search your documents first and let the model answer from what you found. This is the #1 requested "AI for business" project.

Prerequisites

  • Module 08 (AI Agent, tools, memory, chat API)

How RAG works (in one minute)

  1. Ingest (once per document): split the text into chunks → turn each chunk into an embedding (a list of numbers capturing its meaning) → store chunks + embeddings in a vector store.
  2. Ask (every question): embed the question → find the chunks whose embeddings are closest → give those chunks to the model → the model answers from them.

The model never "learns" your documents. It reads the relevant pieces at question time. That's why answers can cite sources and why updating a document doesn't need retraining.

Concepts

  • Chunking: chunks that are too big dilute the match; too small loses context. 800 characters with 100 overlap is a good start for prose.
  • Embeddings model: must be the same for ingest and search (text-embedding-3-small in both places).
  • Simple Vector Store (in-memory): zero setup, perfect for learning. It's cleared when n8n restarts; production uses a persistent store (Exercise 2).
  • Retrieve as tool: the vector store becomes a tool the agent calls with its own search query, like lookup_order in Module 08.
  • Grounded answers: the system message forbids answering without search results and requires a citation. "I couldn't find that" is a feature.

Build it

Part 1: Ingest

  1. Webhook Ingest API: POST, path kb/ingest, respond via node.
  2. IF Has Text?: {{ String($json.body.text ?? '').trim().length }} Number ≥ 20. False → 400.
  3. Simple Vector Store Add to Knowledge Base: operation Insert Documents, memory key (By ID) course09_kb.
  4. Its sub-nodes:
    • Embeddings OpenAI Embeddings (ingest): text-embedding-3-small
    • Default Data Loader Document Loader: type JSON, mode Load Specific Data, data {{ $json.body.text }}, text splitting Custom, metadata source = {{ $json.body.source ?? 'unnamed' }}
    • Recursive Character Text Splitter (on the loader): chunk size 800, overlap 100
  5. Respond to Webhook: {{ { status: 'ingested', source: …, chunks: $input.all().length } }}

Part 2: Ask

  1. Chat Trigger Ask the Docs: public, hosted chat.
  2. AI Agent Docs Agent. System message rules: always call search_docs first; answer only from results; otherwise say "I couldn't find that in the documents."; quote exact numbers and names; end with [source: …]; under 80 words.
  3. Sub-nodes: OpenAI Chat Model (gpt-5-mini), Simple Memory (6), and a Simple Vector Store named search_docs in Retrieve Documents (As Tool for AI Agent) mode, same key course09_kb, limit 4, description "Search the company documents…", with its own Embeddings OpenAI (text-embedding-3-small).

Part 3: Load the handbook

Publish, then:

jq -n --rawfile t assets/sample-handbook.md '{text: $t, source: "handbook"}' \
  | curl -X POST http://localhost:5678/webhook/kb/ingest -H 'Content-Type: application/json' -d @-

Open the chat and ask "How many vacation days do I get?"

Test it

./test/test.sh                 # validation + chat page
WITH_CREDS=1 ./test/test.sh    # ingest handbook + 6 questions with known answers

Common problems

  • "I couldn't find that" for everything: nothing ingested since the last n8n restart (in-memory store), or the two memory keys differ.
  • Wrong or mixed-up answers: different embedding models for ingest and search, or chunks too large.
  • The agent answers without searching: strengthen "ALWAYS search first" in the system message and make the tool description specific.
  • Duplicate chunks: ingesting the same document twice adds it twice. Use Clear Store on insert while experimenting, or dedupe by source (Exercise 1).