Retrieval: Hybrid Search, RRF, LA Bias

TL;DR: Embed the query (1536-dim Azure OpenAI). Vector search top-20 via HNSW index. BM25 re-rank via tsvector ts_rank_cd. Fuse with Reciprocal Rank Fusion (k=60). Weight by freshness. Bias towards the user's local authority. Filter at 0.3 cosine similarity. Expand each surviving chunk with its ±1 neighbours. All in one SQL query.

Code: packages/api/src/chat/agentic-rag.ts (retrieve node, lines 1832–2100), chunk-expansion.ts, send-synonyms.ts, services/location.service.ts.

One-sentence overview

Retrieval is vector-first plus BM25 re-rank, not pure parallel fusion. BM25 on the full 450 000-chunk table takes 30–40 seconds on conversational queries (too many OR-tsquery matches for the GIN index to prune), so we pay for recall with HNSW (fast) and for keyword precision by re-ranking only the top-20 candidates.

The pipeline

Query embedding

Model: Azure OpenAI text-embedding-3-large, 1536 dimensions.

// packages/api/src/chat/agentic-rag.ts:386–393
const embeddings = new AzureOpenAIEmbeddings({
  azureOpenAIApiKey: config.azureOpenAIApiKey,
  azureOpenAIBasePath: `${config.azureOpenAIEndpoint}/openai/deployments`,
  azureOpenAIApiDeploymentName: config.azureOpenAIEmbeddingDeploymentName,
  dimensions: EMBEDDING_DIMENSIONS, // 1536
});

Embeddings are L2-normalised before use (langchain.ts:51–89). All-zero embeddings are rejected. Normalisation matters because cosine similarity on unnormalised vectors gives wrong rankings.

Caching

UserEmbeddingCache (Prisma model) caches query embeddings per user by SHA-256 hash with a TTL. Repeat queries within the window skip the Azure round-trip.

SEND synonym expansion

Before embedding, send-synonyms.ts expands colloquial parent language to legal terminology. Expansion is additive, both registers are preserved, so the vector captures semantic neighbourhoods of both.

Parent phrasing	Expanded terms
"kicked out"	exclusion, permanent exclusion, managed move
"statement"	EHCP, Education Health and Care Plan
"won't assess"	refusal to assess, Section 36
"1:1" / "one-to-one"	individual support, specified provision
"tribunal"	SEND Tribunal, First-tier Tribunal SEND
"annual review"	Section 44, EHCP annual review

Full list at send-synonyms.ts:35–151.

The SQL query (simplified)

The retrieve node issues a single raw SQL query that does vector search, BM25 scoring, RRF fusion, temporal decay, LA filtering, and validation-status filtering in one round trip. Paraphrased shape (the actual query is more elaborate):

WITH vector_search AS (
  SELECT id, content, source, la_code, school_urn, send_tags,
         chunk_index, parent_doc_id, created_at,
         1 - (embedding <=> $1::vector) AS similarity,
         ROW_NUMBER() OVER (ORDER BY embedding <=> $1::vector) AS vector_rank
  FROM knowledge_base
  WHERE embedding IS NOT NULL
    AND validation_status NOT IN ('rejected', 'superseded')
    AND [LA filter]
    AND [tag filter if topic tags present]
  ORDER BY embedding <=> $1::vector
  LIMIT 20
),
bm25_rescore AS (
  SELECT id,
         COALESCE(ts_rank_cd(search_vector, $2::tsquery), 0) AS bm25_score,
         ROW_NUMBER() OVER (ORDER BY ts_rank_cd(search_vector, $2::tsquery) DESC) AS bm25_rank
  FROM vector_search
),
fused AS (
  SELECT vs.*, br.bm25_score, br.bm25_rank,
         (1.0 / (60 + vs.vector_rank) + 1.0 / (60 + br.bm25_rank)) AS rrf_raw
  FROM vector_search vs
  JOIN bm25_rescore br USING (id)
)
SELECT *,
       rrf_raw * (0.8 + 0.2 / (1.0 + EXTRACT(EPOCH FROM NOW() - created_at) / 31536000.0))
         AS rrf_score
FROM fused
ORDER BY rrf_score DESC
LIMIT 10;

Key parameters, all tunable by env var:

• RAG_RRF_K = 60. Standard RRF constant.
• RAG_RELEVANCE_THRESHOLD = 0.3. Final cosine-similarity floor applied after fusion.
• RAG_TOP_K. Max chunks returned after fusion (implementation detail; configurable).

Why vector-first plus re-rank (not parallel fusion)

BM25 alone on the full 290 000+ knowledge_base returns OR-tsquery matches spanning about 28% of rows. Too many for a GIN index to prune efficiently. The query degenerates to a 30–40 second scan. Vector-first with HNSW returns the top-20 in about 200ms. BM25 re-ranks those 20 in milliseconds. Same signal, one to two orders of magnitude faster.

Local Authority bias

The user's local authority is detected by location.service.ts:getUserLocationContext() with a priority chain:

1. Child from the current conversation. child.schoolUrn → schools.laCode.
2. User's own school (professionals). user.schoolUrn → schools.laCode.
3. First child on profile (parents fallback). First user.children[].schoolUrn → schools.laCode.
4. Postcode lookup. PostcodeLaCache reverse-lookup if only postcode is available.
5. NULL. No LA context detected.

Output: { laCode, localAuthority, schoolUrn, homeLaCode, homeLaName, source }.

The SQL then applies LA bias in the WHERE clause:

AND (
  (la_code = ${laCode} AND school_urn IS NULL)        -- user's LA
  OR (la_code = ${homeLaCode} AND school_urn IS NULL) -- cross-border home LA
  OR la_code IS NULL                                  -- national docs always
  OR school_urn = ${schoolUrn}                        -- user's school docs
)

If laCode is null (no profile match), the filter narrows to la_code IS NULL: national docs only. If that returns no hits, a broadening fallback is triggered (noLaBroadFallbackUsed = true) and the response is prefixed with a disclaimer: "I don't have your school on file, so this guidance may not reflect your area's specific offer."

Cross-border placements matter: a child's home LA holds EHCP, transport, and social-care duty, which may differ from the school's LA. The filter includes both so guidance reflects the right authority for each type of query.

Temporal decay

rrf_raw * (0.8 + 0.2 / (1.0 + age_years))

Fresh documents (0 years old) keep their full score. One-year-old documents are weighted at about 0.9. Older documents asymptote to 0.8. Prevents ancient pages (superseded LA policies, older charity guidance) from dominating despite high similarity.

No explicit pre-2014 SEND Code filter. Superseded documents are excluded via validation_status NOT IN ('superseded') instead. See Knowledge Base: Validation.

Relevance threshold

Applied twice:

1. Retrieval floor (RAG_RELEVANCE_THRESHOLD = 0.15). Chunks below this are excluded from the top-20 set entirely.
2. Post-fusion filter (0.3). Only chunks meeting the 30% similarity floor reach relevantDocuments[], which is what the generate node sees as context.

School-specific documents get a lower floor (0.15) in a separate school-boost step: if the user has schoolUrn set, up to 3 school-policy chunks are injected even if they'd otherwise be below threshold. This ensures the user's actual school policy can be cited alongside national guidance.

Document grading: implicit, not LLM

There is no separate LLM grader node. "Grading" is implicit in the threshold filter. Earlier iterations used an LLM-judge grading step; we removed it because:

• It added 1–2 seconds per request.
• The hybrid search already weights for semantic and keyword alignment.
• The grounding pass (post-generation) catches fabricated citations better than pre-generation relevance grading.

Chunk expansion

After retrieval, each surviving chunk is enriched with its ±1 chunk-index neighbours from the same parent_doc_id. Code at packages/api/src/chat/chunk-expansion.ts:105–238:

1. Group surviving chunks by parent_doc_id.
2. Fetch neighbour chunks (chunk_index ±1) from the database in one query.
3. Concatenate: [original chunk content]\n\n[continued]\n\n[next chunk content].
4. Cap at 3 chunks per parent document.
5. Deduplicate (the neighbour of chunk N might already be chunk N+1 in the top-K set).

Gives the LLM more context without re-running retrieval. Legacy single-chunk documents (no parent_doc_id) pass through unchanged.

Controlled by CHUNK_EXPANSION_ENABLED (default true).

Topic pre-filtering

topic-routing.ts:getTagsForQuery() extracts SEND topic slugs (39 patterns: ehcp, tribunal, exclusion, annual-review, ebsa, ld, and so on) and maps them to knowledge_base.send_tags array values.

The retrieval SQL adds an AND send_tags && ARRAY[...]::text[] clause to narrow 450 000 chunks to the 10–50 000 that carry those tags. A material win for the HNSW search.

If the topic filter returns zero results, the node falls back to tag-less retrieval and sets topicFallbackUsed: true for observability.

Vector index configuration

packages/api/prisma/migrations/20260212100000_add_hnsw_vector_index/migration.sql:

CREATE INDEX knowledge_base_embedding_hnsw_idx
  ON knowledge_base USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

CREATE INDEX document_chunks_embedding_hnsw_idx
  ON document_chunks USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

• Index type: HNSW (not IVFFlat). Older docs and memory notes reference IVFFlat; that was an earlier iteration, the current migration is HNSW.
• m = 16. Connections per layer. Higher means better recall, slower build.
• ef_construction = 64. Build-time search width. Higher means better quality, slower build.
• Operator class: vector_cosine_ops. Aligns with the <=> cosine-distance operator.
• Result: about 700ms per query vs about 35 seconds sequential scan (50× improvement).

The HNSW index occupies about 2 GB. The 2026-03-05 dimension-change migration (1024 → 1536) drops and comments out the HNSW rebuild because re-embedding all chunks plus rebuilding the index at the same time caused Postgres OOM on the managed instance. Current procedure: drop index, re-embed all chunks, recreate the index with CONCURRENTLY.

Supporting indexes

• Compound (la_code, validation_status). Speeds the LA plus validation filter. Used on every retrieval.
• GIN on search_vector. The BM25 tsvector column.
• GIN on send_tags. For the topic filter.
• Compound (parent_doc_id, chunk_index). For chunk expansion lookups.

Metrics and observability

The retrieve node records:

• documentsFound, relevantDocuments: counts before and after threshold.
• topSimilarity, topBm25Score, topRrfScore: score distribution.
• vectorOnlyCount, bm25OnlyCount, hybridCount: how many hits came from each signal.
• topicFiltered, topicFallbackUsed: topic filter utility.
• laResultCount, nationalResultCount: LA coverage.
• noLaBroadFallbackUsed: boolean flag when the broad fallback was taken.

Persisted via the VectorSearchStats Prisma model. Aggregate queries in /admin/analytics render the retrieval-health dashboard.

AuditLog action la_coverage_miss fires when a query with known LA context returns zero LA-specific results. Useful for the LA curator to see where the knowledge base needs more coverage.

Security

• No SQL injection. Embeddings are numeric arrays. The tsquery is constructed via plainto_tsquery with user input as a bound parameter. All LA codes and topic tags are parameterised.
• Prompt injection from documents. Retrieved content passes through document-sanitiser.ts before prompt injection. See Safety & Grounding.
• Validation gate. validation_status IN ('approved', 'pending') excludes rejected and superseded content.

Configuration

Environment variables (defaults from code):

Var	Default	Effect
RAG_RELEVANCE_THRESHOLD	0.3 (post-fusion; 0.15 for school-specific boost)	Cosine similarity floor
RAG_RRF_K	60	Standard RRF fusion constant
CHUNK_EXPANSION_ENABLED	true	Expand chunks with ±1 neighbours
RAG_TOP_K	configurable via LLMSettings	Final chunk count after fusion
RAG_MAX_RETRIES	0	Disables query-rewrite retry loop

Further reading

• Knowledge Base. What's actually in the index, and how it got there.
• Agentic RAG. How the retrieved context flows into generation.
• Traces. See retrieval in action on real queries.