RAG Pipeline Builder

RAG Pipeline — Internal Docs QA at Northbeam Analytics

Corpus: ~8,000 internal docs (markdown + PDF), avg 1,200 tokens each, ~10M tokens total. Query pattern: factual ("What's our international travel policy?"), lookup ("Who owns the auth service?"), and light synthesis ("Summarize recent on-call rotation changes"). Stack: TypeScript, Vercel AI SDK, pgvector in existing Postgres, Claude Sonnet for generation. Multi-tenant: No.

Summary

Pipeline Stages

Stage 1 — Chunking

import { encode } from 'gpt-tokenizer'

export function chunkDocument(text: string, opts = { size: 512, overlap: 64 }) {
  const tokens = encode(text)
  const chunks: { tokens: number[]; start: number; end: number }[] = []
  let i = 0
  while (i < tokens.length) {
    const end = Math.min(i + opts.size, tokens.length)
    chunks.push({ tokens: tokens.slice(i, end), start: i, end })
    if (end === tokens.length) break
    i += opts.size - opts.overlap
  }
  return chunks
}

Stage 2 — Embedding

import OpenAI from 'openai'
const openai = new OpenAI()

export async function embed(texts: string[]) {
  const res = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: texts,
    dimensions: 1536,
  })
  return res.data.map((d) => d.embedding)
}

Stage 3 — Hybrid Retrieval

Vector search alone misses keyword-anchored questions like "What is the SAML-SSO setup URL?" — the literal token "SAML-SSO" is what matters, and embeddings smooth over it. BM25 alone misses paraphrased questions. Reciprocal Rank Fusion (RRF) on the two ranked lists is the simplest combiner that works.

Stage 4 — Reranking

import { CrossEncoder } from '@xenova/transformers'
const reranker = await CrossEncoder.from_pretrained('BAAI/bge-reranker-base')

export async function rerank(query: string, candidates: Chunk[], topK = 5) {
  const scores = await reranker.predict(candidates.map((c) => [query, c.text]))
  return candidates
    .map((c, i) => ({ ...c, score: scores[i] }))
    .sort((a, b) => b.score - a.score)
    .slice(0, topK)
}

Stage 5 — Generation

Claude Sonnet with a system prompt that enforces grounded answers and inline citations. The retrieval result is rendered as <doc id="123" path="...">...</doc> blocks, and the prompt requires every claim to cite at least one doc id.

Eval Metrics — 200-question Holdout

Tradeoffs and Open Calls

This sample illustrates the skill's output format. Northbeam Analytics is a fictional company used recurringly across these sample outputs. Real corpus statistics and eval results are never included in sample outputs.