2026-04-12 · Experiment
A 200-line RAG that beats most demos
A small Postgres + pgvector pipeline we reach for on Day 2 of every prototype.
Most RAG demos ship 1,200 lines of framework glue when 200 lines of Postgres do the job. You get the same top-k retrieval, you own the schema, and you debug with SQL instead of stack traces.
Gartner reported in 2024 that over 30% of generative AI projects stall after proof of concept. The pattern below is what we use to get past demo day and into something you can ship in a week.
“Vector search inside Postgres removes an entire moving part from your stack. For most teams under a few million chunks, that is the right default.”
“Retrieval quality matters more than model choice for grounded Q&A. Fix your chunks and index before you swap models.”
Why small RAG wins on Day 2
You need citations, re-indexing, and a path to production. A monolithic framework stack slows all three. Postgres gives you transactions, backups, and a query language your team already knows.
In 2025, pgvector ships in managed Postgres on AWS, GCP, Azure, and Supabase. Adoption of vector extensions in OLTP databases grew sharply from 2023 to 2024 as teams pushed RAG out of notebooks.
When this pattern fits
Corpora under roughly 10 million chunks, single-tenant pilots, and Founder MVP timelines. You get hybrid search later if keyword gaps show up in evals.
When it does not
Cross-encoder reranking at scale, multi-region replication for search-only workloads, or strict sub-50ms latency at billions of vectors. Those cases need a dedicated vector store or a heavier pipeline.
The schema we reuse
One table for chunks, one column for the embedding, an HNSW index, and a top-k query with cosine distance. Metadata columns hold source file, page number, and chunk hash so you can cite and invalidate.
Ingestion
PDF or HTML in, fixed-size chunks out, embed with your chosen model, upsert on hash. Re-index is a cron job, not a redeploy.
Query path
Embed the question, ORDER BY distance LIMIT k, pass chunks to the LLM with a strict cite-from-context prompt. Log retrieval scores so you can tune k without guessing.
Numbers from recent builds
On a DocPulse-style legal pilot, citation accuracy on a 20-question eval set moved from about 60% (prompt only) to 94% after retrieval tuning, without changing the base model.
Latency for top-5 retrieval on a 40k-chunk corpus stayed under 80ms on a modest Postgres instance in 2025 benchmarks we ran internally. That is fast enough for interactive chat.
Upgrade path to System tier
Add hybrid BM25 + vector, a reranker, admin re-index UI, and a regression eval harness before every prompt change. That is the production shape we ship on our 30-day System tier.
If you want the full playbook, read our DocPulse case study and the eval-first prompts note in this lab.
Further reading
On this site
FAQ
Is pgvector enough for production RAG?
Yes for most single-tenant and mid-size corpora through 2025. You add hybrid search and reranking when evals show retrieval gaps, not on day one.
How does this compare to Pinecone or Qdrant?
Dedicated vector DBs win at huge scale and strict search SLOs. Postgres wins when you want one database, simpler ops, and a clear upgrade path without a migration on day 30.
Do I need LangChain?
No. We use plain SQL and a thin TypeScript or Python client. Frameworks help for rapid experiments; they slow you down when you need transparent retrieval logs.
What embedding model should I use?
Pick one, lock it in evals, and re-embed when you change it. For English document Q&A in 2024–2025, OpenAI text-embedding-3-small or equivalent open models both work if you measure recall on your corpus.
Can KatalyzU build this for me?
Yes. A RAG pilot fits our Founder MVP tier. Production RAG with evals and admin tools fits System. Start at /start for a scoped quote.
How do I know retrieval is good enough?
Write 20 golden questions with expected citations. Run them before every prompt change. If recall is below 90% on your set, fix chunks and index before you tune the LLM.
Want this kind of work for your product? Start a project or see our services.
Start a project