Target Duration: ~60–90 seconds (~190–240 spoken words)
Tone: Confident, structured, systems-focused, and natural.
Goal: Deliver a crisp, memorable overview connecting recursive token-aware chunking, dual-indexed PostgreSQL storage, hybrid RRF search, cross-encoder neural reranking, and citation-grounded LLM synthesis.
“I built a Retrieval-Augmented Generation (RAG) assistant for querying PostgreSQL 16 documentation. Rather than using high-level wrapper frameworks, I implemented the pipeline myself to maintain complete control over database indexing, retrieval algorithms, and latency.
The project centers on three core engineering components.
First, document ingestion. I built an automated pipeline that extracts structured sections from documentation, recursively splits them into 512-token chunks, and generates 384-dimensional embeddings using BAAI/bge-small-en-v1.5. Everything is stored in a single PostgreSQL instance using pgvector with an HNSW index for semantic similarity, and PostgreSQL full-text search with a GIN index for keyword matching.
Second, hybrid retrieval and reranking. Vector search is great for conceptual questions, but fails on exact syntax flags, error strings, and configuration keys like shared_buffers. I run vector and full-text search together, merge their candidate lists using Reciprocal Rank Fusion (RRF) to bridge score scale differences, and then pass the top candidates to a cross-encoder for neural reranking to isolate the most relevant excerpts.
Finally, citation-grounded generation. I feed the top passages to Gemini with strict instructions to answer strictly from context, cite every claim using [Source N], and fall back explicitly if information is missing. A regex-based extraction layer parses these citations and maps them directly back to documentation sections and URLs.
Through this project, I gained deep hands-on experience in database vector indexing, hybrid ranking math, and evaluating end-to-end RAG quality.”
ts_rank_cd ($[0, \infty)$) cannot be added directly, leading into rank-based scoring math ($\sum \frac{1}{60 + \text{rank}}$)."Represent this sentence...") while documents are indexed bare.[Source N] markers back to chunk metadata and verifies groundedness.SET hnsw.ef_search.”
-rf or -i), and exact configuration parameters (such as PostgreSQL's shared_buffers or wal_level). An embedding might treat kill -9 and kill -15 as almost identical vectors, but their operational semantics are completely different. Full-Text Search with GIN indexes guarantees exact lexical matching for technical symbols.”
BAAI/bge-small-en-v1.5 for three critical systems and algorithmic reasons:
all-MiniLM-L6-v2) and rivals 768-dim models while containing only ~33M parameters (~130 MB on disk).'Represent this sentence for searching relevant passages: ') bridges the semantic gap between short user queries and expository technical manuals.all-MiniLM-L6-v2 (384-dim): Fast and compact, but lacks modern hard-negative contrastive tuning, yielding lower retrieval recall on technical documentation.bge-base-en-v1.5 (768-dim) / bge-large (1024-dim): Offers only marginal +1–2% gains on MTEB, but doubles/triples HNSW RAM consumption and vector math latency with diminishing returns, since our downstream cross-encoder already fixes fine-grained precision.intfloat/e5-small-v2 (384-dim): Strong alternative, but requires prefixes on both queries ('query: ') and stored documents ('passage: '), bloating database storage and coupling stored data to the embedding model. BGE stores passages clean.OpenAI text-embedding-3-small (1536-dim): Proprietary cloud API that introduces 100–300 ms network round-trip latency, ongoing API billing, and breaks offline/local terminal CLI operation.