Topic 04: Three-Way Query Routing & Symbol Resolution

Target Duration: 2–4 minutes (~300–450 spoken words)
Focus: Pointwise verbal delivery covering zero-overhead rule-based query classification, regex cascade heuristics, exact symbol matching vs semantic vector fallback, and resolving identifier ambiguity.

🎙️ Pointwise Spoken Speech (Word-for-Word Delivery)

• Opening & Scope:
  "In this subsystem (sherlock/router.py), I built a three-way query router that dynamically classifies developer inputs into traceback diagnoses, exact symbol lookups, or semantic vector searches without incurring LLM latency."

• Step 1: The Cost of LLM Query Classifiers:
  "Many modern RAG frameworks use an LLM call or small classifier model to decide how to handle a query. In a local terminal CLI, making an LLM request just to classify a prompt adds 400 to 800 milliseconds of latency before retrieval even starts. To keep response times instantaneous, I designed a deterministic, zero-latency heuristic cascade in router.py:60 (route)."

• Step 2: Gate 1 — Traceback Detection:
  "First, the router evaluates whether the input is a runtime error. In looks_like_traceback() (router.py:17), it inspects the string for signature patterns like Traceback (most recent call last) or generic file-and-line patterns matching GENERIC_FRAME_RE. If matched, it immediately diverts to the stack frame extraction pipeline in diagnose()."

• Step 3: Gate 2 — Exact Identifier Symbol Matching:
  "Next, if the input is not a traceback, the router checks whether the query represents a valid programming identifier using the regex IDENTIFIER_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$") (router.py:9). If the user passes a single word like chunk_file or ChunkSchema, the router executes indexer.symbol_lookup(), which runs an exact SQL prefilter against LanceDB's name column in under a millisecond."

• Step 4: Gate 3 — Semantic Vector Search Fallback:
  "If the query is a multi-word natural language question (e.g. 'how are images cached?'), or if the identifier regex matched but no exact symbol with that name existed in the codebase, the router cleanly falls through to indexer.semantic_search(). This embeds the query and retrieves the top-k nearest neighbors via cosine vector similarity."

• Step 5: Resolving Query Ambiguity:
  "A critical design challenge was resolving ambiguous queries. For example, a query like 'divide' could mean 'find the function named divide' or 'explain division logic'. By establishing strict precedence—symbol lookup first with immediate fallback to semantic search—Sherlock delivers exact precision when the symbol exists while maintaining conversational flexibility when it does not."

📋 Step-by-Step Summary (What, How & Why)

🔍 Under-the-Hood Deep Dive: The Routing Cascade Logic

# The complete router decision engine in router.py
def route(repo_root: Path, query: str, k: int = 8) -> dict:
    query = query.strip()

    # 1. Error traceback detection
    if looks_like_traceback(query):
        return diagnose(repo_root, query, k=k)

    # 2. Bare programming identifier match
    if IDENTIFIER_RE.match(query):
        exact = indexer.symbol_lookup(repo_root, query)
        if exact:
            return {"type": "symbol", "query": query, "chunks": exact}

    # 3. Semantic vector search (natural language or symbol miss)
    results = indexer.semantic_search(repo_root, query, k=k)
    return {"type": "semantic", "query": query, "chunks": results}

💡 Tough Interview Questions & Detailed Answers

Q1: "Why prefer rule-based routing over an LLM function-calling agent?"

Answer: An LLM agent (like OpenAI Tools or LangChain Router) introduces three major problems for a local CLI: (1) Latency: an LLM round-trip takes 500–1500ms just to decide the search mode; (2) Cost & Hardware: it doubles the local inference load on Ollama; and (3) Non-Determinism: an LLM might misclassify an exact function name as a conversational prompt. Regex heuristics execute in under 0.1ms, are 100% deterministic, and cleanly handle 99% of developer workflows.

Q2: "How does the router prevent SQL injection in LanceDB queries?"

Answer: In indexer.py:165, symbol names are sanitized before being interpolated into LanceDB's SQL filter string: safe = name.replace("'", "''"). This escapes single quotes, ensuring that malicious or odd symbol names (e.g. operators or syntax strings) cannot escape the SQL string literal.