Chapter 2

The Indexing Pipeline

How does raw source code become that clean graph, and stay correct? The rule (principle #2) is “always fresh”: the moment you commit, the map updates. Here’s the assembly line that makes it happen.

Explain like I’m 5 Every time you save your work (a “commit”), travsr quietly re-draws the parts of the map that changed, so the map is never out of date, and it never re-draws the bits that didn’t change.

It starts with a git commit

When you run travsr init, it installs a git hook - a tiny script git runs automatically after every commit. That script wakes the travsr daemon (a background helper), which figures out what changed and re-indexes only those files.

Analogy

Think of a city that takes a mini-census every time a building changes. It doesn’t re-survey the whole city, only the block that was renovated. Fast, and always current.

# .git/hooks/post-commit  (installed by travsr init)
#!/bin/sh
exec travsr hook-run --from-hook

crates/travsr-daemon/src/hook.rs

Watch the assembly line

Press play. This is the real sequence the daemon runs for each changed file.

commit → fresh graph

Press play to walk through the pipeline stage by stage.

The stages, explained simply

Find what changed (git diff)

The daemon asks git which files the commit touched. Only those enter the pipeline.

Skip unchanged files (SHA-256 hash)

Each file’s contents are hashed. If the hash matches what’s stored in the files table, the file is byte-for-byte identical and is skipped entirely. This is what makes re-indexing cheap.

crates/travsr-indexer/src/hash.rs

Phase A: Tree-sitter (fast structural pass)

Tree-sitter parses each file into a syntax tree at high speed and pulls out the obvious structure: functions, classes, methods, imports, and the “contains” relationships. Config and data files (JSON, YAML, TOML, XML) are parsed here too, as nodes with configures / external-dependency edges. No language server needed.

Analogy

This is the aerial drone survey: quick, covers everything, gets the shape of the city right.

crates/travsr-analysis · typescript.rs / rust.rs / python.rs / go.rs …

Phase B: LSIF / SCIP (precise semantic pass)

For supported languages, travsr can run real compiler-grade tools (rust-analyzer, scip-python, scip-typescript, scip-java, scip-clang…) that resolve exactly which function a call points to, even across files. 16 languages are in the catalog; the built-ins (TypeScript/JavaScript, Rust, Python) run natively in-process with zero install, the rest ship as downloadable “language plugins”, installed with travsr lang install.

Analogy

This is the ground survey crew: slower, but pinpoint-accurate about which road connects to which.

crates/travsr-indexer/src/lsif.rs · plugin-host (sidecar)

Unify the two passes

The scip_unifier merges Phase B’s precise symbols onto Phase A’s nodes, so you get both full coverage and accuracy. Edges carry a “provenance” marker recording which pass found them.

crates/travsr-indexer/src/scip_unifier.rs

Write to graph.db + recompute importance

New nodes and edges are written to SQLite in one batch. Then travsr recomputes k-core shells (Chapter 3) so it always knows which parts of the code are structurally central.

crates/travsr-store · crates/travsr-retrieval/src/kcore.rs

Keep the graph clean (GC + fsck)

When files are deleted or symbols renamed, their old nodes and edges become orphans. A tiered garbage collector reclaims them, and travsr fsck checks and repairs graph integrity on demand. A delete-trigger also logs removed nodes so the separate embedding database evicts their vectors, no stale answers, ever.

crates/travsr-cli/src/fsck.rs · travsr-store (GC tiers 1–2)

The big payoff

Because steps 1–2 only touch changed files, indexing after a normal commit is near-instant. The graph is never stale, and the AI never reads outdated structure.

Next: the algorithms that read the graph →