Skip to content

Code index ​

The code index keeps a searchable copy of each repository's default branch in PostgreSQL. Reviews use it to find similar code and the places that use a changed definition anywhere in the repository. The design is described in ADR-0017; repository settings are in configuration.md.

Modes ​

The mode is set per repository or in the organisation defaults (features.index.mode). It is off by default.

ModeWhat is storedNeeds
offNothing. An existing index is deleted within the hour.
lexicalFiles, definitions and chunks with a full-text search vector (to_tsvector('simple', …), GIN index).Any supported PostgreSQL
hybridAs lexical, plus one embedding per chunk.pgvector and an embeddings model for the tenant

The Code index tab of a repository shows the state (pending, building, ready, failed, storage limit reached), the indexed commit, file and chunk counts, the size, whether semantic search is available and why not, and the storage used by the organisation. Users with repo:manage can start a rebuild; the request is audited as repository.code_index.rebuild.

How the index is built ​

  • A push to the default branch queues an index job. Pushes to other branches and deleted branches are ignored. When the branch has already moved on and a ready index exists, the job is skipped as superseded.
  • The hourly maintenance task code_index queues a build for every repository with a mode other than off and no index, purges the index of repositories whose mode is off, and marks builds that run for more than an hour as failed. A failed build is tried again after 24 hours or with the next push.
  • The worker checks out the commit with the read-only token of the review app (the work tree is capped at 512 MiB), lists the files with git ls-tree, and reads only blobs it has not indexed before. Each blob is stored once per repository, so unchanged files cost nothing on the next push.
  • Files in TypeScript, JavaScript, Python and every other language with a grammar are split into one chunk per function, class or method (up to 80 lines; longer ones in windows). Other files and the code between definitions are split into windows of 60 lines that overlap by 10. Definition names are stored for definition lookup.
  • Not indexed: symlinks, submodules, files under node_modules, dist and vendor, lock files, minified files and source maps, binary files and files above the size limit.
  • Two ready snapshots are kept per repository, so reviews based on the previous commit still find their snapshot. Blobs that no snapshot uses any more are deleted after each build.
  • At most one build runs per repository; the worker runs one index job at a time by default.

Limits ​

Instance-wide limits are rows in the instance_settings table. Change them with SQL as the system role or an administrator; the worker reads them for every build.

KeyDefaultEffect
index.maxBytesPerTenant2147483648 (2 GiB)Sum of the indexed file sizes per organisation. A build that would exceed it fails with storage_cap and the state becomes Storage limit reached.
index.maxFilesPerRepository50000Files per snapshot; files beyond the limit (in path order) are left out.
index.maxFileBytes262144 (256 KiB)Larger files are left out. At most 1 MiB.
index.maxEmbedChunksPerRun20000Chunks embedded per build. The rest is embedded with the next build.
index.keepSnapshots2Ready snapshots kept per repository (1 to 10).
sql
insert into instance_settings (key, value) values ('index.maxBytesPerTenant', '10737418240')
on conflict (key) do update set value = excluded.value, updated_at = now();

Invalid values fall back to the default. A build also stops after 10 minutes of reading and chunking (time_budget).

pgvector ​

Semantic search needs the PostgreSQL extension vector (pgvector 0.8 or later). It is optional:

  • The init service (quavon db-init) runs with the admin role. When the server offers the extension, it runs CREATE EXTENSION IF NOT EXISTS vector and then applies the separate migration source packages/db/migrations-vector (tracked in quavon_meta.vector_migrations), which adds one partial HNSW index per supported vector size (384, 768, 1024 and 1536 as vector, 3072 as halfvec). pgvector is not a trusted extension, so the migrator role cannot create it.
  • The bundled PostgreSQL images (Compose and Helm) ship pgvector. For an external server, install the extension package first; managed services usually offer it as vector. Run the init service again after installing it.
  • Without pgvector quavon doctor reports QD207 (warning), hybrid repositories index lexically, no embeddings are computed and the Code index tab says "the PostgreSQL server does not have pgvector". Nothing else changes, and upgrades never fail because of it.

Embeddings are stored as real[], so the tables are the same with and without pgvector.

Embeddings and air-gapped installations ​

There is no default embeddings model. In the hybrid mode the worker embeds new chunks with the tenant's optional Embeddings tier (models.md). Any OpenAI-compatible server works, for example Text Embeddings Inference, vLLM or Ollama inside your network, so the code index needs no outbound connection (air-gap.md).

  • Calls are recorded with the feature index and count against the tenant's model budget. When the budget is used up, embedding stops and the snapshot stays ready with lexical search.
  • A model whose vectors have another size than 384, 768, 1024, 1536 or 3072 is refused (unsupported_dimensions).
  • After a model change the new model embeds the snapshot over the next builds; vectors of the old model are deleted once the new one covers the whole snapshot. Reviews compare a query only with vectors of the model that embedded it, so between the model change and the next build they search the index lexically.
  • The text sent is the definition name, the path and the chunk (at most 8,000 characters).

Sizing ​

Measured with 50,000 TypeScript files of 19 lines each (24.7 MB of source) on a developer laptop with PostgreSQL 17 in Docker:

Result
First build (lexical)86 s (about 1.7 ms per file)
Next build after one changed file3 s
Rows50,000 files, 200,000 chunks, 250,000 definitions
Table size including indexes289 MB: code_chunks 209 MB, code_symbols 48 MB, code_index_files 20 MB, code_blobs 12 MB
Removing the repository (cascade)0.8 s

Plan for PostgreSQL storage of up to 12 times the indexed source size in the lexical mode; repositories with many small files are at the upper end, because every chunk and definition has a fixed row and index overhead. A full build at the 50,000-file limit stays well inside the 10-minute budget. In the hybrid mode add per chunk 4 bytes times the vector size plus the HNSW index (for 1536 dimensions about 6 KiB per chunk and again about as much for the index). The storage limit counts the source size only.

Cross-repository context ​

Reviews can draw context from the code index of other repositories of the same organisation: links in .quavon/config.yml (same installation only), links an organisation admin adds on the Code index tab, and up to 3 organisation-wide pattern repositories (configuration.md). Links are stored in the tenant table repository_links; a link to a removed repository is deleted with it.

  • Nothing is fetched from the code host for a linked repository. The worker searches the latest ready snapshot of each linked repository, so linked repositories need their own index.mode and a finished build. Semantic search covers a linked repository only when its snapshot was embedded with the same model as the reviewing repository's query.
  • Linked repositories of a suspended installation, of another tenant and, for public and internal repositories, of a more private visibility are never used.
  • Each review resolves the links again (three small queries). At most 5 linked and 3 pattern repositories are used per review, and linked context takes at most 25 % of the context budget, so linking adds no model calls and only a bounded number of tokens.
  • The GET …/repositories/:repositoryId/links endpoint recomputes the statuses for the panel. It takes the configured names from the run statistics of the repository's most recent review (context_stats.rules.linkedRepositories), because the configuration file is not stored otherwise.

Data protection ​

The index stores source code in plaintext: chunk content and its search vector cannot be encrypted per column without losing full-text search. Protect the database volume with disk or volume encryption. The index tables are tenant tables with row-level security; they are deleted when the mode is switched off, when a repository is removed and when the tenant is deleted. They are not part of DSAR exports, because they hold repository content and no data linked to a person (data-inventory.md). Backups can leave them out (backup.md).

The open core is released under the Apache License 2.0.