ADR-0012: Sandbox runner, signed queue and snapshot transport

Status: accepted (2026-10-13)

Decision

• Separate, pull-based runner. Repository code runs only in apps/sandbox-runner, a BullMQ worker on queue sandbox (prefix quavon-sandbox, attempts: 1, maxStalledCount: 0). Nothing connects into it, it has no HTTP port and shares no volume with the worker. It connects to Valkey as ACL user sandbox (sandbox keys only); the egress proxy uses the read-only user egress.
• Signed jobs and results. HMAC-SHA256 over domain + "\n" + JCS(job) (RFC 8785), key id = first 16 hex of SHA-256(key), current then previous keys, constant-time compare. Jobs carry notAfter (at most 15 min + 30 s skew ahead) and a replay guard (SET … NX); a rejected or replayed job becomes a failed run, never a retry. Results are signed with their own domain, carry runnerId, and the worker checks signature and job id first. Keys are per runner pool; future customer-run runners get their own pool, prefix, key and ACL user.
• Snapshot transport. Snapshots are deterministic gzip tars of regular files (no .git, links or devices; size and entry caps). Each blob is encrypted with AES-256-GCM in 1 MiB chunks under a per-blob HKDF key, a fresh IV per chunk and AAD bound to tenant, blob, chunk index and count, stored in Valkey with the job TTL and deleted by the runner after the job. Artefacts and capped log tails return in the signed result.
• One entry protocol for every backend. /opt/quavon/entry.ts (uid 65532) reads one JSON request line plus the base and overlay tars on stdin, runs an optional setup step (the only step with registry egress, via a per-job proxy token revoked when setup ends) and the run step, and streams a tar of exit.json, logs and requested artefacts. The runner parses that output as untrusted.
• Backends. docker-runsc (gVisor, platform systrap) is the default and works on hosts without KVM. firecracker is a second backend for bare-metal/KVM hosts and fails closed without /dev/kvm. docker-runc-insecure exists for local development only and is refused in production and the enterprise edition. Images are pinned by digest from images.lock.json; tags are refused. Container settings follow the threat analysis (§3): read-only root, all capabilities dropped, no new privileges, no network or the internal egress network, tmpfs work directories, pids/memory/CPU limits.
• Health. Each runner publishes a health record (backend, runtime, platform, images, host id, last escape smoke test) every 30 s; quavon doctor evaluates it.

Consequences

• Valkey carries encrypted repository snapshots for up to 15 minutes; Valkey memory must allow for the configured concurrency times the snapshot cap.
• Single-server installations run the runner next to the control plane on gVisor; a gVisor escape would reach that host. The hosted edition places runners on dedicated nodes.
• A Kubernetes-native backend (pods with a gVisor runtime class) is deferred.