ADR-0010: LLM gateway, model routing and data residency &ZeroWidthSpace;

Status: accepted (2026-10-12)

Decision &ZeroWidthSpace;

All model calls go through @quavon/llm, which calls providers in-process using the Vercel AI SDK (Apache-2.0) with the OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Mistral and OpenAI-compatible providers. There is no separate LLM proxy service.
Provider keys are the tenant's llm_credentials (ADR-0007, encrypted with @quavon/crypto). They are decrypted for a single call and passed explicitly to the provider client; they are never written to another store, logged or returned by the API. Provider clients must never fall back to environment credentials (OPENAI_API_KEY, AWS_*, AWS_BEARER_TOKEN_BEDROCK, ...), so a tenant can never use the operator's credentials.
LiteLLM, vLLM, Ollama or any other gateway can still be used as an openai_compatible credential.

Revision 3 keeps keys per tenant in the UI. A proxy would need a second copy of every key in its own database, plus an extra Python service and database in every installation.
Budgets, cost accounting and routing need tenant and repository context that lives in Postgres anyway.
The LiteLLM PyPI package was compromised in a supply-chain attack in 2026. Fewer shipped services means a smaller attack surface.

Tiers: fast, standard, strong, embed. The tenant default lives in tenant_settings.settings.llm (TenantLlmPolicy). A repository can override tiers and the budget in repo_policies.model_tiers (RepoLlmPolicy); the repository value wins per tier.
A tier without a route is a configuration error (LlmConfigurationError); the feature stays off and the dashboard explains why.
Budgets: the tenant budget caps all calls of the tenant; a repository budget additionally caps the calls of that repository. Both are summed from llm_calls.cost_eur over a rolling period. When a cap is reached, the call is either blocked (BudgetExceededError) or downgraded along strong → standard → fast to the next tier that has a route; once no cheaper routed tier is left (always for fast and embed), the call is blocked. llm_calls stores both the requested and the effective tier. The check is advisory under concurrency (calls in flight can overshoot by at most their own cost).
Costs are in EUR from a versioned price table in @quavon/llm; a route can carry its own price (self-hosted or negotiated models). Unknown models cost 0 and are recorded with price_version null.

The tenant chooses providers and base URLs/regions, so residency follows the tenant's credentials (for example Azure OpenAI EU, Bedrock eu-*, Mistral EU). Quavon itself does not route data to any provider the tenant did not configure.
Edition enterprise (Quavon-hosted) rejects base URLs that resolve to loopback, private, link-local or metadata addresses, both when a credential is saved and when it is used. self_hosted allows private hosts so local models work.

Prompts and responses are never logged at info level. llm_calls stores counts, cost, model and a prompt hash. Raw prompts are stored only when the tenant enables debugPrompts, redacted, encrypted in llm_prompt_logs and deleted by the retention task after ttlHours.