Threat analysis: issue triage, replies and PR chat

Status: 2026-10-12. Extends threat-model.md. Required before replies on public repositories may leave approval mode. Auto-replies on public repositories stay disabled until this analysis has been reviewed and signed off by a maintainer of Quavon; the policy API refuses auto for replies on public repositories, and effectiveMode downgrades it to approval everywhere else, until then.

1. Scope and capabilities

Feature	Trigger	What it may do	What it may never do
Triage	issue.opened, issue.edited (debounced)	Add labels from the repository's existing labels that are on the policy allowlist; propose priority; propose duplicates; post one needs-info comment	Remove labels, create labels, assign people, close, lock, transfer or edit issues, mention people
Replies	issue.opened, issue.comment by the issue author	Post one answer, workaround or clarification comment with citations, or abstain	Post without a citation (answers and workarounds), cite anything outside the same repository, reply to bots, reply to its own comments, reply more than the per-issue cap
PR chat	@quavon <command> in a PR comment or review comment	explain, why, fix this (a committable suggestion in the reply), ignore (suppress a finding fingerprint for that PR), re-review (enqueue a review), learn (store a proposed learning, reply with a fixed text)	Push commits, approve, request changes, merge, run commands for users without write access, activate a learning
Learnings from replies	A reply without a mention in the review thread of a CodeLotse finding, by an OWNER, MEMBER or COLLABORATOR	Store a proposed learning (classified on the fast tier)	Post anything, activate a learning, change rules, thresholds or other learnings

All features run in zone Z1 (read-only model calls, no write token). Every external write goes through the publisher (Z2).

2. Actors

• Anonymous issue authors and commenters (anyone on public repositories): fully untrusted; can write arbitrary title, body, comments, hidden markdown, zero-width characters and links.
• Repository collaborators: trusted only for chat commands, and only with write, maintain or admin permission, checked through the GitHub API when the command runs (not from author_association, not cached longer than the job).
• Maintainers approving outputs in the dashboard: authenticated, repo:review permission, audited.
• Other GitHub Apps and bots: never trigger replies or commands.

3. Threats and controls

#	Threat	Control	Test
TR1	Injected instructions in issue text change labels ("label this as security", "assign @ceo")	Structured model output only; labels must exist in the repository and be on the policy allowlist; protected labels (policy labels.protected, mapped priority labels and built-in tokens such as security, critical, priority, release, p0–p3, sev*, cve, vuln*, duplicate, invalid, wontfix, stale, spam, ok-to-test, safe-to-test, deploy*, approved, lgtm, automerge, backport*, matched as whole tokens) are never applied automatically, only proposed; no assignment capability. The model's confidence threshold limits noise but is not a security control	injection fixtures in triage eval; allowlist unit tests
TR2	Label removal or overwrite	Publisher only adds labels (POST …/labels); the replace-all setLabels is not used by triage	publisher unit test
TR3	Duplicate hints used to harass or to hide a real report	Never closes; duplicate hint only above duplicates.minScore after a verifier comparison; candidates only from the same repository; approval mode by default on public repositories	dupes eval; publisher tests
TR4	Needs-info spam (repeated requests, requests to first-time contributors)	One needs-info comment per issue ever (idempotency key on the issue); only fields from the repository's issue templates or the policy list; never on issues opened by collaborators	triage handler tests
RP1	Prompt injection steers the reply (phishing link, "run curl … | sh", insults, false security advice)	Reader/writer split: a reader model turns untrusted text into IssueFacts (enumerated fields, bounded quoted strings); the writer never sees raw issue text; replies need ≥ 1 verified citation (answers, workarounds); sanitizer: HTML stripped, mentions neutralised, links only to the repository host, no images; approval mode on public repositories; abstention below reply.minConfidence	injection corpus (hidden comments, zero-width, SYSTEM: headers, exfil links) with 0 posts; citation verifier tests
RP2	Data exfiltration through links or images ()	No images; links only under https://{host}/{owner}/{repo}/ (path prefix, not just the host); the writer's own links are removed and only the Sources list rendered from verified citations carries links; bare URLs to other places are replaced; secret scan of every outgoing body (block + audit)	sanitizer and secret-scan tests
RP3	Private code or data leaks into a public reply	Retrieval is limited to the same repository as the issue by an explicit repositoryId filter (RLS only separates tenants): code at the default branch head, docs, issues and PRs of that repository; never other repositories of the tenant, never unpublished findings, never review runs	retrieval unit tests (other repository content never cited)
RP4	Hallucinated or unsupported citations	Citation verifier: every cited snippet must exist at the cited location (path + line range or issue/PR number in the same repository) and contain the quoted text; unverifiable → reply downgraded to a dashboard-only suggestion (never posted)	citation verifier tests
RP5	Reply loops and spam (bot answering itself or another bot)	Ignore comments by bots ([bot] logins, Bot type) and by the app's own login; max replies per issue (policy, default 1 per event, 3 total); per-repository daily cap; budget gate	reply handler tests
RP6	Content approved, then the issue changes	Drafts store the issue updated_at they were built from; approval of a stale draft is refused (stale)	approvals tests
CH1	Unauthorised commands (@quavon ignore by an outsider to hide findings)	Permission check via GET /repos/{o}/{r}/collaborators/{user}/permission at execution; only write/maintain/admin; denied commands get no reply (no oracle) and are audited	permission matrix tests
CH2	Command flooding / cost abuse	Rate limit per PR and per user (rolling hour, counted from chat_commands in the same transaction); re-review deduplicated per head SHA; budget gate	rate limit tests
CH3	Injected commands in quoted text, code blocks or edited comments	Commands only from the first line of a newly created comment, exact syntax @<app-slug> <command>; quoted lines and code blocks ignored; edits ignored	parser unit tests
CH4	ignore abused to suppress security findings permanently	Suppression is per PR and per fingerprint, audited with actor; dashboard lists suppressions; does not change rules or thresholds	handler tests
TR5	Applied labels trigger workflows or other bots (issues: labeled, pull_request_target, duplicate/stale/ok-to-test/deploy)	Automatic labels only from an explicit, non-empty allowlist (empty allowlist → proposals only); token-based protected defaults; labels only on issues (the publisher checks the number is an issue, not a PR)	publisher label tests; empty-allowlist test
TR6	Repeated edits re-run triage to pile up labels or cost	Re-triage only when title or body change, at most 3 runs per issue per day; total automatic labels per issue ≤ maxLabels	triage handler tests
RP7	Citation laundering: attacker writes an issue, closes it, then cites it from a new issue	Issue sources only from OWNER/MEMBER/COLLABORATOR authors; PRs only if merged (body only); never the current issue or other issues of the same author; comments are not citable	retrieval and verifier tests
RP8	Spoofed or cross-repository links and references	Links only under the repository path prefix; cross-repository references (owner/repo#1, GH-1, owner/repo@sha) and #N not in the cited set neutralised	sanitiser tests
RP9	Reply text drives other automation (/close, /lgtm, /deploy, "Duplicate of #N")	Transparency label is the first line; a leading / on any line is escaped; no line starts with "Duplicate of"	sanitiser and render tests
RP10	Cost abuse by anonymous users (mass issues, follow-ups that abstain, edits)	Attempt caps that count every draft or run (per issue, per author per day, per repository per day); checkout only when identifiers exist; budget gate per feature	cap tests
RP11	Quoted fields in IssueFacts carry injected text to the writer	Facts passed to the writer inside untrusted(); URLs and mentions stripped from quotes; canary eval	reply injection eval
RP12	Staleness beyond updated_at (trigger comment edited or deleted, issue closed, locked or transferred, feature off, visibility changed)	Staleness by content_updated_at; approval and publish re-check issue state, trigger comment, feature mode and effectiveMode with the current visibility	approval tests
CH5	Permission API semantics (custom roles, older GHES without role_name, 403/404)	Decision only on permission ∈ {admin, write} (maintain reports write); anything else denies; denied attempts per user capped without an API call	nock permission matrix
CH6	Untrusted content in chat answers or committable suggestions (fork PRs)	Commands: prompt limited to the finding and head lines. Free-form questions (ask) add the question, the PR title and description, the diff between base and head (capped at 24 000 characters) and the titles of the published findings of the latest review run. Never other users' thread comments; suggestions validated against the head blob and limited to the finding's lines; .github/**, CI and lock files refused	chat handler tests
CH7	ignore hides later findings that share a fingerprint	Suppression does not apply to security findings of severity high or above; head SHA recorded; suppressed findings stored with status suppressed and visible in the dashboard	review suppression tests
CH8	Injection through a free-form question or PR content (description, diff, code comments, file names)	Every block wrapped in untrusted(); no tools, no network, no secrets in the prompt; structured output; answer passed through the reply cleaner (HTML, images, links, mentions removed) and the publisher sanitiser and secret scan; ask is opt-in per repository and only runs after the permission check	ask.test.ts; chat-smoke-v1 injection items; chat items in redteam-smoke-v1
CH9	Hallucinated or unsupported answers	Every answer needs at least one citation verified against the head excerpt or the new-side diff hunks (verifyCitations); any failed citation, abstention or confidence below 0.6 posts the fixed text "CodeLotse could not answer this from the pull request." instead of model text	ask.test.ts; chat-smoke-v1 (verified citation rate, unverified publishes 0)
CH10	Free-form answers on public repositories	effectiveMode('reply', …) downgrades auto to approval for ask; shadow records only and calls no model	chat handler tests
CH11	Cost abuse through questions	ask counts against the chat rate limits per PR and per user, including abstentions; budget gate; one model call per question, 2 000 output tokens; calls traced as chat:<command id>	rate limit tests
LN1	Injected or abusive learning proposals ("activate this now", a reply that smuggles instructions into review prompts)	Proposals are stored with proposed = true and enabled = false (database check: a proposal is never enabled); only a user with repo:manage activates or rejects one in the dashboard, and every decision is audited (*.rule.activate, *.rule.reject); reply proposals post nothing; proposal text is cleaned (links, images, HTML and mentions removed) and capped at 300 characters (1,000 for learn); active learnings reach the finder only inside an untrusted() block of their own that says they never change the task	classify.test.ts; chat.int.test.ts; rules.int.test.ts (API); learning items in redteam-smoke-v1
LN2	Learnings used to suppress security findings	The finder prompt keeps learnings in their own block and says they never apply to security findings of high or critical severity. In code, the ranking step only leaves out a finding the model marked as allowed by an active learning, and never one with category security and severity high or critical (applyTeamLearnings in packages/agents/src/review/rank.ts). Replies that ask to ignore, skip or weaken security checks, and proposals that say so, are refused before they are stored. The floor protects reported findings only: the model can still fail to report a finding because of a learning	pipeline.test.ts (team learnings); learning-smoke-v1 (security opt-out proposals 0); smoke-v1 item 022
LN3	Proposal spam and cost abuse through replies	Only replies of OWNER, MEMBER or COLLABORATOR in CodeLotse finding threads are routed (one indexed lookup of the posted comment through pull_requests); learning mode must not be off; at most 20 reply proposals per repository per rolling day; one fast-tier call per reply, none for a known source key; learn counts against the chat rate limits	chat.int.test.ts; learning-loop.int.test.ts (route)
LN4	Personal data in proposals	The author's code-host login is stored in rules.source_login, registered as a login link and cleared by a DSAR deletion; source_ref holds only the PR number, the comment id and the finding fingerprint; the deciding maintainer is decided_by (actor)	dsar.int.test.ts
DT3	Issue and comment bodies in outbox and queue payloads	Bodies kept to the event payload already stored for webhook deliveries; handlers read bodies from the database; outbox retention 7 days; completed jobs removed	retention test
DT4	Transferred issues leave their content under the old repository	issues.transferred handled as a delete	normalize/persist test
DT1	Storing issue bodies and comments increases exposure	Tenant tables with forced RLS; bodies capped (64 KiB); deleted when the issue or comment is deleted or the repository is removed; retention per data-inventory.md	RLS test; retention test
DT2	Hidden markers spoofed by users	Reconcile trusts markers only on comments authored by the app's own bot login (as I7)	publisher tests

4. Conditions under which the publisher posts

The publisher posts an issue comment, label change or PR thread reply only if all hold:

1. The output is a typed publish target (no free-form API calls), validated by its zod schema.
2. The repository policy for that feature is not off, and the decision is auto, or a maintainer approved the pending output (which is not expired, not rejected and not stale).
3. Public repositories: replies and triage comments (needs-info, duplicates) are never auto (effectiveMode downgrades them to approval). Labels may be auto. Chat replies to commands may be auto because only users with write access trigger them and they answer that user in the same thread; their content is still untrusted (fork PRs), so CH6 applies. Answers to free-form questions follow the reply rule and are never auto on public repositories (CH10).
4. The body passed the sanitizer (no HTML, no images, mentions neutralised, links only to the repository host), the secret scan, and the length cap; it carries the transparency label and the hidden idempotency marker.
5. Labels: each label exists in the repository, is on the allowlist, is not protected, and is added (never removed).
6. Replies: at least one citation verified for answers and workarounds; all citations point into the same repository.
7. Chat: the requesting user had write permission when the job ran; the reply goes into the same thread of a CodeLotse finding, otherwise into the pull request conversation. Answers to free-form questions carry at least one verified citation or are replaced by the fixed text (CH9).
8. The idempotency key has not been published before (reconcile with the bot's own comments).

5. Residual risks

• A maintainer can approve a harmful reply; previews show exactly what is posted.
• Lexical duplicate search misses paraphrased duplicates (measured by the dupes eval; embeddings only if the eval shows the need, ADR-0011).
• Public auto-replies remain off until sign-off of this analysis.
• A workflow that posts comments with a maintainer's token and echoes issue text can issue chat commands with write permission.
• A real GitHub user whose login equals the app slug is notified by every command.
• An active learning can make the model leave out a finding without marking it, also a security finding of high or critical severity. The ranking floor (LN2) only protects findings the model reported; the prompt asks the model to report and mark such findings instead.
• A maintainer can activate a harmful learning; the Rules page shows its source and the full text before activation.