Dry runs ​
A dry run shows what CodeLotse would have said on the last merged pull requests of a repository, next to the comments your reviewers actually left. Nothing is posted to the code host. Use it before you turn reviews on, or after you changed rules or models, to judge whether the findings are worth it.
Start a dry run ​
You need to be a maintainer of the repository or an organisation administrator. Dry runs work on GitHub only; other code hosts show that the feature is not available yet.
- Open the repository and choose the Dry run tab. The last setup step offers the same with Open dry run.
- Select Start dry run.
- Enter Pull requests to review: between 1 and 20 of the most recently merged pull requests, 10 by default.
- Check the Estimated cost. It is based on the average review in this repository, or unknown when the repository has no finished review yet.
- Start the run. The page updates while the pull requests are reviewed.
Each pull request is reviewed with the repository's current policy, rules and active learnings, like a live review in shadow mode, but without deterministic scanners and without the acceptance criteria check. The model calls count against the organisation budget.
One dry run can be active per repository, and at most three can run per repository in 24 hours. A run that has not finished after two hours is marked as timed out. Cancel dry run stops it early.
Read the result ​
The list on the Dry run tab shows each run with its status (Queued, Running, Completed, Failed or Cancelled), Progress, Cost and who Started by.
Open a run to see the Comparison:
| Total | Meaning |
|---|---|
| Found by both | CodeLotse raised a finding and a reviewer commented on the same file within 3 lines. |
| Only CodeLotse | Findings no reviewer raised. Read these first: they are either what CodeLotse adds, or noise you can tune away with rules, learnings or a higher minimum severity. |
| Only reviewers | Line comments CodeLotse did not match. |
Below, every pull request shows CodeLotse findings next to Reviewer comments, with the badges Also raised by a reviewer and Also found by CodeLotse, and a link to Open pull request. Comments by bots and by CodeLotse itself are left out.
What happens to the data ​
The reviewer comments are stored only to show them here and are never sent to a model. Dry runs follow the organisation's data retention and are included in data subject requests.