docs(ci): document the AI pull request review (#28989)

* docs(ci): document the AI pull request review

Adds an "AI Pull Request Review" page under Platform Testing and CI:
who can use it, the !ai-review command and its options, what a review
contains, cost, how it works and its trust model, and where to change
it. It is marked as a trial. The maintainers page lists the new
permission during onboarding, and the AI coding assistants page tells
contributors how to treat an AI review on their PR.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Ramon Roche <mrpollo@gmail.com>

* docs(ci): mention the AI review check on the PR

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Ramon Roche <mrpollo@gmail.com>

* docs(ci): correct the AI review options and re-run behaviour

The model and effort options only change the first pass; the confirming pass always runs Opus at high. A re-run collapses the old summary but leaves its inline comments, and the command only works on open PRs.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Ramon Roche <mrpollo@gmail.com>

---------

Signed-off-by: Ramon Roche <mrpollo@gmail.com>
This commit is contained in:
Ramon Roche authored and GitHub committed 2026-10-12 00:17:03 +11:00
1 parent e1c28e2755
commit bf6603b42b
5 files changed
+117

No files matched your search

+1
View File
@@ -952,6 +952,7 @@
- [Fuzz Tests](test_and_ci/fuzz_tests.md)
- [Sanitizers](test_and_ci/sanitizers.md)
- [Continuous Integration](test_and_ci/continous_integration.md)
- [AI Pull Request Review](test_and_ci/ai_review.md)
- [Integration Testing](test_and_ci/integration_testing.md)
- [MAVSDK Integration Testing](test_and_ci/integration_testing_mavsdk.md)
- [PX4 ROS 2 Interface Library Integration Testing](test_and_ci/integration_testing_px4_ros2_interface.md)
+4
View File
@@ -77,3 +77,7 @@ Contributions that ignore them may be closed by maintainers without detailed rev
- **Review discussion is between humans.**
Using an AI tool to help you understand a review comment is fine; posting model output you do not understand as your reply is not.
AI-assisted review is acceptable when clearly labeled as such; presenting undisclosed AI review as your own reading is not.
- **Automated AI reviews.**
Maintainers may request an [AI pull request review](../test_and_ci/ai_review.md) of your PR, which is posted as a labelled review comment.
Treat its findings like any other reviewer's: fix them or reply explaining why they do not apply.
It can be wrong, and maintainers make the final decision.
+1
View File
@@ -47,6 +47,7 @@ Once accepted every maintainers will go through the following process:
1. Permission to merge the PR of any of PX4 workspace repositories after it's approved
2. Permission to trigger GitHub actions when a new contributor opens a PR.
3. Permission to edit Issue/PR contents.
4. Permission to request an [AI pull request review](../test_and_ci/ai_review.md) by commenting `!ai-review` on a PR.
3. **Add your info to official PX4 channels**:
1. Add your information to the internal Dronecode database of maintainers to keep you in sync.
2. Community introduction to the new maintainer in the form of a forum post, which is promoted through ever growing official channels
+110
View File
@@ -0,0 +1,110 @@
# AI Pull Request Review
PX4 maintainers can ask an AI model to review a pull request.
The review is posted on the PR as a regular GitHub review comment, with inline comments on specific lines where it found problems.
::: warning
This is a trial.
Reviews can be wrong, and the tool may change as we learn what works.
Feedback from maintainers is welcome in the maintainers' channel or as an issue.
:::
## Who Can Use It
Only people with write, maintain or admin access to the PX4-Autopilot repository can start a review.
In practice this is the [Dev Team](https://github.com/orgs/PX4/teams/dev-team) (see [Maintainers](../contribute/maintainers.md)).
A request from anyone else is ignored.
## Starting a Review
Post a comment on the open pull request that contains only this text:
```txt
!ai-review
```
The comment must contain nothing else.
A sentence that mentions the command, a quote of it, or the command with a typo does not start a review.
To pick the model or the effort level, add options on the same line:
```txt
!ai-review model=sonnet effort=high
```
- `model`: `opus` (the default) or `sonnet`.
- `effort`: `low`, `medium`, `high`, `xhigh` (the default) or `max`.
Both options apply to the first review pass only.
The second pass that confirms findings always uses Claude Opus at `high` effort (set in `Tools/ci/ai_review/models.json`), and its cost is included in the total.
Within a few seconds your comment gets a 👀 reaction, which means the review has started.
When it finishes, the comment gets 🚀, or 😕 if the run failed.
A review takes about 5 to 15 minutes, depending on the size of the PR.
While it runs, an "AI PR Review" entry in the PR's checks shows its progress and links to the workflow run.
This check is informational and never blocks a merge.
You can also start a review from the **Actions** tab: open the [AI PR Review](https://github.com/PX4/PX4-Autopilot/actions/workflows/pr-ai-review.yml) workflow, select **Run workflow** and enter the PR number.
::: tip
Use reviews where a second opinion matters most: flight-critical code, new boards, large changes, and contributions from people new to the project.
Every review costs money (see [Cost](#cost)).
:::
## What the Review Contains
For an example, see [PR #28901](https://github.com/PX4/PX4-Autopilot/pull/28901).
- Inline comments on specific lines, for findings that a second, independent model pass confirmed with high confidence.
When a small change fully fixes the problem, the comment includes a suggestion that the author can apply with one click.
- A "Before merge" list for issues with the pull request as a whole, such as missing test evidence, a description that does not match the change, or missing upgrade notes or documentation.
- A checklist covering whether the problem is real, test evidence, the description, compatibility, and documentation.
- A collapsed list of lower-confidence findings that may still be worth checking.
- The cost of the review, the model used and how long it took, at the bottom.
The review is always a comment.
It never approves a pull request or requests changes: maintainers decide.
Running the review again on the same pull request collapses the previous review's summary and marks it as superseded.
Its inline comments stay on the pull request and may refer to an older commit.
If the model's safety filter declines to review a pull request, which occasionally happens, the bot posts that no review was done and quotes the reason it was given.
## Cost
Each review is billed to the PX4 project.
So far, reviews have cost between about $0.60 for a small change and $5 to $8 for a large board addition with Claude Opus, and less with Claude Sonnet.
The exact cost is shown at the bottom of every review.
Claude Sonnet is faster and cheaper, but in testing it missed real problems that Claude Opus found, so Opus is the default.
## How It Works
1. A small job checks that the comment is exactly the command and that the commenter has write access, using the GitHub API.
If either check fails, nothing else runs.
2. The review job runs _Claude Code_ with Claude Opus on Amazon Bedrock, on a PX4 CI runner.
It reads the pull request, the surrounding code, linked issues, commit messages, CI status and git history.
It reviews against the same criteria as the `review-pr` skill used by maintainers locally, plus the contribution requirements (test evidence, an accurate description, upgrade notes and documentation).
3. A second model pass checks each finding against the code.
Only findings it confirms with high confidence are posted as inline comments.
4. The result is handed to the existing PR review poster workflow, which posts it on the pull request.
The pull request is treated as untrusted input:
- The reviewer can only read files and run a few read-only commands.
It never builds or runs the pull request's code.
- When it needs to check a calculation, it runs Python in an isolated `px4-dev` container with no network access and no credentials.
- The review job has no write access to GitHub; only the separate posting workflow does.
- The review code always comes from `main`.
A pull request cannot change how it is reviewed, and changes to the review tool only take effect once they are merged.
## Changing the Review
The code is in [`Tools/ci/ai_review/`](https://github.com/PX4/PX4-Autopilot/tree/main/Tools/ci/ai_review), a Python package with its own [README](https://github.com/PX4/PX4-Autopilot/blob/main/Tools/ci/ai_review/README.md).
The workflow is [`.github/workflows/pr-ai-review.yml`](https://github.com/PX4/PX4-Autopilot/blob/main/.github/workflows/pr-ai-review.yml).
- The models and their default effort levels are set in `Tools/ci/ai_review/models.json`.
- The review criteria shared with the `review-pr` skill are in `.agents/skills/review-pr/review-criteria.md`.
- The CI-only prompts are in `Tools/ci/ai_review/prompts/`.
Pull requests to improve the review are welcome.
+1
View File
@@ -10,6 +10,7 @@ Test topics include:
- [Unit Tests](../test_and_ci/unit_tests.md)
- [Sanitizers](../test_and_ci/sanitizers.md) - Build SITL with ASan/TSan to catch memory errors and data races
- [Continuous Integration (CI)](../test_and_ci/continous_integration.md)
- [AI Pull Request Review](../test_and_ci/ai_review.md) - Ask an AI model to review a pull request (maintainers, trial)
- [Integration Testing](../test_and_ci/integration_testing.md)
- [MAVSDK Integration Testing](../test_and_ci/integration_testing_mavsdk.md)
- [PX4 ROS 2 Interface Library Integration Testing](../test_and_ci/integration_testing_px4_ros2_interface.md)