Files
PX4-Autopilot/docs/en/sensor/optical_flow.md
T
440179383a feat(SIH): Added optical flow sensor continues on #28270 (#29676)
* feat(sih): model the distance sensor range limits, quality and noise

The simulated distance sensor reported a noise free reading with no signal
quality, and saturated only against the upper range limit. Readings below the
minimum range were passed through unchanged, which is not what a range finder
does.

Report the raw saturated value with a zero quality outside of the measurement
range, derate the quality over the last 20% of the range, and add configurable
Gaussian noise (SIH_DISTSNSR_NOI). The quality is decided on the true distance
so that a measurement sitting at a range limit does not flicker in and out of
validity.

The sensor is now enabled with SIH_DISTSNSR_EN instead of by an out of band
value of the override parameter, and the ground distance computation moves into
a helper.

* feat(sih): add optical flow simulation

Simulate a downward facing optical flow sensor in SIH so that flow aided
navigation can be exercised without a rendered camera image. The flow rate is
the sum of the body rates and of the ground relative velocity scaled by the
distance to the ground, integrated over one publication interval and published
on sensor_optical_flow with configurable rate (SIH_OF_RATE) and noise
(SIH_OF_NOISE).

The sensor reports the integrated flow only: the flow module completes the
sample with the vehicle gyro and the downward facing distance sensor, which
therefore has to be enabled as well. Quality drops to zero above
SIH_OF_H_MAX or when the flow rate exceeds what the sensor can correlate.

The topic is advertised before the simulated clock starts, because the sensors
module brings up its flow pipeline on the first appearance of the topic, and
publishing during startup deadlocks the lockstep clock this thread advances.

Add the quadx_flow and standard_vtol_flow models with matching airframes:
SYS_AUTOSTART 10046 flies the multicopter on flow only with no GNSS, 10047
fuses the flow next to GNSS on a VTOL.

* prune configs

Signed-off-by: RomanBapst <bapstroman@gmail.com>

* more config cleanup

Signed-off-by: RomanBapst <bapstroman@gmail.com>

* added EKF2_HGT_REF to vtol flow config

Signed-off-by: RomanBapst <bapstroman@gmail.com>

* feat(simulation): added tests for optical flow for both quadx and stanard vtol
modes and enable them in CI

- for the test we monitor error on ground truth position and velocity

Signed-off-by: RomanBapst <bapstroman@gmail.com>

* feat(docs): document addition of optical flow in SIH

Signed-off-by: RomanBapst <bapstroman@gmail.com>

* fix(mavsdk_tests): unsubscribe flow ground truth comparison on destruction

The ground truth comparison callbacks capture `this` and touch members of
AutopilotTesterFlow. If a REQUIRE fails between start and stop, the tester
is destroyed during unwinding while the subscriptions are still active,
so a callback could run on the destroyed mutex and sample buffers.

Make stop idempotent and call it from the destructor, and stop any running
comparison before starting a new one.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: RomanBapst <bapstroman@gmail.com>

* tests for python

* fix(mavsdk_tests): start the VTOL flow loiter already on its radius

The loiter was centered on the vehicle's position right after the
transition, so it started at the center and had to fly straight out to the commanded
radius before the lateral guidance curved onto the track.

Assisted-by: Claude:claude-sonnet-5

* fix(mavsdk_tests): resend the flow test EKF origin until it is taken

The SITL startup script starts ekf2 after MAVLink. When MAVSDK connected
first, the single SET_GPS_GLOBAL_ORIGIN was published before ekf2 existed
and MAVSDK's RUN_PREARM_CHECKS followed it. Ekf2 then subscribed, saw
only the newest vehicle command and never got an origin. The Coverage
build starts slowly enough to hit this every time.

Assisted-by: Claude:claude-opus-5-5

* ci(sitl_tests): run the VTOL flow test without coverage

As a Coverage job it built with coverage on every PR and uploaded the
sitl-vtol flag from PRs, which replaced main's carried-forward VTOL
coverage with the flow-only flight. The quadx_flow entry also sat
between the Coverage comment and the standard_vtol entry it describes.

Assisted-by: Claude:claude-opus-5-5

* fix(mavsdk_tests): sum the flow test heading sweep with its sign

Adding up the absolute heading changes counted yaw oscillation toward the
360 degrees, so the wait could end before a full orbit was flown.

Assisted-by: Claude:claude-opus-5-5

* refactor(airframes): keep only the non-default params in the SIH flow airframes

SIH_OF_H_MAX, SIH_DISTSNSR_EN and EKF2_OF_CTRL were set to their defaults
and SENS_EN_GPSSIM is already set by the quadx base airframe. The quadx
forced EKF2_RNG_CTRL to 2 while the VTOL used the conditional default for
no stated reason, so both now use the default.

Assisted-by: Claude:claude-opus-5-5

* docs(sim_sih): mark the SIH flow airframes as new in PX4 v2.0

release/1.18 has branched, so these airframes first ship in v2.0, not
v1.17.

Assisted-by: Claude:claude-opus-5-5

---------

Signed-off-by: RomanBapst <bapstroman@gmail.com>
Co-authored-by: RomanBapst <bapstroman@gmail.com>
Co-authored-by: Jacob Dahl <dahl.jakejacob@gmail.com>
2026-11-09 11:13:16 -07:00

135 lines
8.1 KiB
Markdown

# Optical Flow
_Optical Flow_ uses a downward facing camera and a downward facing distance sensor for velocity estimation.
It can be used to determine speed when navigating without GNSS — in buildings, underground, or in any other GNSS-denied environment.
The video below shows PX4 holding position using the [Ark Flow](../dronecan/ark_flow.md) sensor for velocity estimation in [Position Mode](../flight_modes_mc/position.md):
<lite-youtube videoid="aPQKgUof3Pc" title="ARK Flow with PX4 Optical Flow Position Hold"/>
<!-- ARK Flow with PX4 Optical Flow Position Hold: 20210605 -->
The image below shows an optical flow setup with a separate flow sensor ([PX4Flow](../sensor/px4flow.md)) and distance sensor ([Lidar-Lite](../sensor/lidar_lite.md)):
![Optical flow lidar attached](../../assets/hardware/sensors/optical_flow/flow_lidar_attached.jpg)
## Setup
An Optical Flow setup requires a downward facing camera and a downward facing [distance sensor](../sensor/rangefinders.md) (preferably a LiDAR).
These can be combined in a single product, such as the [ARK Flow](../dronecan/ark_flow.md), [ARK Flow MR](../dronecan/ark_flow_mr.md) and [Holybro H-Flow](https://holybro.com/products/h-flow), or they may be separate sensors.
The sensor(s) can be connected via MAVLink, I2C or any other bus that supports the peripheral.
::: info
If connected to PX4 via MAVLink the Optical Flow camera sensor must publish the [OPTICAL_FLOW_RAD](https://mavlink.io/en/messages/common.html#OPTICAL_FLOW_RAD) message, and the distance sensor must publish the [DISTANCE_SENSOR](https://mavlink.io/en/messages/common.html#DISTANCE_SENSOR) message.
The information is written to the corresponding uORB topics: [DistanceSensor](../msg_docs/DistanceSensor.md) and [ObstacleDistance](../msg_docs/ObstacleDistance.md).
:::
The output of the flow when moving in different directions must be as follows:
| Vehicle movement | Integrated flow |
| ---------------- | --------------- |
| Forwards | + Y |
| Backwards | - Y |
| Right | - X |
| Left | + X |
::: info
The integrated flow values are **angular measurements** (radians) representing rotation of the image about the sensor's body axes using the right-hand convention.
They are _not_ translational displacements along those axes, which is why forward vehicle movement (along X) appears in the Y flow axis, and rightward movement (along Y) appears in the X flow axis.
Specifically, forward movement causes the ground image to rotate about the Y axis (+ Y), while rightward movement causes a negative rotation about the X axis (- X).
:::
Sensor data from the optical flow device is fused with other velocity data sources.
The approach used for fusing sensor data and any offsets from the center of the vehicle must be configured in the [estimator](#estimators).
### Scale Factor
For pure rotations the `OPTICAL_FLOW_RAD.integrated_xgyro` and `OPTICAL_FLOW_RAD.integrated_x` (respectively `integrated_ygyro` and `integrated_y`) have to be the same.
If this is not the case, the optical flow scale factor can be adjusted using [SENS_FLOW_SCALE](../advanced_config/parameter_reference.md#SENS_FLOW_SCALE).
:::tip
The low resolution of common optical flow sensors can cause slow oscillations when hovering at a high altitude above ground (> 20m).
Reducing the optical flow scale factor can improve the situation.
:::
## Flow Sensors/Cameras
### Agam FloRange
[Agam FloRange](../dronecan/agam_florange.md) is a [DroneCAN](../dronecan/index.md) optical flow sensor, [distance sensor](../sensor/rangefinders.md), and IMU.
It has a PixArt PAA3905 optical flow sensor, Broadcom AFBR-S50LV85D 30-metre distance sensor, and InvenSense ICM-42688-P 6-Axis IMU.
### ARK Flow & ARK Flow MR
[ARK Flow](../dronecan/ark_flow.md) is a [DroneCAN](../dronecan/index.md) optical flow sensor, [distance sensor](../sensor/rangefinders.md), and IMU.
It has a PAW3902 optical flow sensor, Broadcom AFBR-S50LV85D 30 meter distance sensor, and Invensense ICM-42688-P 6-Axis IMU.
[ARK Flow MR](../dronecan/ark_flow_mr.md) is a [DroneCAN](../dronecan/index.md) optical flow sensor, [distance sensor](../sensor/rangefinders.md), and IMU, for mid-range applications.
It has a PixArt PAA3905 optical flow sensor, Broadcom AFBR-S50LX85D 50 meter distance sensor, and Invensense IIM-42653 6-Axis IMU.
### Holybro H-Flow
The [Holybro H-Flow](https://holybro.com/products/h-flow) is a compact [DroneCAN](../dronecan/index.md) optical flow and [distance sensor](../sensor/rangefinders.md) module.
It combines a PixArt PAA3905 optical flow sensor, a Broadcom AFBR-S50LV85D distance sensor, and an InvenSense ICM-42688-P 6-axis IMU.
An all-in-one design that simplifies installation, with an onboard infrared LED enhances visibility in low-light conditions.
### PMW3901-Based Sensors
[PMW3901](../sensor/pmw3901.md) is an optical flow tracking sensor similar to what you would find in a computer mouse, but adapted to work between 80 mm and infinity.
It is used in a number of products, including some from: Bitcraze, Tindie, Hex, Thone and Alientek.
### Other Cameras/Sensors
It is also possible to use a board/quad that has an integrated camera.
For this the [Optical Flow repo](https://github.com/PX4/PX4-OpticalFlow) can be used (see also [snap_cam](https://github.com/PX4/snap_cam)).
## Range Finders
You can use any supported [distance sensor](../sensor/rangefinders.md).
However we recommend using LIDAR rather than sonar sensors, because of their robustness and accuracy.
## Estimators
Estimators fuse data from the optical flow sensor and other sources.
The settings for how fusing is done, and relative offsets to vehicle center must be specified for the estimator used.
The offsets are calculated relative to the vehicle orientation and center as shown below:
![Optical Flow offsets](../../assets/hardware/sensors/optical_flow/px4flow_offset.png)
Optical Flow based navigation is enabled by both [EKF2](#ekf2) and LPE (deprecated).
### Extended Kalman Filter (EKF2) {#ekf2}
For optical flow fusion using EKF2, set [EKF2_OF_CTRL](../advanced_config/parameter_reference.md#EKF2_OF_CTRL).
If your optical flow sensor is offset from the vehicle centre, you can set this using the following parameters.
| Parameter | Description |
| -------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| <a id="EKF2_OF_POS_X"></a>[EKF2_OF_POS_X](../advanced_config/parameter_reference.md#EKF2_OF_POS_X) | X position of optical flow focal point in body frame (default is 0.0m). |
| <a id="EKF2_OF_POS_Y"></a>[EKF2_OF_POS_Y](../advanced_config/parameter_reference.md#EKF2_OF_POS_Y) | Y position of optical flow focal point in body frame (default is 0.0m). |
| <a id="EKF2_OF_POS_Z"></a>[EKF2_OF_POS_Z](../advanced_config/parameter_reference.md#EKF2_OF_POS_Z) | Z position of optical flow focal point in body frame (default is 0.0m). |
See [Using PX4's Navigation Filter (EKF2) > Optical flow](../advanced_config/tuning_the_ecl_ekf.md#optical-flow) for more information.
## Simulation
Optical flow can be simulated with [SIH](../sim_sih/index.md), which computes the flow from the simulated vehicle motion instead of rendering a camera image.
As with real hardware, the simulated sensor reports the integrated flow only, so a downward facing [distance sensor](../sensor/rangefinders.md) has to be simulated alongside it — SIH provides one.
Two ready-made airframes have flow enabled and the estimator configured for it:
```sh
# Multicopter navigating on optical flow only (GNSS fusion disabled)
make px4_sitl_sih sihsim_quadx_flow
# Standard VTOL navigating on optical flow only (GNSS fusion disabled)
make px4_sitl_sih sihsim_standard_vtol_flow
```
The sensor itself is configured with the [`SIH_OF_*` parameters](../sim_sih/index.md#optical-flow) (rate, noise, and maximum ground distance).
Optical flow is also simulated by [Gazebo Classic](../sim_gazebo_classic/index.md), which renders the downward facing camera image and runs flow computation on it.