Files
PX4-Autopilot/docs/en/debug/system_wide_replay.md
T
Jacob Dahl e40501eba0 refactor(msg): rename GPS to GNSS in messages, topics and sensor parameters (#24399)
* refactor(msg): rename SensorGps to SensorGnss and add VehicleGnss

GPS is one constellation; the receivers and the rest of PX4 handle GNSS. Field names lose their units (latitude, speed_accuracy, course, ...), which move into the metadata comments. The sensors module's output becomes VehicleGnss on vehicle_gnss: the selected receiver's sample next to what the sensors module adds to it (corrected sample time, antenna position, selected instance), so a consumer can tell the two apart and later selection and check results have a place to go.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/gps): publish sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/septentrio): publish sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/pps_capture): read sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(lib/gnss): select receivers on sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(sensors): publish VehicleGnss on vehicle_gnss and rename SENS_GPS* to SENS_GNSS*

The sensors module's output nests the selected receiver's sample, with the corrected sample time, antenna position and selected instance next to it. SENS_GPS_MASK/TAU/PRIME and SENS_GPSn_ID/OFFX/OFFY/OFFZ/DELAY take the SENS_GNSS name the heading parameters already use; saved parameters migrate on import, and the EKF2_GPS_POS/EKF2_GPS_DELAY translations now land on the new names directly.

Assisted-by: Claude:claude-opus-5-5

* refactor(logger): log sensor_gnss and vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(ekf2): fuse vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(replay): replay vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(local_position_estimator): read vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(attitude_estimator_q): read vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(lib/terrain_estimation): take sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(lib/wind_estimator): read vehicle_gnss in the replay script

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(vision_target_estimator): read vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(navigator): read vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(commander): read sensor_gnss and vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(mavlink): stream sensor_gnss and vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(uxrce_dds_client): bridge vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(zenoh): bridge vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(failure_injection): inject GNSS failures on sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/ins): publish sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/uavcan): publish sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/uavcannode): read sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/cyphal): bridge sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/telemetry): read vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/rc): read vehicle_gnss for telemetry

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/osd): read vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(drivers/transponder): read sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(boards/modalai): publish sensor_gnss from the voxl2 drivers

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(examples): publish sensor_gnss and read vehicle_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(simulation): publish sensor_gnss

Follows the SensorGps to SensorGnss and vehicle_gps_position to VehicleGnss rename.

Assisted-by: Claude:claude-opus-5-5

* refactor(gnss): name GNSS variables gnss in the sensors module and receiver drivers

GPS is one constellation. Identifiers that hold or handle GNSS samples, or bind to the renamed parameters, follow the message and parameter rename; classes, files, modules and names tied to parameters that keep GPS stay.

Assisted-by: Claude:claude-opus-5-5

* refactor(gnss): name GNSS variables gnss in the estimators and navigator

GPS is one constellation. Identifiers that hold or handle GNSS samples, or bind to the renamed parameters, follow the message and parameter rename; classes, files, modules and names tied to parameters that keep GPS stay.

Assisted-by: Claude:claude-opus-5-5

* refactor(gnss): name GNSS variables gnss in commander, mavlink and failure injection

GPS is one constellation. Identifiers that hold or handle GNSS samples, or bind to the renamed parameters, follow the message and parameter rename; classes, files, modules and names tied to parameters that keep GPS stay.

Assisted-by: Claude:claude-opus-5-5

* refactor(gnss): name GNSS variables gnss in the remaining drivers and simulation

GPS is one constellation. Identifiers that hold or handle GNSS samples, or bind to the renamed parameters, follow the message and parameter rename; classes, files, modules and names tied to parameters that keep GPS stay.

Assisted-by: Claude:claude-opus-5-5

* docs(gnss): reference sensor_gnss, vehicle_gnss and SENS_GNSS* parameters

The upgrade guide covers the topic, field, ROS 2, parameter and Cyphal register renames.

Assisted-by: Claude:claude-opus-5-5

* fix(boards): name EKF2::UpdateGnssSample in the fmu-v6xrt ITCM lists

The rename left the old UpdateGpsSample symbol in the lists, so the ITCM check fails and the function drops out of ITCM on the boards it doesn't check.

Assisted-by: Claude:claude-opus-5-5

* chore(drivers/gps): bump PX4-GPSDrivers to the merged sensor_gnss rename

The submodule pointed at the PX4-GPSDrivers#243 branch commit; #243 is merged.

Assisted-by: Claude:claude-opus-5-5

* docs(msg): restore WGS84 in SensorGnss and point VehicleGlobalPosition at vehicle_gnss

The rename dropped the only mention of the WGS84 datum and ellipsoid, and left VehicleGlobalPosition referring to the removed vehicle_gps_position topic.

Assisted-by: Claude:claude-opus-5-5

* refactor(msg): drop selected_instance from VehicleGnss

receiver.device_id already names the selected receiver, and 0 while blended. The sensor_gnss instance depends on the order drivers advertise at boot, so it doesn't identify a receiver across boots or logs, and nothing reads it. It would also be meaningless on the per-receiver VehicleGnss instances proposed in #28894.

Assisted-by: Claude:claude-opus-5-5

* refactor(gnss): fuse vehicle_gnss.timestamp_sample directly and document the receiver's corrected time

vehicle_gnss.timestamp is now the sensors module's publish time, so EKF2's check of timestamp_sample against it was always true; the sensors module always fills timestamp_sample. The message docs described receiver.timestamp_sample as the driver's uncorrected value, but the sensors module writes the corrected time into it as well.

Assisted-by: Claude:claude-opus-5-5

* docs(msg): point VehicleGnss users at the top-level timestamp_sample

uXRCE-DDS shifts only top-level timestamp and timestamp_sample into agent time, so receiver.timestamp_sample reaches ROS 2 on the PX4 boot clock even though it holds the same value on uORB.

Assisted-by: Claude:claude-opus-5-5

* docs(releases): tell ROS 2 users which VehicleGnss timestamps to read

Only top-level timestamps are converted to ROS time, so a port that reads receiver.timestamp gets the PX4 boot clock, and vehicle_gnss.timestamp is now the sensors module's publish time rather than the receiver's.

Assisted-by: Claude:claude-opus-5-5
2026-09-29 10:36:49 -06:00

7.6 KiB

System-wide Replay

It is possible to record and replay arbitrary parts of the system based on ORB messages.

Replay is useful to test the effect of different parameter values based on real data, compare different estimators, etc.

Prerequisites

The first step is to identify the module or modules that should be replayed. Then, identify all the inputs to these modules, i.e. subscribed ORB topics. For system-wide replay, this consists of all hardware input: sensors, RC input, MAVLink commands and file system.

All identified topics need to be logged at full rate (see logging). For ekf2 this is already the case with the default set of logged topics.

It is important that all replayed topics contain only a single absolute timestamp, which is the automatically generated field timestamp. Should there be more timestamps, they must be relative to the main timestamp. For an example, see SensorCombined.msg. Reasons for this are given below.

Usage

  • First, choose the file to replay and build the target (from within the PX4-Autopilot directory):

    export replay=<absolute_path_to_log_file.ulg>
    make px4_sitl_default
    

    This will create the build/make output in a separate build directory build/px4_sitl_default_replay (so that the parameters don't interfere with normal builds). It's possible to choose any posix SITL build target for replay, since the build system knows through the replay environment variable that it's in replay mode.

  • Add ORB publisher rules in the file build/px4_sitl_default_replay/rootfs/orb_publisher.rules. This file defines the modules that are allowed to publish particular messages. It has the following format:

    restrict_topics: <topic1>, <topic2>, ..., <topicN>
    module: <module>
    ignore_others: <true/false>
    

    This means that the given list of topics should only be published by <module> (which is the command name). Publications to any of these topics from another module are silently ignored. If ignore_others is true, publications to other topics from <module> are ignored.

    For replay, we only want the replay module to be able to publish the previously identified list of topics. So, for replaying ekf2, the rules file should look like this:

    restrict_topics: sensor_combined, vehicle_gnss, vehicle_land_detected
    module: replay
    ignore_others: true
    

    With this, the modules that usually publish these topics don't need to be disabled for the replay.

  • (Optional) Setup parameter overrides (see instructions below).

  • (Optional) Copy a dataman mission file from the SD card to the build directory. This is only necessary if a mission should be replayed.

  • Start the replay:

    make px4_sitl_default none_iris
    

    This will automatically open the log file, apply the parameters and start the replay. Once done, it will report the outcome and exit. The newly generated log file can then be analyzed. It can be found in rootfs/fs/log, in subdirectories organised by date. Replayed log file names will have the _replayed suffix.

    Note that the above command will show the simulator as well, but - depending on what is being replayed - it will not show what's actually going on. It is still possible to connect via QGC and, for example, view the changing attitude during replay.

  • Finally, unset the environment variable, so that the normal build targets are used again:

    unset replay
    

Overriding Parameters in the Original Log

By default, all parameters from the original log file are applied during a replay. If a parameter changes during recording, it will be changed at the right time during the replay.

Parameters can be overridden during a replay in two ways: fixed and dynamic. When parameters are overridden, corresponding parameter changes in the log are not applied during replay.

  • Fixed parameter overrides will override parameters from the start of the replay. They are defined in the file build/px4_sitl_default_replay/rootfs/replay_params.txt, where each line should have the format <param_name> <value>. For example:

    EKF2_RNG_NOISE 0.1
    
  • Dynamic parameter overrides will update parameter values at specified times. These parameters will still be initialised to the values in the log or in the fixed overrides. Parameter update events should be defined in build/px4_sitl_default_replay/rootfs/replay_params_dynamic.txt, where each line has the format <param_name> <value> <timestamp>. The timestamp is the time in seconds from the start of the log. For example:

    EKF2_RNG_NOISE 0.15 23.4
    EKF2_RNG_NOISE 0.05 56.7
    EKF2_RNG_DELAY 4.5 30.0
    

Important Notes

  • During replay, all dropouts in the log file are reported. These have a negative effect on the replay, so care should be taken to avoid dropouts during recording.
  • It is currently only possible to replay in 'real-time': as fast as the recording was done. This is planned to be extended in the future.
  • A message that has a timestamp of 0 will be considered invalid and not be replayed.

EKF2 Replay

This is a specialization of the system-wide replay for fast EKF2 replay.

::: info The recording and replay of flight logs with multiple EKF2 instances is not supported. To enable recording for EKF replay you must set the parameters to enable a single EKF2 instance. :::

In EKF2 mode, the replay will automatically create the ORB publisher rules described above.

To perform an EKF2 replay:

  • Record the original log with SDLOG_MODE set to 1 to log from boot.

  • In addition to the replay environment variable, set replay_mode to ekf2:

    export replay_mode=ekf2
    export replay=<absolute_path_to_log.ulg>
    
  • Run the replay with the none target:

    make px4_sitl none
    
  • Once finished, unset both replay and replay_mode.

    unset replay; unset replay_mode
    

Adjusting EKF2-specific Parameters for the Replay

First install pyulog:

pip install --user pyulog

Extract the original log's parameters to replay_params.txt:

ulog_params -i "$replay" -d ' ' | grep -e '^EKF2' > build/px4_sitl_default_replay/rootfs/replay_params.txt

Adjust these as desired, and add dynamic parameter overrides in replay_params_dynamic.txt if necessary.

Behind the Scenes

Replay is split into 3 components:

  • A replay module These have a negative effect on replay, so care should be taken to avoid dropouts during recording.
  • It is currently only possible to replay in 'real-time': as fast as the recording was done.

The replay module reads the log and publishes the messages at the same speed as they were recorded. A constant offset is added to the timestamp of each message to match the current system time (this is the reason why all other timestamps need to be relative). The command replay tryapplyparams is executed before all other modules are loaded and applies the parameters from the log and user-set parameters. Then as the last command, replay trystart will again apply the parameters and start the actual replay. Both commands do nothing if the environment variable replay is not set.

The ORB publisher rules allow to select which part of the system is replayed, as described above. They are only compiled for the posix SITL targets.

The time handling is still an open point, and needs to be implemented.