Commit Graph
7 Commits
Author SHA1 Message Date
af006cf1c3 feat(motor_failure_detector): add current-residual detection (#27711)
* feat(motor_failure_detector): add current-residual detection library

Per-motor model-based motor-failure detector for multirotors. Predicts the
expected ESC current from the commanded actuator signal
(I_exp = current_slope_a*u + current_idle_a) and trips on a low-pass-filtered
residual held past a trip band for a persistence time. The command (not measured
rpm) is the fault-independent reference: a dead motor that draws no current while
still commanded produces a large residual, whereas a current-vs-rpm model would
mask a clean stop.

Band and persistence are configurable per residual direction. Undercurrent is the
fault-consistent side -- an open circuit, a shed propeller or a desync makes a
motor draw less than commanded -- while overcurrent covers a winding short,
mechanical jam or demagnetization. They are separate because the benign residual
is far from symmetric: a fast throttle increase draws acceleration current the
static model cannot predict, so overcurrent excursions are large and unbounded,
whereas an undercurrent residual is floored at -I_exp. A single band sized to
reject the overcurrent transients can therefore end up wider than the largest
undercurrent fault a motor can physically produce, which makes a full current
collapse undetectable rather than merely slow. Set undercurrent_threshold_a <= 0
to keep one symmetric band on the residual magnitude.

Built on PX4 primitives rather than hand-rolled equivalents: update() takes a
timestamp (esc_status.timestamp in flight, the log timestamp in replay) and
derives dt internally; the residual filter is a mathlib AlphaFilter; the
persistence debounce is a systemlib::Hysteresis per direction (with a sticky latch
on top, and reset across data gaps so a dropout can't complete a pending trip).

The data-gap threshold (kMaxGap = 0.3 s) is sized above the ~10 Hz arming-check
feed interval and below the 400 ms ESC-offline timeout: a live feed is never
mistaken for a gap (a smaller value equal to the feed interval reset the filter
and debounce on the majority of jittered ticks, so a sustained fault never
survived the persistence window -- offline replay: 0/100 sampling phases latched
vs 100/100 gap-free), while a genuine outage still resets and is handled by the
separate ESC liveness guard.

Includes an 18-case gtest spanning the failure-mode spectrum and edge cases:
healthy-never-trips (incl. high throttle), open-circuit and overcurrent trips,
severe prop-loss undercurrent, the missed ~50% partial loss (documents the
detection floor), the per-direction bands (a tighter undercurrent band catching a
collapse one symmetric band misses, band independence, per-side persistence),
persistence (short spike crosses-threshold-but-recovers), gap reset, the 10 Hz
feed vs. real-gap boundary, dropout-holds, multi-motor, the exclusion gate,
monitor-only, and reset/latch.

* refactor(motor_failure_detector): resolve the undercurrent band at configure time

An unset undercurrent threshold now mirrors the overcurrent band in
configure(), so update() runs the same two bands in every configuration and
no longer carries a second threshold test that reads like a repeat of the
disable gate. A symmetric band is the two-band case with equal bands.

Also name esc_status CONNECTED_ESC_MAX at kMaxMotors so the constant is
findable, use the PX4 time literals for the persistence conversions, and
drop the excluded temporary.

* feat(commander): use current-residual detector for the ESC motor-failure check

Replace the flat, instantaneous current check in EscChecks::checkMotorStatus with
the MotorFailureDetector library (added in the parent PR). Per motor it predicts
the expected ESC current from the commanded thrust
(I_exp = MOTFAIL_C2T*u + MOTFAIL_IDLE) and trips on the low-pass-filtered
residual, with an independent band and hold time per direction: MOTFAIL_UNDER held
for MOTFAIL_UND_TIME below the expectation, MOTFAIL_OVER held for MOTFAIL_OVR_TIME
above it.

Because the two directions need separate bands, the single band and hold time are
renamed to name the direction they act on: MOTFAIL_OFF -> MOTFAIL_OVER and
MOTFAIL_TIME -> MOTFAIL_OVR_TIME, migrated in param_translation.cpp. The old band
applied to both directions, so it carries over to the overcurrent side, and
MOTFAIL_UNDER defaults to 0, which keeps a single symmetric band -- an existing
vehicle behaves as before. MOTFAIL_IDLE also defaults to 0, and the whole check
stays gated by FD_ACT_EN (off by default).

The detector is driven off esc_status.timestamp, so a telemetry stall freezes the
filter instead of integrating stale current; the separate 400 ms ESC telemetry
timeout stays in checkEscOnline(). It is reconfigured from parameters while
disarmed, so a parameter change never resets a latched failure mid-flight. The
motor_failure_mask still feeds failure_detector_status via
FailureDetector::publishStatus, leaving the ControlAllocator reaction path
untouched.

The mask reports the detector's sticky latch alone, not whether the motor is
currently evaluable. That preserves the invariant the previous check spelled out
-- "Only set, never clear mid-air: stopping the motor in response could make it
appear healthy again" -- because handling a failure makes exactly that motor
unevaluable: the control allocator drops it, which publishes a NAN command,
which excludes it from the detector. Re-deriving the mask from the exclusion
state clears it, the allocator restores all motors, and the next check re-trips.
Measured in SIH on a hexarotor with a missing propeller: 83 mask transitions in
nine seconds, with a reversed recovery motor swinging between idle and near-full
reverse at the 10 Hz check rate. An ESC going offline after a real failure
retracts the report the same way. The latch is only ever set on a tick where the
motor was evaluable, so reporting it alone cannot report a motor that was never
judged.

* feat(motor_failure_detector): add offline ulog replay tool

Add a host tool that replays a recorded .ulg through the real MotorFailureDetector
and prints the per-motor trip verdict, for validating and tuning the detector
offline (EKF2-replay style: the real library drives recorded data, not a
re-implementation).

Built on ulog_cpp (PX4's C++ ULog library, added as a git submodule and built for
the posix host target only), so it reads ESC current, motor command and arming
state by field name from the log's own format -- one binary replays logs of any
PX4 message version with no rebuild. The feed mirrors the runtime in escCheck:
evaluated only while armed, at the runtime's 10 Hz cadence, with the per-ESC
current-report startup latch and reset-on-disarm. Config comes from the log's
MOTFAIL_* parameters with a built-in calibration fallback, and --set
MOTFAIL_NAME=value overrides any knob for what-if retuning. Because the runtime's
exact 10 Hz sample phase cannot be reconstructed offline, a single pass over the log
drives several detector instances on staggered grid offsets and reports a per-motor
peak band and trip fraction rather than a single phase's verdict, flagging when the
result is phase-dependent.

batch_replay.sh runs the tool over a folder of logs and summarizes each verdict
(OK / FAIL / SKIP) from its exit code.

Off by default so it never enters the firmware build or CI: it is a posix-only,
EXCLUDE_FROM_ALL host executable gated behind the MFD_REPLAY_TOOL cache option.
Build it with:

    cmake -DMFD_REPLAY_TOOL=ON build/px4_sitl_default
    ninja -C build/px4_sitl_default mfd_replay

* docs(motor_failure_detector): document the current-residual motor-failure check

Add a configuration page for motor failure detection, next to the other
actuator topics, and a developer page for the offline replay tool:

- what the check compares and why the under/overcurrent bands are separate
- the new MOTFAIL_* parameters, including the rename migration
- how to fit the current model and pick the bands from the vehicle's own
  logs with mfd_replay / batch_replay.sh

The "Motor Failure Trigger" section in the failsafe page described the
pre-rename parameters, the unfiltered symmetric comparison and claimed the
failure was non-latching, so it is reduced to a summary of the trigger and
its action, pointing at the new page for the thresholds.

* docs(docs): Subedit for simpler introductions and terminology

* docs(docs): Improve upgrading from v1.18 section

* feat(motor_failure_detector): add current-model fitting tool

mfd_fit derives MOTFAIL_C2T and MOTFAIL_IDLE from recorded logs, so the model
and the thresholds are taken from the same data.

Like mfd_replay it reads ESC current, motor command and arming state through
ulog_cpp by field name, so one binary handles any log version. Each log is a
single streaming pass that accumulates only the sums a least-squares line needs,
so memory does not grow with the log or with the number of logs, and a whole
corpus can be fitted in one invocation.

Alongside the parameters it reports the residual spread the trip bands have to
clear, and each motor's bias against the shared model.

* docs(docs): document motor failure detection tuning and the fitting tool

Follow-up to review of the detection and replay pages:

- fix the link left behind by the Failure Response -> Failure Action rename
- drop the Multicopter badge: the check has no vehicle-type gate, so a VTOL's
  multirotor motors and its forward-flight motor are both checked
- describe the model-fitting step without the statistics jargon
- state that only healthy logs may be used, for the fit and the thresholds
- document mfd_fit, and reduce the release note to a pointer

* feat(motor_failure_detector): report residual extremes and add a band sweep

The replay tool printed the largest residual magnitude, which hid whether an
excursion was above or below the model. Since the two trip bands are chosen per
direction, that left the reader running one band at a time to find out. It now
prints the signed minimum and maximum instead, so each band's headroom is
readable straight from the table.

Add sweep_matrix.sh alongside batch_replay.sh: it sweeps one band against its
hold time over a set of logs and prints how many logs false-trip at each pair,
so the lowest usable band per hold time is the first zero in a row.

Both tools now number motors from 1, matching the "Motor N ..." the vehicle
reports, rather than from 0.

* docs(docs): rework the motor failure calibration walkthrough

Follow a real calibration through both pages and fix what it ran into:

- the replay page's worked example is one vehicle end to end, with every
  number and pasted output taken from that corpus
- the fitting tool, the single-log run and the band sweeps are ordered the way
  the work is actually done, with replaying a whole fleet last
- the sweeps are shown as a matrix of false trips per (hold time, band), and
  the grids are derived from the residual extremes rather than guessed
- commands match the output they produce, and say what they need: mfd_replay
  without --set uses a fallback calibration that belongs to another vehicle
- vehicle_status is optional, not required, for the replay
- drop the repetition: batch_replay.sh, the hover-current ceiling and the
  phase-sensitivity note were each explained two or three times

* fix(motor_failure_detector): separate configure and reset with the appropriate triggers

* refactor(motor_failure_detector): clarify leading edge hysteresis as trigger time compared to "persistence"

* fix(escCheck): encapsulate _motor_failure_direction handling into MotorFailureDetector

* fix(astyle): exclude ulog_cpp submodule form check

* fix(motor_failure_detector_tools): fail log replay when no log is present

* refactor(motor_failure_detector): Improve documentation

* fix(escCheck): check load per ESC not per motor function

This was already done in my original implementation. It supports checking multiple ESCs mapped to the same motor function individually. It's also consistent with other ESC checks that run for every ESC that reports telemetry.

* fix(motor_failure_detector): run update() per motor and consider individual telemetry timestamps

* fix(motor_failure_detector): simplify output interface to getters of actually used results

* fix(motor_failure_detector): inline what doesn't need to be separate

* fix(motor_failure_detector): make the bands independent and drop special cases

- Each trip band is now independent: a threshold <= 0 disables only that band. The undercurrent band no longer copies the overcurrent one.
- Drop the MOTFAIL_OFF/MOTFAIL_TIME parameter migration. The new parameters start at their defaults.
- Treat a non-finite current like a non-finite command (not evaluated, filter and debounce reset). No driver reports NaN current, so the path only existed for its test.
- Compute the residual as max(current, 0) - I_expected. Same as before, but written such that it's obvious: regeneration counts as 0A.
- Drop unnecessary trigger time parameter clamp to >= 0 before converting to unsigned microseconds, metadate already decalres that as out of range.
- Count each ESC sample once by its own timestamp in mfd_fit, like the motor_failure_detector.

* fix(motor_failure_detector): check all still considered online ESC telemetry

* fix(motor_failure_detector): declare all link dependencies

* fix(motor_failure_detector_tools): reuse log reading logic to reduce duplication

* fix(motor_failure_detector): replace reversible flag getting passed into the detector for separate handling

Having a NAN command already handles that.

* test(escChecks): add unit tests for load based failure detection

---------

Co-authored-by: Hamish Willee <hamishwillee@gmail.com>
Co-authored-by: Matthias Grob <maetugr@gmail.com>
Co-authored-by: Andrea Bernasconi <andrea.bernasconi@auterion.com>
2026-10-26 16:45:00 +01:00
Silvan 43c6d582ad feat(fw_rate_control): remove 2% thrust motor cutoff
This avoids non-linear beahvior around the 2% mark, especially when the motor
has a slew rate defined (as it is the case for IC engines).

Signed-off-by: Silvan <silvan@auterion.com>
2026-10-15 14:09:11 +02:00
PX4BuildBot 86716507be docs: auto-sync metadata [skip ci]
Co-Authored-By: PX4 BuildBot <bot@px4.io>
2026-09-24 10:17:06 +00:00
Silvan 1bcefdd86c feat(docs): improve IC engine docs
- add dedicated page for ICE config and explanation
- update diagrams and screenshots

Signed-off-by: Silvan <silvan@auterion.com>
2026-09-24 12:10:34 +02:00
3a35c014be feat(control_allocator): reversible-motor failure recovery for hexarotors (#28078)
* feat(control_allocator): add reversible-motor failure recovery for hexarotors

On a single motor failure (CA_FAILURE_MODE), handle the failed motor's
geometric-opposite motor on a hexarotor to recover the lost control authority:
  1 = remove failed + shut off opposite (restores symmetry)
  2 = remove failed + make opposite reversible (assumes a reverse-capable ESC)
Non-hex frames have no opposite and keep the upstream remove-only behavior.

The opposite motor is found geometrically (opposite spin, closest antipode) and
only for hexarotors. Reverse thrust is modeled as CA_REV_THR_FRAC of forward
thrust and the reversible actuator range is rescaled to [-1, 0] on publish.
Consumes failure_detector_status.motor_failure_mask, independent of detection.

* feat(uavcan): render reversible motor outputs as signed RawCommand

Re-encode reversible channels from the mixer's [min,max] range to a signed
RawCommand (negative = reverse). A channel at its disarmed value is passed
through untouched, so a stopped motor is never inverted to reverse.

* fix(dshot): match per-channel minimums to the 3D encoding

calculate_output_value() applies the 3D split when DSHOT_3D_ENABLE is set or the
channel is reversible, but the per-channel minimum was derived from the
reversible bit alone. Two problems followed.

A reversible channel made reversible at runtime kept a stale minimum, so its
neutral no longer landed on the mid-throttle split. A non-reversible channel
with 3D enabled was worse: it mapped from DSHOT_MIN, which sits below the split,
so zero commanded thrust decoded as near-full reverse.

Select the minimum from the same two facts the encoder uses. A reversible
channel keeps the range bottom so neutral lands on the split, a forward-only
channel starts just above it, and without 3D the DSHOT_MIN idle floor applies as
before. For a split channel the idle floor is applied per direction inside
convert_output_to_3d_scaling(), so it must not be added to the minimum. The
deadband is now applied only to reversible channels, since a channel that cannot
be commanded negative has no neutral to guard.

Apply the minimums from a new OutputModuleInterface::reversibleMaskChanged(),
called once the new mask is known and before any output is computed with it.
Applying them from Run() left the range one cycle behind the mask it depends on,
which in 3D mode changed the sign of the output on the transition.

The reversible mask can now change while armed, since failure recovery reverses a
motor in flight, so the reversibleOutputs() contract is updated to say so.

Changes output values for every DSHOT_3D_ENABLE=1 setup that has any
non-reversible motor. Verified on fmu-v4 with 3D-mode ESCs: forward-only
channels floor at 1001 with no samples below the split across 85 s armed, and a
runtime-reversed recovery motor tracks yaw input through the split in both
directions.

* docs(control_allocator): document motor failure recovery

Add a Motor Failure Recovery page covering CA_FAILURE_MODE, the hexarotor
stop-or-reverse behaviour of the opposite motor, CA_REV_THR_FRAC, the
detection prerequisites and which output protocols can drive a motor in
reverse.

Document the signed DroneCAN RawCommand encoding for reversible motors,
and correct the failure detector section, which claimed motor failures
are non-latching while the current-based check holds a flagged motor
until the vehicle disarms.

Add the page to the navigation and to the v1.18 release notes.

* docs(docs): subedit

* docs(docs): subedit

* refactor(control_allocator): align CA_REV_THR_FRAC naming and description

Rename the parameter member to _param_ca_rev_thr_frac to match the
parameter name, and note in the description that the reverse thrust
fraction is mostly a property of the propeller.

* docs(motor_failure_recovery): move under Actuators and target v1.19

The page documents how control allocation is reconfigured, so list it
under Actuators next to Actuator Allocation instead of under Safety, and
link it from the Actuators and multicopter Features indexes. Safety
configuration still links to it from the motor failure trigger section.

The release branch for v1.18 is already cut, so move the release notes
to main.md and set the version badges to main (PX4 v1.19).

* docs(motor_failure_recovery): spell out the reversing requirements

Any of the six motors can fail, so every motor has to be reverse-capable,
not just one. List the settings that mode 2 needs, split the per-protocol
detail out (DroneCAN reverses from the signed RawCommand alone, DShot also
needs the ESCs in 3D mode and DSHOT_3D_ENABLE to match), and say what
happens when they are not set: a reverse command comes out as forward
thrust, and on PWM the recovery motor sits at around half throttle.

Also note that motors do not have to be marked bidirectional (CA_R_REV)
for this, since the recovery motor is made reversible at runtime.

* Apply suggestions from code review

Co-authored-by: Hamish Willee <hamishwillee@gmail.com>

* fix(uavcan_main): log final value in actuator_outputs, always run through reversible logic

We want worst case execution time all the time and as little control paths as possible. Also actuator_outputs is used to debug ESC behavior and should contain the final value that is sent out over the wire.

* refactor(DShot): directly calculate minimum dshot value in apply_min_values()

* refactor(control_allocator): static_cast, correct CA_FAILURE_MODE description

* fix(control_allocator): treat undefined parameter values as default, use matrix function to zero a collumn

---------

Co-authored-by: Hamish Willee <hamishwillee@gmail.com>
Co-authored-by: Matthias Grob <maetugr@gmail.com>
2026-08-07 09:46:17 +00:00
Matthias Grob 2bc04f91f8 docs: fix all end of file newlines
Note: Some files are autogenerated and I did not yet
go through each source of generation to fix it there.
Instead I adjusted the filter to only exclude those
such that we can fix things in steps.
2025-06-26 11:03:23 -07:00
Hamish WilleeandRamon Roche 88d623bedb Move PX4 Guide source into /docs (#24490)
* Add vitepress tree

* Update existing workflows so they dont trigger on changes in the docs path

* Add nojekyll, package.json, LICENCE etc

* Add crowdin docs upload/download scripts

* Add docs flaw checker workflows

* Used docs prefix for docs workflows

* Crowdin obvious fixes

* ci: docs move to self hosted runner

runs on a beefy server for faster builds

Signed-off-by: Ramon Roche <mrpollo@gmail.com>

* ci: don't run build action for docs or ci changes

Signed-off-by: Ramon Roche <mrpollo@gmail.com>

* ci: update runners

Signed-off-by: Ramon Roche <mrpollo@gmail.com>

* Add docs/en

* Add docs assets and scripts

* Fix up editlinks to point to PX4 sources

* Download just the translations that are supported

* Add translation sources for zh, uk, ko

* Update latest tranlsation and uorb graphs

* update vitepress to latest

---------

Signed-off-by: Ramon Roche <mrpollo@gmail.com>
Co-authored-by: Ramon Roche <mrpollo@gmail.com>
2025-03-13 16:08:27 +11:00