Files
PX4-Autopilot/Tools/esc_eeprom_decode.py
T
Jacob Dahlanddanielbuleandra ae7977f868 feat(dshot): log the ESC EEPROM dumps (#28800)
* fix(dshot): retry telemetry init until the motor mask is known

rc.serial issues `dshot telemetry` right after `dshot start`, and the first
Run() usually handles it before the mixer has reported the outputs.
initSettingsHandlers() then latches with an empty motor mask, so no ESC
settings handler is ever created and the EEPROM is never read.

Co-authored-by: danielbuleandra <daniel.buleandra@auterion.com>
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>

* feat(logger): log the ESC EEPROM dumps

The raw dump is logged through the existing queued esc_eeprom_read topic.
The layout depends on the ESC firmware and its EEPROM revision, so decoding
happens offline in Tools/esc_eeprom_decode.py against the schema the ESC
vendor publishes.

A subscriber that joins after the first publish only sees the newest queued
entry. The logger subscribes on a sweep that can lag a new topic by half a
second while all ESCs answer within milliseconds of each other, so the topic
is advertised when the settings handlers are created, well before the first
request goes out at ESC_INIT_TELEM_DELAY.

Co-authored-by: danielbuleandra <daniel.buleandra@auterion.com>
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>

* feat(dshot): publish and refresh cached ESC settings

Logs can start after boot or during flight, so cached settings must be published independently of arming. The queue must retain a complete set of ESC dumps for the logger and MAVLink stream.

Keep programming uninterrupted and cached data valid across saves and reconnects. Failed reads remain pending with spaced retries so one unresponsive ESC cannot starve the rest.

Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>
Assisted-by: Codex

* fix(tools): decode repeated ESC dumps across firmware layouts

Logs contain repeated dumps and may mix EEPROM layouts. Collapse unchanged samples, apply the schema's version bounds and disabled values, and keep overlays immutable so one ESC cannot change another's decoding.

Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>
Assisted-by: Codex

* fix(dshot): bound ESC settings read retries

A pending settings read blocks the telemetry round-robin until it completes or times out, and the retry spacing was kept per ESC. With several ESCs that never answer, reads ran back to back, telemetry went stale and the arming check reported every ESC offline.

Space retries across all ESCs, give up on one after three failed reads until it reconnects or is asked for again, and time a read out at 100 ms: AM32 answers within milliseconds of the sixth frame, and anything under the 400 ms arming-check window keeps telemetry fresh.

Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>

* fix(dshot): keep programming active until the save is sent

The flag was cleared when the save command was queued, so it no longer guarded the save itself. Only the finished() gate in the vehicle command loop kept a CONFIGURE_ACTUATOR from replacing it. Clear the flag when the last save frame goes out so the guard covers the whole sequence.

Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>

* fix(dshot): send each ESC EEPROM read once per MAVLink link

The once-a-second republish that keeps the dumps in every log also went out on every link with ESC_EEPROM configured, low-bandwidth ones included, one message per ESC per second. timestamp_sample records when a dump was read, so the stream forwards each read once per link and still reaches a link that connects later.

A motor waiting on an EDT retry or out of EDT attempts took the command slot without sending anything, so settings reads never ran while any ESC left EDT unconfirmed. Vehicle commands now wait for an outstanding settings read, which ends within 100 ms, instead of CONFIGURE_ACTUATOR being rejected.

Assisted-by: Claude:claude-opus-5-5

* fix(msg): queue one ESC EEPROM dump per esc_status ESC

The republish queues every cached dump at once, so the queue needs one entry per ESC and no more. DSHOT_MAX_MOTORS is capped at CONNECTED_ESC_MAX, and the static_assert in AM32Settings.cpp keeps the two in step.

Assisted-by: Claude:claude-opus-5-5

* fix(dshot): mark the settings save explicitly

Programming ended when any finished command carried the value 12, which the address and value frames do whenever they write offset 12 (active brake power) or a value of 12. The write then stopped part way, with the ESC still in programming mode and the next write resuming mid-sequence.

Assisted-by: Claude:claude-opus-5-5

* fix(dshot): keep settings requests and writes to configured motors

An all-ESC save requested settings for every motor slot, and the single-ESC request and write paths took any index below the motor limit. Reads only go to motors with a settings handler, so the other bits stayed pending in dshot status forever. A write to an index with no output was never sent, so the command never finished and blocked every later command and the serial telemetry until reboot.

Assisted-by: Claude:claude-opus-5-5

* docs(dshot): describe the logged ESC settings and their decoding

The page said the settings are only read at boot and only reach a ground station.

Assisted-by: Claude:claude-opus-5-5

* feat(dshot): set the ciclicity to 10s for eeprom logging

---------

Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>
Co-authored-by: danielbuleandra <daniel.buleandra@auterion.com>
2026-11-03 08:14:51 -07:00

332 lines
11 KiB
Python
Executable File

#!/usr/bin/env python3
"""
Decode the ESC EEPROM dumps logged in the esc_eeprom_read topic of a PX4 ULog file.
The schema describes the EEPROM layout for one ESC firmware and EEPROM revision. It is published by
the ESC firmware or its vendor, not by PX4, so it has to be passed in with --schema.
Install: pip install pyulog
Run: ./esc_eeprom_decode.py <log.ulg> --schema eeprom.json
./esc_eeprom_decode.py <log.ulg> --schema eeprom.json --esc 1
"""
import argparse
import json
import sys
TOPIC = 'esc_eeprom_read'
# MAVLink ESC_FIRMWARE enum, as logged in the 'firmware' field
ESC_FIRMWARE = {0: 'UNKNOWN', 1: 'AM32'}
def load_dumps(log_path):
"""Return every EEPROM dump found in the log, as {'time', 'esc', 'firmware', 'data'} dicts."""
from pyulog import ULog
ulog = ULog(log_path, [TOPIC])
dumps = []
for dataset in ulog.data_list:
if dataset.name != TOPIC:
continue
fields = dataset.data
for row in range(len(fields['timestamp'])):
length = int(fields['length'][row])
data = bytes(int(fields[f'data[{i}]'][row]) for i in range(length))
dumps.append({
'time': int(fields['timestamp'][row]) / 1e6,
'esc': int(fields['index'][row]) + 1,
'firmware': int(fields['firmware'][row]),
'data': data,
})
dumps.sort(key=lambda dump: (dump['time'], dump['esc']))
return dumps
def collapse_repeats(dumps):
"""Drop dumps identical to the previous one from the same ESC, counting them on the one kept. The
driver republishes every cached dump every 10 seconds so that logs started later still carry them."""
kept = []
last = {}
for dump in dumps:
previous = last.get(dump['esc'])
if previous and (previous['firmware'], previous['data']) == (dump['firmware'], dump['data']):
previous['repeats'] += 1
previous['last_time'] = dump['time']
continue
dump['repeats'] = 0
last[dump['esc']] = dump
kept.append(dump)
return kept
def load_schema(schema_path):
"""Load the EEPROM layout schema from a local JSON file."""
with open(schema_path, encoding='utf-8') as schema_file:
return json.load(schema_file)
def firmware_key(text):
"""Turn '2.18' into the tuple (2, 18), so firmware versions can be compared with <, >=, etc."""
major, _, minor = text.partition('.')
return (int(major), int(minor or 0))
def applies(item, eeprom_version, firmware_version):
"""Whether a field or group exists at these EEPROM layout and firmware versions, bounded the same
way as in the configurator."""
if item.get('minEepromVersion') is not None and eeprom_version < item['minEepromVersion']:
return False
if item.get('maxEepromVersion') is not None and eeprom_version > item['maxEepromVersion']:
return False
if item.get('minFirmwareVersion') and firmware_version < firmware_key(item['minFirmwareVersion']):
return False
if item.get('maxFirmwareVersion') and firmware_version > firmware_key(item['maxFirmwareVersion']):
return False
return True
def apply_overlay(resolved, overlay):
"""Apply a version overlay onto a field definition: objects merge one level deep, everything
else replaces the previous value."""
for key, value in overlay.items():
if isinstance(value, dict) and isinstance(resolved.get(key), dict):
resolved[key] = {**resolved[key], **value}
else:
resolved[key] = value
def resolve_field(field, eeprom_version, firmware_version):
"""Collapse a field definition and its version overlays into a single definition.
Overlays are applied in the order the schema defines: 'default' first, then 'eeprom:N+' in
ascending N, then 'firmware:X.Y+' in ascending X.Y, keeping only those the reported versions
satisfy.
"""
resolved = dict(field)
versions = resolved.pop('versions', {})
if 'default' in versions:
apply_overlay(resolved, versions['default'])
eeprom_overlays = []
firmware_overlays = []
for key, overlay in versions.items():
if key.startswith('eeprom:') and key.endswith('+'):
eeprom_overlays.append((int(key[len('eeprom:'):-1]), overlay))
elif key.startswith('firmware:') and key.endswith('+'):
firmware_overlays.append((firmware_key(key[len('firmware:'):-1]), overlay))
# The 'N+' / 'X.Y+' thresholds in one field are always unique, so sorting the (threshold,
# overlay) pairs directly sorts by threshold.
eeprom_overlays.sort()
firmware_overlays.sort()
for minimum, overlay in eeprom_overlays:
if eeprom_version >= minimum:
apply_overlay(resolved, overlay)
for minimum, overlay in firmware_overlays:
if firmware_version >= minimum:
apply_overlay(resolved, overlay)
return resolved
def read_raw(data, offset, size):
"""Read a little-endian unsigned integer, or None if it falls outside the dump."""
if offset + size > len(data):
return None
return int.from_bytes(data[offset:offset + size], 'little')
def is_disabled(field, raw):
"""Whether the raw value switches the setting off: the field's disabled value, or anything outside
its raw range once the field has a disabled value at all."""
disabled = field.get('disabledValue')
if not disabled:
return False
limits = field.get('raw') or {}
return raw == disabled.get('raw') or raw < limits.get('min', raw) or raw > limits.get('max', raw)
def format_value(field, raw):
"""Render a raw value using the field's enum table or display scaling."""
field_type = field.get('type')
unit = field.get('unit', '')
if is_disabled(field, raw):
return f'{field["disabledValue"].get("display", "Disabled")} ({raw})'
if field_type == 'enum':
for entry in field.get('values', []):
if entry.get('raw') == raw:
return f'{entry.get("name", raw)} ({raw})'
return f'{raw} (not in schema)'
if field_type == 'bool':
return f'{"On" if raw else "Off"} ({raw})'
display = field.get('display')
if display:
scaled = raw * display.get('factor', 1) + display.get('offset', 0)
text = f'{scaled:.{display.get("decimals", 0)}f}'
return f'{text} {unit}'.strip() + f' (raw {raw})'
return f'{raw} {unit}'.strip()
def eeprom_version_of(data, fields):
"""Read the byte the schema calls 'eepromVersion', or 0 if the schema has no such field."""
field = fields.get('eepromVersion')
if not field:
return 0
return read_raw(data, field['offset'], field.get('size', 1)) or 0
def firmware_version_of(data, fields):
"""Read the (major, minor) firmware version the schema reports, or (0, 0) if it can't."""
major_field = fields.get('firmwareMajor')
minor_field = fields.get('firmwareMinor')
if not major_field or not minor_field:
return (0, 0)
major = read_raw(data, major_field['offset'], major_field.get('size', 1)) or 0
minor = read_raw(data, minor_field['offset'], minor_field.get('size', 1)) or 0
return (major, minor)
def decode(dump, schema):
"""Decode one raw dump into {name: (label, text)} for every setting present in it."""
data = dump['data']
fields = schema['fields']
eeprom_version = eeprom_version_of(data, fields)
firmware_version = firmware_version_of(data, fields)
settings = {}
for name, field in fields.items():
if field.get('type') == 'reserved':
continue
if not applies(field, eeprom_version, firmware_version):
continue
resolved = resolve_field(field, eeprom_version, firmware_version)
raw = read_raw(data, resolved['offset'], resolved.get('size', 1))
if raw is not None:
settings[name] = (resolved.get('name', name), format_value(resolved, raw))
return eeprom_version, firmware_version, settings
def print_dump(dump, schema, index, total):
eeprom_version, firmware_version, settings = decode(dump, schema)
layout = schema.get('eepromVersions', {}).get(str(eeprom_version), {})
title = f' ESC {dump["esc"]} at {dump["time"]:.1f} s '
if dump.get('repeats'):
title += f'(+{dump["repeats"]} unchanged, last {dump["last_time"]:.1f} s) '
if total > 1:
title += f'({index}/{total}) '
print()
print(title.center(64, '='))
print(f' firmware type {ESC_FIRMWARE.get(dump["firmware"], dump["firmware"])}')
print(f' firmware {firmware_version[0]}.{firmware_version[1]}')
print(f' EEPROM layout {eeprom_version} ({layout.get("description", "unknown")})')
groups = schema.get('groups', {})
order = list(schema.get('groupOrder', []))
for group_key in groups:
if group_key not in order:
order.append(group_key)
placed = set()
for group_key in order:
group = groups.get(group_key, {})
names = [name for name in group.get('fields', []) if name in settings]
placed.update(names)
# A group the ESC's versions rule out hides its fields rather than moving them to 'Other'
if not names or not applies(group, eeprom_version, firmware_version):
continue
print(f'\n {group.get("name", group_key)}')
for name in names:
label, text = settings[name]
print(f' {label:<28} {text}')
leftover = [name for name in settings if name not in placed]
if leftover:
print('\n Other')
for name in leftover:
label, text = settings[name]
print(f' {label:<28} {text}')
def main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('log', help='ULog file (.ulg) containing esc_eeprom_read entries')
parser.add_argument('-s', '--schema', required=True, help='EEPROM layout schema JSON, published by the ESC firmware')
parser.add_argument('-e', '--esc', type=int, help='only show this ESC (1-based, as printed by PX4)')
args = parser.parse_args()
dumps = load_dumps(args.log)
if args.esc is not None:
dumps = [dump for dump in dumps if dump['esc'] == args.esc]
dumps = collapse_repeats(dumps)
if not dumps:
sys.exit(f'no {TOPIC} messages in {args.log}')
schema = load_schema(args.schema)
print(f'log: {args.log}')
print(f'schema: {schema.get("title", "untitled")} v{schema.get("version", "?")}')
for index, dump in enumerate(dumps, start=1):
print_dump(dump, schema, index, len(dumps))
if __name__ == '__main__':
try:
main()
except Exception as error:
sys.exit(f'error: {error}')