diff --git a/text/0018-extended-parameter-values.md b/text/0018-extended-parameter-values.md new file mode 100644 index 0000000..51ce934 --- /dev/null +++ b/text/0018-extended-parameter-values.md @@ -0,0 +1,256 @@ + * Start date: 2026-07-21 + * Contributors: Andrew Tridgell + * Related issues: incorporates explicit bytewise types and requester negotiation from [RFC PR #18](https://github.com/mavlink/rfcs/pull/18) + +# Summary + +Extend the existing parameter protocol with explicit bytewise types and +GCS opt-in. Signed and unsigned 32-bit integers occupy the existing four-byte +`param_value` field. Signed and unsigned 64-bit integers, REAL64, and 128-byte custom values use a +typed MAVLink2 extension. A negotiated IN_PROGRESS status supports asynchronous +writes. Together these allow components to replace PARAM_EXT with the existing +parameter microservice. This replaces the earlier extended-int32 encoding. + +# Motivation + +C-cast parameter transport rounds large int32 values. Legacy bytewise +transport avoids that rounding, but its type values overlap C-cast types and +float handling can corrupt integer bit patterns that represent signalling +NaNs. Explicit types make decoding unambiguous; raw-byte transport preserves +every bit. + +Parameter-over-MAVFTP already supports exact parameter reads and writes, +including individual reads. This proposal complements that transport by making +value encoding explicit in the existing PARAM_* messages, including set +acknowledgements, change notifications and asynchronous write status. + +# Wire format + +Add these entries to `MAV_PARAM_TYPE`: + +| Value | Name | Value location | +| --- | --- | --- | +| 11 | `MAV_PARAM_TYPE_EXTENDED` | `extended_data`, interpreted using `extended_type` | +| 12 | `MAV_PARAM_TYPE_BYTEWISE_INT32` | Four little-endian bytes in `param_value` | +| 13 | `MAV_PARAM_TYPE_BYTEWISE_UINT32` | Four little-endian bytes in `param_value` | +| 14 | `MAV_PARAM_TYPE_IN_PROGRESS` | Write status in PARAM_VALUE only; no stored value | + +`PARAM_VALUE` and `PARAM_SET` gain these extension fields, in this order: + +```xml + +Datatype of extended_data. +Extended parameter value. +``` + +The first byte of the extension area is therefore the type discriminator. +`MAV_PARAM_EXTENDED_TYPE` defines: + +| Value | Name | Encoding | +| --- | --- | --- | +| 0 | `MAV_PARAM_EXTENDED_TYPE_NONE` | No extended value | +| 1 | `MAV_PARAM_EXTENDED_TYPE_BYTEWISE_INT64` | Little-endian signed integer in the first eight data bytes | +| 2 | `MAV_PARAM_EXTENDED_TYPE_BYTEWISE_UINT64` | Little-endian unsigned integer in the first eight data bytes | +| 3 | `MAV_PARAM_EXTENDED_TYPE_BYTEWISE_REAL64` | IEEE-754 binary64, little-endian in the first eight data bytes | +| 4 | `MAV_PARAM_EXTENDED_TYPE_CUSTOM` | Opaque 128-byte value | + +Signed integers use two's complement. For either 32-bit type, `extended_type` +and all `extended_data` bytes must be zero. For an extended type, `param_type` +is EXTENDED and `param_value` is NaN. For INT64, UINT64 and REAL64 the +remaining 120 data bytes are zero. Unknown extended types must not be +interpreted as a float value. + +CUSTOM has the same capacity as PARAM_EXT_TYPE_CUSTOM. All 128 bytes, +including trailing zeros, belong to the value. MAVLink2 trimming removes zero +bytes only on the wire: receivers restore the full array before interpreting +it. Shorter application values are zero-padded; variable-length schemas must +encode their own length. Metadata or a component-specific schema defines +interpretation. There is no implicit text encoding or NUL termination. No +extended int32 subtype is defined. + +# Negotiation + +Append a `uint32_t supported_types` MAVLink2 extension to both +`PARAM_REQUEST_READ` and `PARAM_REQUEST_LIST`. It is a bitmask of +`MAV_PARAM_TYPES_SUPPORTED`, independent of the numeric values of the type enums: + +| Bit value | Encoding understood | +| --- | --- | +| 1 | BYTEWISE_INT32 | +| 2 | BYTEWISE_UINT32 | +| 4 | EXTENDED / BYTEWISE_INT64 | +| 8 | EXTENDED / BYTEWISE_UINT64 | +| 16 | EXTENDED / BYTEWISE_REAL64 | +| 32 | EXTENDED / CUSTOM | +| 64 | IN_PROGRESS write status | + +A zero or missing extension requests the existing encoding. Unknown bits are +ignored. The component may only emit a new type understood by the receiving +clients. Support applies to read replies, list streams, set acknowledgements +and unsolicited updates, not just the response to that particular request. + +A component advertises `MAV_PROTOCOL_CAPABILITY_PARAM_BYTEWISE` (2097152) when +it implements explicit bytewise encodings and requester negotiation. This does +not imply storage support for all storage types. A GCS checks this capability +before sending a new PARAM_SET type and advertises its receive mask first, +including when its parameter dictionary was obtained over FTP. + +PARAM_VALUE is broadcast and carries no destination. Implementations must +account for clients sharing a channel. The ArduPilot reference implementation +records each channel's first requester, allows that sole requester to replace +its mask, and intersects masks once another requester appears. After multiple +requesters have been observed it never enlarges the common mask until the +channel is reinitialized. A legacy request therefore prevents new encodings +from being sent on that shared channel. Replies queued before a downgrade are +also downgraded before transmission. This is conservative if clients leave; it +cannot detect passive listeners or multiple clients sharing one identity. +Separate links should be used when clients require different encodings. + +New types are only selected after MAVLink2 negotiation. MAVLink1 requests +imply zero support. A component must not silently cast extended-only values +into the legacy float field. If a requested parameter cannot be represented +using the negotiated mask, return PARAM_ERROR_TYPE_UNSUPPORTED with its name +and index. For a list, preserve a stable catalog, count and indices: send +PARAM_VALUE for representable entries and PARAM_ERROR_TYPE_UNSUPPORTED for the +others. An updated client records those indices as unavailable, rather than +retrying forever or treating an error as a value. If every entry is +unavailable, PARAM_ERROR alone provides no count; the client must report an +incomplete download after a bounded timeout. A legacy client cannot obtain a +complete catalog containing such entries; use a separate component/link or +retain PARAM_EXT while migrating. Do not renumber parameters depending on +requester capability. ArduPilot has no extended-only parameter storage in this +implementation. + +# Asynchronous writes + +A requester advertises IN_PROGRESS before writing. For a write that takes longer +than the normal response timeout, the component sends PARAM_VALUE with +`param_type = MAV_PARAM_TYPE_IN_PROGRESS`, the pending parameter's name, +`param_value = NaN`, and zero extension fields. Count and index are ignored. +This is a status, not a parameter type or value: it is invalid in PARAM_SET and +must never replace cached values/types or advance parameter-download accounting. + +Match a notification by source system, source component and parameter name. +Only a matching pending write has its response timeout restarted; unrelated +notifications have no effect. Send progress before the client's normal timeout +expires and repeat it while the operation is running. Clients may impose an +application-level total deadline. If progress stops, normal bounded retries +and timeout apply. Retransmitting an identical write must not restart an +operation already in progress. Components must serialize writes to each +parameter and reject conflicting writes with PARAM_ERROR_PERMISSION_DENIED +while busy. + +Completion is an ordinary PARAM_VALUE with the actual type and stored value. +Validation failures use existing PARAM_ERROR codes; add +`MAV_PARAM_ERROR_WRITE_FAIL = 9` for a supported write that fails during +execution. An error terminates the pending operation and does not change the +client's cached value. Clients check PARAM_ERROR destination as well as source +and parameter name, and may read again after failure to obtain the current value. +The resulting value must be acknowledged in an encoding the requester accepts; +otherwise reject the write before modifying storage. + +Unlike PARAM_EXT_ACK, PARAM_VALUE has no transaction ID or explicit ACCEPTED +result. Clients serialize writes per component/name and compare the returned +value with the requested value; unrelated reads or concurrent writers can +still make correlation ambiguous. Exact-value acknowledgements are the +contract here: a component must reject an unacceptable value rather than +silently clamp it. For REAL64 and CUSTOM, compare encoded bytes, preserving +signed zero, NaN payloads and custom padding. PARAM_ERROR also has no +operation identifier, so avoid concurrent read/write operations on the same +parameter during a pending write. + +If IN_PROGRESS is not negotiated, a server must not emit that marker. It can +complete within the normal timeout, safely coalesce retries until completion, +or reject an asynchronous operation with PARAM_ERROR_PERMISSION_DENIED before +starting it. This permits migration without exposing a new status to legacy +clients. + +# Replacing PARAM_EXT + +PARAM_EXT uses binary values in its 128-byte container, not decimal strings. +The replacement mapping is: + +| PARAM_EXT type or result | Replacement | +| --- | --- | +| UINT8, INT8, UINT16, INT16, REAL32 | Existing PARAM types and component's existing encoding | +| INT32, UINT32 | Explicit BYTEWISE_INT32 / BYTEWISE_UINT32 | +| INT64, UINT64 | EXTENDED with BYTEWISE_INT64 / BYTEWISE_UINT64 | +| REAL64 | EXTENDED with BYTEWISE_REAL64 | +| CUSTOM | EXTENDED with CUSTOM, all 128 bytes preserved | +| ACK ACCEPTED | PARAM_VALUE with actual type and exact stored value | +| ACK IN_PROGRESS | Negotiated PARAM_VALUE / IN_PROGRESS | +| ACK VALUE_UNSUPPORTED | PARAM_ERROR_VALUE_OUT_OF_RANGE or TYPE_MISMATCH as appropriate | +| ACK FAILED | PARAM_ERROR_WRITE_FAIL, or a more specific existing error | + +This covers the value capacity and asynchronous-write operations of PARAM_EXT, +with the correlation limitations above. Migration is per component: existing +PARAM_EXT clients still need that protocol until they adopt the new +negotiation and status handling. No PARAM_EXT messages or enum values are +removed. + +# Implementation requirements + +The 32-bit float field is a byte container for the explicit bytewise types. +Never obtain those bytes by converting an integer to a floating-point number, +or round-trip them through a float variable: signalling-NaN patterns may +change. Preserve raw payload bytes when decoding and forwarding messages, and +place raw bytes into outgoing packets before checksum and signature +calculation. Payload offsets are independent of header extensions such as +32-bit system IDs. + +ArduPilot sends BYTEWISE_INT32 for all int32 parameters on opted-in MAVLink2 +channels, including small values. Other parameter types retain legacy +encoding. It accepts BYTEWISE_INT32 into int32 storage and BYTEWISE_UINT32 +when the value fits signed int32. Type mismatch and overflow return +PARAM_ERROR; unsupported extended types and IN_PROGRESS writes return +TYPE_UNSUPPORTED. Rejected writes do not change storage. `MAV_OPTIONS` bit 1 +disables the capability and bytewise output; receive support remains +available. + +pymavlink provides raw field access before packing, exact decoding, request +mask helpers, and exact integer parsing and file storage up to UINT64_MAX. Its +Python generator preserves raw PARAM_VALUE/PARAM_SET bytes when forwarding +explicit 32-bit types. Low-level generated send functions that take a float +must not be used directly to send bytewise32 values: use the raw-byte wrapper +instead. pymavlink and MAVProxy extend pending-write timeouts for matching +IN_PROGRESS notifications and terminate on PARAM_ERROR. Neither caches a +progress marker. MAVProxy negotiates before bytewise sets and retains exact +values from messages and FTP logs. Floating-point parameter files use +round-trip decimal precision; CUSTOM values use an explicit `hex:` prefix +followed by hexadecimal bytes, alongside legacy numeric file entries. PX4 is +outside this project's scope. + +# Bandwidth and compatibility + +Message IDs, base layouts and CRC extras are unchanged. MAVLink2 removes zero +extension tails, so explicit 32-bit values have the same payload length as +legacy values: 25 bytes for PARAM_VALUE and 23 for PARAM_SET. A 64-bit value +adds at most nine bytes (one type and eight data bytes), with zero tails +removed. CUSTOM uses up to 129 extension bytes, making maximum PARAM_VALUE and +PARAM_SET payloads 154 and 152 bytes respectively, within MAVLink2's 255-byte +limit. Request masks add up to four bytes; zero masks add nothing. IN_PROGRESS +has zero extensions and the ordinary 25-byte PARAM_VALUE base payload. + +Legacy clients receive legacy encoding unless they share an already opted-in +channel without identifying themselves. Legacy servers ignore the new request +extensions. Updated clients must continue supporting legacy C-cast and +bytewise protocols for existing vehicles. Numeric assignments in this proposal +are provisional until upstream allocation. + +# Alternatives + +* Putting int32 in the extension avoids float API issues but costs up to five + extra bytes per value. This proposal instead requires raw-byte access and + uses negotiation to keep legacy clients on their existing path. +* PARAM_EXT_* provides a 128-byte binary container and explicit write results, + but requires a separate message family and does not preserve the compact + four-byte path for common values. +* MAVFTP provides another transport for exact reads and writes, including + individual reads. This proposal addresses the encoding of PARAM_* messages. +* A double extension holds int32 exactly but cannot hold every int64/uint64. + +# References + +* [Parameter protocol](https://mavlink.io/en/services/parameter.html) +* [Earlier explicit encoding proposal](https://github.com/mavlink/rfcs/pull/18) +* [ArduPilot project](https://github.com/orgs/ArduPilot/projects/34)