[PATCH v10] api: packet: redefine packet data sharing with packet references by JannePeltonen · Pull Request #2287 · OpenDataPlane/odp

JannePeltonen · 2025-11-11T09:48:04Z

Current API regarding dynamic packet references is not easy to efficiently support in some ODP implementations with HW accelerated packet output due to the need to update reference counts after TX completion and based on that conditionally free packet buffers. odp_packet_ref() has been implemented only as a full packet copy in all ODP implementation.

Redefine the semantics of odp_packet_ref() and the packets involved in packet data sharing in a way that hopefully makes it simpler to support packet data sharing with restrictions in packet output and with per-packet reference counting instead of per-segment reference counting. The new semantics also enable use cases that were not previously possible.

Specify better the rules regarding packet data layout manipulation operations, allowing private packet data also in the tail of the packets that reference other packets. Pull and truncate operations may involve shared packet data, in which case they just remove that data from the packet, not affecting the packet owning the data. All extend and truncate operations always produce non-shared packet data.

Change odp_packet_has_ref() to indicate whether a packet is referenced by another packet, not whether the packet shares data with another packet. In partivular, odp_packet_ref() returns a packet that shares data with the given packet but is not itself referenced by other packets. Specify that packet data and layout of packets referenced by other packets can not be modified at all.

Introduce pktio capability bits to indicate whether a pktio supports sending packets with shared data without using the dont_free flag. Sending with the dont_free flag is supported with all types of packets.

ashwinyes · 2025-11-20T06:34:52Z

include/odp/api/spec/packet.h

+ * Packets returned by this function can be used normally in ODP API functions
+ * unless otherwise noted. There are restrictions in packet output.
+ *
+ * Referenced packets can be used in ODP API functions that treat packet data


I think we should use one word to refer. Either base packet or referenced packet.

Changed in v2 so that we talk about referenced packets, not base packets.

ashwinyes · 2025-11-20T06:37:21Z

include/odp/api/spec/packet.h

+ * of a static reference it also shares metadata. Packet data and data layout
+ * of packet that have references must be treated as read only.
+ *
+ * Packets created through odp_packet_ref() or odp_packet_ref_hdr() share data


odp_packet_ref_hdr() --> odp_packet_ref_pkt()

Fixed in v2

JannePeltonen · 2025-12-02T14:08:44Z

v2: Many text clarifications and typo fixes. Removed base packet from terminology. Made the packet I/O capabilities more granular. Added reference related text in odp_packet_concat() specification. Added a new function to test whether a packet is a referencing packet. Mentioned that the functions that create referencing packets and static references can be called simultaneously in multiple threads for the same packet.

JannePeltonen · 2025-12-09T11:11:08Z

v3: Specified that sending shared packets to TM must obey the restrictions of the egress pktio (but currently TM does not support the dont_free flag, which maybe needs to change).

JannePeltonen · 2025-12-10T17:04:52Z

v4:

Renamed odp_pktout_shared_packet_capability_t to odp_pktout_packet_ref_capability_t and rewrote the description for better clarity.
Renamed odp_pktio_capability_t::shared_pkt_consume to odp_pktio_capability_t::packet_ref
Added text to odp_packet_free() to describe handling of packet references
Removed the restriction that the ODP_PACKET_FREE_CTRL_DONT_FREE may not be set in packets with multiple references.
Added TM capability bits that tell whether different types of packet references may be enqueued to TM. Since TM capabilities are separate for different egresses or TM systems, the capability to handle packet references can depend on the underlying pktio. The don't_free packet option continues to not have an effect in odp_tm_enqueue().
fixed a couple of typos

include/odp/api/spec/ipsec.h

include/odp/api/spec/packet.h

include/odp/api/spec/packet_io.h

include/odp/api/spec/packet_io_types.h

include/odp/api/spec/packet_io.h

include/odp/api/spec/packet.h

JannePeltonen · 2025-12-23T12:43:40Z

v5: Updated based on review comments. Text clarifications only, no semantic changes.

include/odp/api/spec/packet.h

JannePeltonen · 2025-12-31T11:32:17Z

v6: cosmetic changes based on review comments, rebased

include/odp/api/spec/ipsec.h

ashwinyes · 2026-01-28T05:30:04Z

Acked-by: Ashwin Sekhar T K asekhar@marvell.com

include/odp/api/spec/packet.h

psavol · 2026-03-19T13:55:31Z

include/odp/api/spec/packet_io_types.h

+	 /** Referencing packets can be consumed */
+	uint8_t referencing :1;
+
+} odp_pktout_packet_ref_capability_t;


Could we use the same type for both pktout and TM capas? In both cases the flag tells if packet output can consume the packet, and that common part of the documentation could remain here (the first paragraph). The differing part could be documented in the capa documentation using the type (second paragraph about dont_free moved into pktio capa).

Type name could be shortened to odp_pktout_ref_capability_t.

JannePeltonen · 2026-03-20T13:40:25Z

v8:

Changed odp_packet_is_referencing() to return 1 only for referencing packets. Now it returns 0 for static references. This clarifies the API and makes things easier in the implementation since referencing packets and static references are quite different from each other. Now it will not be easy to distinguish between a static reference and referenced packet, but a future API revision can introduce a new function for that purpose if it turns out necessary.
Changed pktio and TM capabilities to use a common odp_packet_ref_types_t struct when describing what types of packet are supported.
removed some unnecessary and redundant spec text as suggested by Petri
fixed a couple of typos and grammar issues

include/odp/api/spec/packet_io_types.h

include/odp/api/spec/traffic_mngr.h

JannePeltonen · 2026-03-23T14:50:28Z

v9:

removed stale references to a capability typedef removed in v8
added short descriptions for Doxygen in a couple of places

psavol

Reviewed API commits.

Make packet reference tests forward compatible with the following changes that are coming in the API version by removing or changing the conflicting code. - odp_packet_has_ref() returns 0 for packets created using odp_packet_ref() and odp_packet_ref_pkt(). - Referenced packets may not be modified - Creating references of references is not allowed Also fix a couple of asserts that checked the wrong packet handles. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

Specification text of odp_packet_head() says that odp_packet_data() and odp_packet_l2_ptr() can be used to return packet data start address. The latter is incorrect as there is packet data before the l2_ptr if l2 offset is nonzero. Remove the incorrect reference to odp_packet_l2_ptr(). Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>

Current API regarding dynamic packet references is not easy to efficiently support in some ODP implementations with HW accelerated packet output due to the need to update reference counts after TX completion and based on that conditionally free packet buffers. odp_packet_ref() has been implemented only as a full packet copy in all ODP implementations. Redefine the semantics of odp_packet_ref() and the packets involved in packet data sharing in a way that hopefully makes it simpler to support packet data sharing with restrictions in packet output and with per-packet reference counting instead of per-segment reference counting. The new semantics also enable use cases that were not previously possible. Specify better the rules regarding packet data layout manipulation operations, allowing private packet data also in the tail of the packets that reference other packets. Pull and truncate operations may involve shared packet data, in which case they just remove that data from the packet, not affecting the packet owning the data. All push and extend operations always produce non-shared packet data. Change odp_packet_has_ref() to indicate whether a packet is referenced by another packet, not whether the packet shares data with another packet. In particular, odp_packet_ref() returns a packet that shares data with the given packet but is not itself referenced by other packets. Specify that packet data and layout of packets referenced by other packets cannot be modified at all. Add new odp_packet_is_referencing() to test whether a packet references another packet but is not a static reference. Introduce pktio capability bits to indicate whether a pktio supports sending packets with shared data without using the dont_free flag. Sending with the dont_free flag is supported with all types of packets. Introduce TM capability bits that indicate if referencing and referenced packets and static packet references can be enqueued to TM. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>

Maximum CoS index is 63 but the field for it in packet header is of type uint16_t. Change the field to uint8_t to make room for other things without a need to change the memory layout of the header. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

Introduce the concept of indirect segments whose data pointers point to data areas of other segments. Use the existing per-segment reference counts to keep track of the usage of shared segments. Implement dynamic packet references, or referencing packets, using indirect segments. A referencing packet has a chain of segments so that some of the segments may be indirect. Add special handling for indirect segments in packet manipulation functions wherever necessary. This commit tries to keep the implementation simple and leaves optimization opportunities for the future. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

Add support for packet references to all pktios. Many pktios copy packets before output processing already. Check for referencing packets and make a copy if needed in loop, ipc and xdp pktio and in the dpdk pktio in the zero copy case. Set the relevant pktio capabilities in the common packet i/o code. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

If a referencing packet is passed to crypto, copy it before processing to avoid writing to the read-only parts of the packet. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

If a referencing packet is passed to TM, copy it before processing to avoid writing to the read-only parts of the packet. Unconditional copying at TM enqueue time could be avoided by making the copy only when needed in various marking functions but this commit keeps things simple and does the copy always. Copying for referenced packets and static packet references was already in place. Add capability bits to indicate support for packet references. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

If a referencing packet is passed to ipsec, copy it before processing to avoid writing to the read-only parts of the packet. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

Add testing with referencing and referenced packets in addition to normal packets and static references. Add checking of the related new pktio capabilities. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

Add testing with referencing and referenced packets in addition to normal packets and static references. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

Add testing of odp_packet_is_referencing(). Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

Check that odp_pktio_capability_t::packet_ref.static_ref capability is present in the used pktios when static reference TX mode is requested. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

Print the odp_pktout_packet_ref_capability_t flags for pktios. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

JannePeltonen · 2026-03-27T21:51:53Z

v10:

added implementation
added packet refs to pktio and crypto validation tests
added capa printing to sysinfo and other small changes

I will add new packet API validation tests later as those need still some work.

odpbuild changed the title ~~api: packet: redefine packet data sharing~~ [PATCH v1] api: packet: redefine packet data sharing Nov 11, 2025

MatiasElo added this to the v1.49.0 milestone Nov 19, 2025

ashwinyes reviewed Nov 20, 2025

View reviewed changes

JannePeltonen force-pushed the api-pkt-ref branch from fc5ce20 to 3a94a2c Compare December 2, 2025 14:02

odpbuild changed the title ~~[PATCH v1] api: packet: redefine packet data sharing~~ [PATCH v2] api: packet: redefine packet data sharing Dec 2, 2025

JannePeltonen force-pushed the api-pkt-ref branch from 3a94a2c to 3a39a0c Compare December 9, 2025 11:09

odpbuild changed the title ~~[PATCH v2] api: packet: redefine packet data sharing~~ [PATCH v3] api: packet: redefine packet data sharing Dec 9, 2025

JannePeltonen force-pushed the api-pkt-ref branch from 3a39a0c to a728f44 Compare December 10, 2025 16:59

odpbuild changed the title ~~[PATCH v3] api: packet: redefine packet data sharing~~ [PATCH v4] api: packet: redefine packet data sharing Dec 10, 2025

JannePeltonen marked this pull request as ready for review December 12, 2025 12:05

TuomasTaipale reviewed Dec 15, 2025

View reviewed changes

psavol reviewed Dec 22, 2025

View reviewed changes

include/odp/api/spec/packet.h Outdated Show resolved Hide resolved

JannePeltonen force-pushed the api-pkt-ref branch from a728f44 to 1fec78c Compare December 23, 2025 12:42

odpbuild changed the title ~~[PATCH v4] api: packet: redefine packet data sharing~~ [PATCH v5] api: packet: redefine packet data sharing Dec 23, 2025

psavol reviewed Dec 23, 2025

View reviewed changes

include/odp/api/spec/packet.h Outdated Show resolved Hide resolved

psavol reviewed Dec 23, 2025

View reviewed changes

include/odp/api/spec/packet.h Outdated Show resolved Hide resolved

MatiasElo removed this from the v1.49.0 milestone Dec 29, 2025

JannePeltonen force-pushed the api-pkt-ref branch from 1fec78c to 25f5e64 Compare December 31, 2025 11:33

odpbuild changed the title ~~[PATCH v5] api: packet: redefine packet data sharing~~ [PATCH v6] api: packet: redefine packet data sharing Dec 31, 2025

JannePeltonen changed the title ~~[PATCH v6] api: packet: redefine packet data sharing~~ [PATCH v6] api: packet: redefine packet data sharing with packet references Dec 31, 2025

TuomasTaipale reviewed Jan 12, 2026

View reviewed changes

include/odp/api/spec/ipsec.h Outdated Show resolved Hide resolved

MatiasElo added this to the v1.50.0 milestone Mar 12, 2026

JannePeltonen force-pushed the api-pkt-ref branch from 25f5e64 to 3c3e8f7 Compare March 17, 2026 10:05

psavol reviewed Mar 19, 2026

View reviewed changes

include/odp/api/spec/packet.h Outdated Show resolved Hide resolved

psavol reviewed Mar 19, 2026

View reviewed changes

include/odp/api/spec/packet.h Outdated Show resolved Hide resolved

psavol reviewed Mar 19, 2026

View reviewed changes

include/odp/api/spec/packet.h Outdated Show resolved Hide resolved

psavol reviewed Mar 19, 2026

View reviewed changes

JannePeltonen force-pushed the api-pkt-ref branch from 3c3e8f7 to 8b9e863 Compare March 20, 2026 13:26

odpbuild changed the title ~~[PATCH v7] api: packet: redefine packet data sharing with packet references~~ [PATCH v8] api: packet: redefine packet data sharing with packet references Mar 20, 2026

psavol reviewed Mar 20, 2026

View reviewed changes

include/odp/api/spec/packet_io_types.h Show resolved Hide resolved

psavol reviewed Mar 20, 2026

View reviewed changes

include/odp/api/spec/traffic_mngr.h Show resolved Hide resolved

JannePeltonen force-pushed the api-pkt-ref branch from 8b9e863 to a5f7b0f Compare March 23, 2026 14:48

odpbuild changed the title ~~[PATCH v8] api: packet: redefine packet data sharing with packet references~~ [PATCH v9] api: packet: redefine packet data sharing with packet references Mar 23, 2026

psavol approved these changes Mar 23, 2026

View reviewed changes

JannePeltonen added 14 commits March 27, 2026 23:05

linux-gen: crypto: add support for referencing packets

e7b8f94

If a referencing packet is passed to crypto, copy it before processing to avoid writing to the read-only parts of the packet. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

linux-gen: ipsec: add support for referencing packets

8328763

If a referencing packet is passed to ipsec, copy it before processing to avoid writing to the read-only parts of the packet. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

validation: pktio: add testing with referencing and referenced packets

6a8d3e9

Add testing with referencing and referenced packets in addition to normal packets and static references. Add checking of the related new pktio capabilities. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

validation: crypto: add testing with referencing and referenced packets

09ae42a

Add testing with referencing and referenced packets in addition to normal packets and static references. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

test: bench_packet: add odp_packet_is_referencing()

8fdf5af

Add testing of odp_packet_is_referencing(). Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

test: packet_gen: check pktio capa when using static reference TX mode

56aa6c9

Check that odp_pktio_capability_t::packet_ref.static_ref capability is present in the used pktios when static reference TX mode is requested. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

example: sysinfo: print the new packet ref related pktio capa flags

df92333

Print the odp_pktout_packet_ref_capability_t flags for pktios. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>

JannePeltonen force-pushed the api-pkt-ref branch from a5f7b0f to df92333 Compare March 27, 2026 21:47

odpbuild changed the title ~~[PATCH v9] api: packet: redefine packet data sharing with packet references~~ [PATCH v10] api: packet: redefine packet data sharing with packet references Mar 27, 2026

Conversation

JannePeltonen commented Nov 11, 2025

Uh oh!

ashwinyes Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

JannePeltonen Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

ashwinyes Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

JannePeltonen Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

JannePeltonen commented Dec 2, 2025

Uh oh!

JannePeltonen commented Dec 9, 2025

Uh oh!

JannePeltonen commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JannePeltonen commented Dec 23, 2025

Uh oh!

Uh oh!

Uh oh!

JannePeltonen commented Dec 31, 2025

Uh oh!

Uh oh!

ashwinyes commented Jan 28, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

psavol Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JannePeltonen Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

JannePeltonen commented Mar 20, 2026

Uh oh!

Uh oh!

Uh oh!

JannePeltonen commented Mar 23, 2026

Uh oh!

psavol left a comment

Choose a reason for hiding this comment

Uh oh!

JannePeltonen commented Mar 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

JannePeltonen commented Dec 10, 2025 •

edited

Loading

psavol Mar 19, 2026 •

edited

Loading