| Internet-Draft | IOAM On-Path Delay | October 2026 |
| Huang Feng, et al. | Expires 19 April 2027 | [Page] |
This document defines extensions to In Situ Operations, Administration, and Maintenance (IOAM) that enable IOAM nodes to compute on-path delay samples. These samples are the basis for the on-path delay statistics that can be exported using the IP Flow Information Export (IPFIX) Information Elements defined in RFC 9951.¶
The document defines an On-Path Delay IOAM data field and extends the IOAM Direct Export (DEX) Option-Type with a timestamp reference inserted by the IOAM encapsulating node. These extensions enable transit and decapsulating nodes to compute and locally aggregate on-path delay. The resulting on-path delay can then be made available by different mechanisms, for example IPFIX export or the YANG data model for on-path telemetry.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 19 April 2027.¶
Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Network operators use delay measurements to detect and localize performance degradation, including congestion, affecting customer traffic. RFC 9951 defines IP Flow Information Export (IPFIX) [RFC7011] Information Elements for exporting statistical views of the on-path delay measured between an IOAM encapsulating node and subsequent transit and decapsulating nodes.¶
To produce these statistics, the on-path delay needs to be computed at the node performing the aggregation. In postcard mode, this allows delay samples to be aggregated directly in the context of the customer flow that triggered their collection, without requiring subsequent correlation between separately exported telemetry data and customer traffic.¶
IOAM Direct Export (DEX) [RFC9326] provides such a mechanism for postcard mode. A packet carrying the DEX Option-Type triggers an IOAM node to collect, locally aggregate, and/or export the IOAM data selected by the IOAM Trace-Type. However, the DEX Option-Type currently does not provide the timestamp reference required by an IOAM transit or decapsulating node to compute the on-path delay defined in RFC 9951.¶
This document defines two IOAM extensions to enable this computation:¶
A new On-Path Delay IOAM data field, which identifies the on-path delay as an IOAM telemetry metric and allows it to be selected by the IOAM Trace-Type.¶
An extension to the IOAM DEX Option-Type carrying a timestamp reference inserted by the IOAM encapsulating node.¶
When processing a packet carrying these extensions, an IOAM transit or decapsulating node computes the on-path delay as the difference between its current timestamp and the timestamp reference carried in the packet. The resulting delay sample can be locally aggregated in the context of the customer flow and subsequently exported, for example using the IPFIX Information Elements defined in RFC 9951.¶
This document defines the IOAM mechanisms required to compute and identify the on-path delay metric. The aggregation and export mechanisms are not redefined by this document.¶
This document uses the following terms as defined in [RFC9197] and [RFC9326].¶
IOAM domain¶
IOAM encapsulating node¶
IOAM transit node¶
IOAM decapsulating node¶
IOAM Direct Export (DEX) Option-Type¶
This document also uses the terms postcard mode and passport mode as described in the Introduction of [RFC9951]. In postcard mode, IOAM transit nodes also export on-path telemetry data. In passport mode, only the IOAM decapsulating node of the IOAM domain reports metrics.¶
The IOAM Direct Export (DEX) Option-Type defined in [RFC9326] provides a mechanism for postcard mode in which packets trigger IOAM nodes to collect, locally aggregate, and/or export selected telemetry data. The metrics to be processed by a DEX node are identified using the IOAM-Trace-Type field carried in the DEX Option-Type.¶
Computing the on-path delay requires two pieces of information that are not currently available together in IOAM DEX. First, the on-path delay needs an allocation in the IOAM Trace-Type registry so that DEX can identify it as a metric to be aggregated or exported. Second, the DEX Option-Type needs to carry the timestamp of the IOAM encapsulating node, which provides the reference from which a transit or decapsulating node computes the delay.¶
The IOAM encapsulating node inserts the timestamp reference in the DEX Option-Type. When a transit or decapsulating node processes the packet, it subtracts this timestamp from its current timestamp to obtain the on-path delay.¶
IOAM Domain
.........................................
. .
. D1 .
. <------> .
. .
. D2 .
. <--------------------> .
. .
. D3 .
. <-----------------------------------> .
. .
(H1) ------ (R1) ------- (R2) ------- (R3) -------- (R4) ------ (H2)
IOAM IOAM IOAM IOAM
Host 1 Encapsulating Transit Transit Decapsulating Host 2
Node Node 1 Node 2 Node
. .
. .
.........................................
D1, D2, and D3 represent the on-path delay from the IOAM encapsulating node R1 to R2, R3, and R4, respectively.¶
Together, the On-Path Delay data field and the DEX timestamp extension allow an in-flight customer packet to trigger computation and local aggregation of the on-path delay at each participating node. For the per-flow aggregation use case described in this document, the IPFIX mechanism defined in [RFC9951] is the preferred export method because the delay samples can be aggregated in the context of the customer flow that triggered their collection, avoiding subsequent correlation of telemetry records with customer traffic.¶
The extensions defined here are not tied to IPFIX export. Other mechanisms can make the resulting on-path telemetry available, including the YANG data model and YANG-Push mechanisms described in [I-D.ietf-ippm-on-path-telemetry-yang]. Export mechanisms are outside the scope of this document and are discussed further in Section 6.¶
This document defines a new IOAM Trace-Type data field for the on-path delay. The field represents the delay between the IOAM encapsulating node and the transit or decapsulating node where the value is computed.¶
The allocation in the IOAM Trace-Type registry is required for IOAM DEX because the DEX Option-Type defined in [RFC9326] uses the IOAM-Trace-Type field to identify which telemetry data is to be collected, locally aggregated, and/or exported. Allocating an On-Path Delay data field therefore allows a DEX node to select the computed delay for processing in the same way as other IOAM telemetry metrics.¶
The same IOAM Trace-Type allocation can also be used by the Pre-allocated and Incremental Trace Option-Types defined in [RFC9197]. Consequently, the allocation enables the on-path delay metric for both postcard mode and passport mode, although the DEX use case motivates the allocation in this document.¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |O| On-Path Delay | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The On-Path Delay data field is a 4-octet unsigned integer in the range 0 to 2^31-1. It is the on-path delay between the IOAM encapsulating node and the transit or decapsulating node, measured in microseconds. Note that this unit differs from the nanoseconds used by the transit delay field of [RFC9197].¶
The value is computed as specified in Section 5. If the delay exceeds 2^31-1 microseconds (about 35 minutes), the top bit 'O' is set to indicate overflow and the value is set to 0x80000000. If the node cannot compute the delay, for example because the difference is negative as a result of clocks of the IOAM encapsulating node and of the computing node that are not sufficiently synchronized, the field MUST be set to 0xFFFFFFFF to mean not populated. A delay sample with either of these two values MUST NOT be included in local aggregation.¶
The data field is allocated by IANA as defined in Section 9.¶
In postcard mode, a packet carrying the DEX Option-Type can trigger telemetry data to be exported directly or locally aggregated before export, as defined in [RFC9326]. The telemetry metrics subject to this processing are selected using the IOAM-Trace-Type field.¶
The On-Path Delay data field defined in this document identifies the delay as one of these metrics. However, computing its value additionally requires the timestamp of the IOAM encapsulating node. This document therefore extends the IOAM DEX Option-Type with a timestamp reference. Two optional fields are defined: Timestamp Seconds and Timestamp Fraction. The fields are enabled using the IOAM DEX Extension-Flags.¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Namespace-ID | Flags |Extension-Flags| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IOAM-Trace-Type | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Flow ID (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Timestamp Seconds (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Timestamp Fraction (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
As in [RFC9326], the bits of the IOAM DEX Extension-Flags are numbered from 0, with bit 0 being the most significant bit of the field. Bit [TBD1] of the Extension-Flags is used for Timestamp Seconds. Bit [TBD2] is used for Timestamp Fraction. The Timestamp Seconds and Timestamp Fraction fields MUST be used in combination.¶
The Timestamp Seconds and Timestamp Fraction fields use the encoding formats defined in Section 5 of [RFC9197].¶
The timestamp reference is inserted at the IOAM encapsulating node. At each subsequent IOAM transit or decapsulating node, the computing node obtains the on-path delay, represented by the On-Path Delay data field, by subtracting the timestamp reference from its current timestamp, using the seconds and fraction fields of the format in use at the full resolution of that format. The sign of the difference is evaluated at this resolution, before any conversion. The difference is then converted to microseconds and rounded to the nearest microsecond. For the PTP truncated format, the fraction is in nanoseconds. For the NTP 64-bit format, the fraction is in units of 2^(-32) seconds. For the POSIX format, the fraction is already in microseconds.¶
A packet that carries only one of the Timestamp Seconds and Timestamp Fraction fields does not conform to this specification. A node receiving such a packet MUST NOT use the timestamp reference to compute the On-Path Delay data field value, and MUST set the field to 0xFFFFFFFF as described in Section 4.¶
Each of these fields is an optional 4-octet field as defined in [RFC9326], so the length of the DEX Option-Type increases by 4 octets for each field present. The Extension-Flags explicitly indicate this length, as specified in [RFC9326]. For example, when the DEX Option-Type is carried in IPv6 as specified in [RFC9486], the Opt Data Len field MUST reflect the resulting length, the 4-octet alignment is preserved because each field is 4 octets long, and the total length of the IOAM Option-Types remains subject to the 255-octet limit defined in that Other IOAM encapsulations, as presented in Section 5 of [RFC9378] have their own constraints, which this document does not change. A node that does not support these fields ignores them as specified in [RFC9326].¶
The computed value is meaningful only if the clocks of the IOAM encapsulating node and of the computing node are synchronized, and the accuracy of the result is bounded by the accuracy of that synchronization. Nodes participating in this measurement MUST use the same timestamp format, as determined by the management plane (see [RFC9197]). Deployment guidance on clock synchronization is provided in Section 8.¶
This document does not define an export mechanism. It defines the IOAM extensions required for an IOAM node to compute the on-path delay and to identify that value as a telemetry metric that can be locally aggregated and/or exported.¶
[RFC9951] defines IPFIX Information Elements and mechanisms for exporting on-path telemetry, including statistical views of on-path delay. This is particularly suitable when DEX processing aggregates delay samples on a per-flow basis, because the resulting telemetry can be exported in the context of the customer flow that triggered the measurements.¶
Other export mechanisms can also be used. The YANG data model defined in [I-D.ietf-ippm-on-path-telemetry-yang] provides on-path telemetry data that can be published using YANG-Push. In addition, [I-D.spiegel-ippm-ioam-rawexport] describes a mechanism for exporting raw IOAM data. The choice of export mechanism is outside the scope of this document.¶
The security considerations for the IOAM Trace-Type and IOAM DEX Option-Type are defined in [RFC9197] and [RFC9326]. The additional considerations below apply to the extensions defined in this document.¶
The on-path delay is computed from a timestamp reference carried in the packet and from the local clock of the computing node. An attacker that can modify the timestamp reference in a packet, or that can disturb the time synchronization of the IOAM nodes, can corrupt the delay samples. The consequences include delay values that are too small or too large, samples rejected as not populated, and locally aggregated statistics that hide or exaggerate a performance problem. Operators SHOULD protect the time source and the synchronization protocol used by the IOAM nodes, as discussed in [RFC9378], and use the extensions only within a limited IOAM domain, with packets filtered at the domain boundary.¶
The Timestamp Seconds and Timestamp Fraction fields add 8 octets to the DEX Option-Type for each packet that carries them. This increases the per-packet overhead and the MTU and size considerations of the encapsulation that carries the option, for example [RFC9486] for IPv6, and the rate-limiting guidance of [RFC9326] continues to apply.¶
The on-path delay is computed from the timestamps of two different nodes, so the accuracy of the result depends on how closely the clocks of the IOAM encapsulating node and of the computing node are synchronized. Nodes participating in this measurement SHOULD synchronize their clocks using a protocol such as PTP or NTP. Operators need to ensure that the synchronization accuracy is small compared to the delays being measured.¶
When synchronization degrades, the difference between the timestamps can become negative, and the node then reports the On-Path Delay data field as not populated, as described in Section 4. A higher rate of such values can indicate a synchronization problem, so operators are advised to monitor the clock status of the IOAM nodes that take part in the measurement. The protection of the time source is discussed in Section 7.¶
This document requests IANA to allocate the following bit in the "IOAM Trace-Type" registry.¶
Bit: [TBD3]
Description: On-path delay between the IOAM encapsulating node
and the transit or decapsulating node measured in
microseconds.
Reference: [This-RFC-to-be]
¶
This document also requests IANA to allocate the following two bits in the "IOAM DEX Extension-Flags" registry.¶
Bit: [TBD1] Description: Timestamp Seconds Reference: [This-RFC-to-be]¶
Bit: [TBD2] Description: Timestamp Fraction Reference: [This-RFC-to-be]¶
The authors would like to thank Greg Mirsky for the review and valuable comments. The authors also thank Haoyu Song for the review and valuable comments.¶