IPFIX Export of RoCEv2 Sequence Observations

Introduction Two metering implementations can see the same PSNs yet report different gap counts if they use different history, initialization, or closure rules. A common export profile is useful when a collector consumes such observations from multiple implementations and must distinguish unresolved history from observed packets. The interoperability target is the interpretation of exported measurements, not an implementation-specific packet-processing architecture. defines raw RoCEv2 BTH Information Elements. Those fields remain useful for packet traces and validation. This document reuses its destination QP and partition identifiers and adds an explicitly specified summary. A raw-field export need not export every packet; however, a sampled packet trace cannot reconstruct the exact state of this profile. No throughput or export-volume improvement is claimed without measurement. A shared collector may correlate queue and sequence observations, but such correlation alone does not establish which link lost a packet or whether an ECN policy is effective. A sequence position may be observed after reordering, a retry, duplication, or a capture fault. These causes cannot in general be distinguished by the counters defined here.

Requirements Language The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Applicability and Provisioning The profile covers only a provisioned, unidirectional RoCEv2 Reliable Connection request stream using BTH OpCodes 0x00 through 0x0B: SEND and RDMA WRITE, including the immediate-data variants in that range. Originally transmitted requests advance by one PSN modulo 2^24. Other RC operations, UC, UD, XRC, and inferred READ spans are outside this profile. Provisioning MUST establish the operation restriction for the whole connection, rather than merely filter the listed opcodes. RDMA READ can reserve sequence space for response packets; provides a public implementation example. Filtering READ from a mixed stream does not make its remaining request PSNs consecutive. An observed unsupported request or malformed request attributable to the provisioned connection MUST close the epoch with profileViolation and disable metering for that connection until explicitly reprovisioned. The offending packet is not counted. A subsequent SEND or WRITE MUST NOT automatically re-enable the profile. Response packets and CNPs do not change this request state. The observation point MUST process all request packets in the provisioned scope, without sampling, selective omission, or known capture loss, and MUST cover all paths carrying that stream. A single ECMP branch without complete coverage is not sufficient. Deployment provisioning MUST establish namespace separation and path coverage. If capture completeness cannot be established, this profile MUST NOT be enabled. A capture gap closes affected epochs with captureDiscontinuity; metering resumes only after complete observation has been re-established and starts fresh state. An unclassifiable capture fault affecting several contexts closes all contexts that may be affected. Serial interpretation also requires that a packet cannot legitimately be separated from the current high-water position by half or more of the PSN serial space, including very old retries. This is an applicability condition, not something a 24-bit field alone can verify. If endpoint behavior or retention bounds cannot establish it, this profile MUST NOT be used for that connection. Known violations close the epoch. An exactly half-space distance is always treated as ambiguous.

Identity and Epoch Lifecycle The record key comprises the IPFIX Transport Session, Observation Domain ID, observationPointId, source and destination IP addresses of the RoCEv2 packet, rdmaDestinationQP, rdmaPartitionKey, any additional tenant or tunnel namespace required to avoid collisions, and rdmaSequenceEpochId. A physical interface identifier alone is not a tenant namespace. Address family and encapsulation layer MUST be fixed and unambiguous in each Template. UDP source port MUST NOT split a connection context. The Exporting Process MUST allocate a nonzero, monotonically increasing epoch ID across all contexts in each Transport Session and Observation Domain when the first report of that epoch is serialized. Thus first reports are serialized in increasing epoch-ID order; the ID does not order the first packet times. An ID MUST NOT be reused in that scope. IDs need not be consecutive at a collector. A new Transport Session starts fresh metering state. An epoch is closed on a known connection-generation change, idle expiration, policy or window change, capture discontinuity, profile violation, resource eviction, serial discontinuity, or impending counter/report-ID exhaustion. Unless connection generation is independently established by trusted provisioning or endpoint integration, every report MUST set generationUnverified. A local epoch is not proof of a connection lifetime. Neither an idle timer nor an apparently new initial PSN proves that destination QP reuse was detected. Known connection restarts MUST NOT be joined even if addresses, QP, and PSN repeat. A serial discontinuity caused by a valid eligible request closes the old epoch before counting that request, then starts a new epoch with that request. The request is counted exactly once. Other closure causes require the applicable validity conditions to be re-established before restart. A final snapshot abandons all pending positions. If it cannot be exported because of failure or resource exhaustion, the collector sees an incomplete epoch; a final record is never fabricated by the collector.

Bounded Sequence Observation Algorithm Let M=2^24 and 1<=W<=2^22. The exporter maintains a local extended integer H, a fixed initial extended position S, and a retained interval [L,H], where L=max(S,H-W+1). Retained positions are either SEEN or PENDING. Positions before S or before L are unclassified, including positions numerically within W of H but preceding the first observed request. No initialized prefix is presumed seen or pending. Local extended integers are not exported PSN values. For an arriving PSN p, compute u=(p-(H mod M)) mod M. If u=M/2, comparison is ambiguous. Otherwise d=u for u<M/2, and d=u-M for u>M/2. This uses the serial comparison boundary in . Implementations MUST use checked arithmetic for counters and local extended positions; they MUST close and restart before overflow rather than wrap a cumulative counter. Per-Request Transitions

Condition	Required action
No active epoch	Set S=H=p; retain only p as SEEN; requests=1; all other counters=0.
u=M/2 or d>W	Close with serialDiscontinuity; start fresh state with p; do not account p to the old epoch.
1<=d<=W	Count request; add H+1 through H+d-1 as PENDING and add d-1 to skipped; advance H by d; mark H SEEN; expire each pending position below new L once.
d<=0 and H+d in [L,H] is PENDING	Count request; change that position to SEEN; increment filled once.
d<=0 and H+d in [L,H] is SEEN	Count request; increment repeated.
d<=0 and H+d<L	Count request; increment unclassifiedLate; do not change H or retained state.

At most W position labels are retained. Ordinary reports do not reset counters or pending state. On closure, add the number of pending positions to abandoned and clear pending. A position can be filled or expired or abandoned, never more than one of these. Expiry is caused by high-water advance, not by an elapsed-time estimate of network loss. skipped = filled + expired + abandoned + pending 0 <= pending <= W - 1 repeated + unclassifiedLate + filled <= requests - 1 These relationships MUST hold at every snapshot. Abandoned is zero in an active epoch and pending is zero in a final epoch. Their validity establishes internal accounting only. A malicious exporter can satisfy them with fabricated measurements.

Export Profile and Template This profile uses IPFIX over TCP to order reports from an Exporting Process. A profile implementation MUST support this binding. Selecting TCP for this profile does not remove the general implementation requirements of RFC 7011, including its SCTP requirement, and does not update that RFC. Reconnection creates a new session and new epochs. TCP reliability does not guarantee delivery of a final report after process or connection failure. Each Data Record MUST contain every field listed in , plus any required namespace keys. It MUST be an atomic snapshot of a single epoch. All counters are cumulative from the first accounted request. The report ID starts at one and increments once per serialized snapshot; retransmission of an identical snapshot retains its ID. The ID and each total counter MUST NOT wrap. flowStartMilliseconds and flowEndMilliseconds are the first and last accounted packet arrival times; they do not measure the export delay or the final closure time. A Template MUST use exactly one address family. A field is not conditionally omitted within a record. All named fields occur once in this profile. The exporter MUST advertise the Flow Keys using the Flow Keys Options Template in RFC 7011, including record-level namespace and epoch identifiers but excluding report ID. The Transport Session and Observation Domain are outer context. An observationPointId change requires a new epoch. The unassigned identifiers in this draft and its dependency are not usable wire numbers. Profile Fields in Example Order

Information Element	Octets
observationPointId	8
sourceIPv4Address / sourceIPv6Address	4 / 16
destinationIPv4Address / destinationIPv6Address	4 / 16
rdmaDestinationQP	4
rdmaPartitionKey	2
flowStartMilliseconds	8
flowEndMilliseconds	8
rdmaSequenceEpochId	8
rdmaSequenceReportId	8
rdmaSequenceWindow	4
rdmaSequenceFlags	2
skippedPsnTotalCount	8
filledPsnTotalCount	8
expiredPsnTotalCount	8
abandonedPsnTotalCount	8
pendingPsnCount	4
repeatedPsnTotalCount	8
requestPacketTotalCount	8
unclassifiedLatePacketTotalCount	8

With assigned standard IEs, no extra namespace fields, and full-length encodings as shown, there are 19 fields: 120 octets per IPv4 Data Record or 144 per IPv6 Data Record. The Template Record is 80 octets (4+19*4); its containing Template Set is 84 octets before any padding. These are distinct from the IPFIX Message length. An enterprise-specific field adds a four-octet PEN to its Field Specifier; therefore these Template lengths do not apply unchanged to experimental PEN mappings. No Template ID is reserved by this profile. Other legal integer lengths require explicit agreement and MUST retain the same semantics. The destination QP uses the unsigned32 definition from , with its upper eight bits zero. rdmaPartitionKey is unsigned16. Both remain dependent on the referenced individual proposal; its progression or allocation is not assumed. Prior to publication, the definitions and assignments must be resolved with that work or replaced through explicit review, rather than silently inventing competing fields.

Collector Processing The collector MUST validate Template bindings, field ranges, required keys, and the arithmetic relationships in . It compares cumulative counters only for an identical full epoch key. Window, connection-generation qualification, flow start time, and namespace are immutable within the epoch. Closure-reason flags appear only in final records. Each final record has pending zero; each non-final record has abandoned zero. Unknown flag bits make the record an unsupported qualification; the collector MUST preserve the raw record and MUST NOT derive unqualified measurements from it. For a retained epoch, a larger report ID is processed only if total counters do not decrease and immutable fields agree. A report gap, including a first received ID greater than one, marks missing report coverage. It does not invalidate an otherwise self-consistent cumulative total, but differences across the gap MUST NOT be assigned to the missing subintervals. An identical repeat of the latest report ID is ignored; different decoded content for that ID invalidates the epoch. A smaller report ID is stale and MUST NOT modify already published values. An implementation MAY keep older raw records for offline validation. After an accepted final record, no larger report ID can extend that epoch. A conflicting or extending record received while closure state is retained invalidates it. Collectors MUST set finite resource and retention limits and expose those limits to their operators. On retirement, they MUST preserve an outcome of complete, incomplete, or invalid in the stored result. Offline recomputation from an explicit archive is a separate operation. For bounded retirement, the collector retains an epoch-admission high-water ID per Transport Session and Observation Domain. It first looks up retained context by epoch ID, rejecting a changed full key for that ID. For an unknown epoch, an ID at or below the high-water ID is an orphan and MUST NOT create or reopen context. An unknown larger ID advances the high-water ID, including when admission is refused because of resource limits. It can create a new context with missing initial reports explicitly qualified. This rule relies on the specified ordering of first reports. Retirement therefore needs no unbounded tombstone set. The high-water ID is cleared only with a new Transport Session. A collector MUST NOT silently forget it while continuing that session. Silence alone is not a final report. A session failure or configured inactivity timeout without finalEpoch leaves the epoch incomplete. Complete report coverage requires reports from ID one through a valid final record without report gaps; metering qualifications still apply and this does not imply coverage of an entire connection lifetime. A missing final does not erase useful observed counters, but outstanding positions remain unresolved. Collectors MUST distinguish metering qualifications, report gaps, and collector resource drops from network loss. Expired positions MUST NOT be labeled lost packets. Repeated PSNs MUST NOT be labeled confirmed retransmissions. Filled positions MUST NOT be labeled proven reorderings. Aggregate counters from different observation points MUST NOT be subtracted to claim a loss location: the records lack common event identities, and windows, paths, and epochs may differ.

Information Element Definitions The following definitions follow and . Each requested IE has status current and reference RFC XXXX. TBD labels denote unassigned Element IDs. The units none are intentional for sequence-position quantities; packets is used only for counts of observed packet arrivals.

rdmaSequenceEpochId Nonzero epoch identifier allocated monotonically across contexts in a Transport Session and Observation Domain. It is never reused within that scope. Registration Properties

Property	Value
Element ID	TBD1
Abstract Data Type	unsigned64
Data Type Semantics	identifier
Units	none
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

rdmaSequenceReportId Snapshot identifier starting at one and increasing per serialized report within an epoch. Retransmission of an identical snapshot retains the identifier. Registration Properties

Property	Value
Element ID	TBD2
Abstract Data Type	unsigned64
Data Type Semantics	identifier
Units	none
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

rdmaSequenceWindow Configured history width W in PSN positions, 1 through 4194304; constant throughout an epoch. A position is a serial-space value, not a claim about a distinct network packet. Registration Properties

Property	Value
Element ID	TBD3
Abstract Data Type	unsigned32
Data Type Semantics	quantity
Units	none
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

rdmaSequenceFlags Numeric qualification masks defined in the flags table; serialized in network byte order. Registration Properties

Property	Value
Element ID	TBD4
Abstract Data Type	unsigned16
Data Type Semantics	flags
Units	none
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

skippedPsnTotalCount Number of sequence positions introduced as PENDING by bounded forward advances since epoch creation. Registration Properties

Property	Value
Element ID	TBD5
Abstract Data Type	unsigned64
Data Type Semantics	totalCounter
Units	none
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

filledPsnTotalCount Number of PENDING positions changed to SEEN while retained. A position contributes at most once. Registration Properties

Property	Value
Element ID	TBD6
Abstract Data Type	unsigned64
Data Type Semantics	totalCounter
Units	none
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

expiredPsnTotalCount Number of PENDING positions retired by high-water advance outside the retained window. Registration Properties

Property	Value
Element ID	TBD7
Abstract Data Type	unsigned64
Data Type Semantics	totalCounter
Units	none
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

abandonedPsnTotalCount Number of PENDING positions retired at epoch closure. Zero before closure. Registration Properties

Property	Value
Element ID	TBD8
Abstract Data Type	unsigned64
Data Type Semantics	totalCounter
Units	none
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

pendingPsnCount Current PENDING positions; at most W-1 and zero on closure. Registration Properties

Property	Value
Element ID	TBD9
Abstract Data Type	unsigned32
Data Type Semantics	quantity
Units	none
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

repeatedPsnTotalCount Number of accounted packets whose PSN maps to an already SEEN retained position. Registration Properties

Property	Value
Element ID	TBD10
Abstract Data Type	unsigned64
Data Type Semantics	totalCounter
Units	packets
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

requestPacketTotalCount All eligible requests accounted to the epoch, including its first request, repeated arrivals, and unclassified late arrivals. Excludes offending profile-violation packets and responses. Registration Properties

Property	Value
Element ID	TBD11
Abstract Data Type	unsigned64
Data Type Semantics	totalCounter
Units	packets
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

unclassifiedLatePacketTotalCount Accounted requests that map before L, including before the first observed position S. Their earlier observation status is not known. Registration Properties

Property	Value
Element ID	TBD12
Abstract Data Type	unsigned64
Data Type Semantics	totalCounter
Units	packets
Range	As specified in the description and profile
Status	current
Reference	RFC XXXX

Qualification Flags The following hexadecimal masks, not bit positions in a drawing, define the wire value. Closure causes can be combined. generationUnverified is set in every report unless generation was independently established. Other defined reason bits MUST be zero in non-final reports; finalEpoch alone permits ordinary closure. Senders MUST set undefined bits to zero. Defined Masks

Mask	Name	Meaning
0x0001	finalEpoch	Epoch is closed.
0x0002	profileViolation	Unsupported or malformed request/profile violation.
0x0004	captureDiscontinuity	Complete observation interrupted.
0x0008	stateExhaustion	Context resource eviction.
0x0010	serialDiscontinuity	Unsupported jump or ambiguous serial interpretation.
0x0020	generationUnverified	Connection generation not independently established.
0xFFC0	Unassigned	Reserved for future IANA assignment.

Verification Examples With W=4 and PSNs 100,102,101,103,102: requests=5, skipped=1, filled=1, repeated=1, pending=expired=abandoned=unclassifiedLate=0. These observations are compatible with multiple network histories. With W=4 and PSNs 100,99,100: position 99 precedes S even though it lies inside the nominal width. The result is requests=3, unclassifiedLate=1, repeated=1, skipped=filled=pending=0. Treating 99 as an initialized missing or previously seen position is incorrect. With W=4 and PSNs 100,102,106: skipped=4, expired=1, pending=3. Closing produces abandoned=3 and pending=0. A later packet cannot fill the closed result. PSNs 16777214,16777215,0,1 advance continuously with no skipped positions. A jump of 2^23 closes as ambiguous. With W=1, any forward advance greater than one starts a fresh epoch; no pending positions exist. An observed READ disables the provisioned profile, including after subsequent SEND packets. A capture fault requires a fresh epoch after capture validity returns. Neither mechanism carries pending state across the interruption.

Security Considerations The IPFIX security considerations in apply. Exporters and collectors for this profile MUST support TLS 1.3 over TCP and MUST use mutually authenticated, confidential export sessions with explicitly authorized peers. Current TLS deployment guidance is in . A transport reconnect changes the epoch scope even when certificates identify the same exporter. Addresses, QP identifiers, partition identifiers, and times can reveal tenant activity. Operators SHOULD restrict observation and retention to the required scope and authorize collector access. Malformed packets and fabricated identities can exhaust state or induce closure storms. Exporters MUST bound concurrent contexts and window size, restrict provisioning to authorized scopes, and expose capture/resource failures. Resource controls can lose reports; they MUST NOT silently convert incomplete observations into complete results. An authenticated exporter may still be faulty or compromised. Arithmetic validation does not establish truth. This profile supplies no instructions to endpoints and MUST NOT be used as authoritative evidence to force retransmission or disable congestion feedback.

IANA Considerations IANA is requested to allocate the twelve IEs in in the IPFIX Information Elements registry . No numeric IDs are requested. Experimental implementations MUST use enterprise-specific IEs under an assigned PEN and an explicitly shared mapping; they MUST NOT use TBD labels or unassigned standard numbers as wire values. IANA is also requested to create an rdmaSequenceFlags registry within IPFIX assignments. Entries are individual unsigned16 masks, description, and reference. Initial entries are masks 0x0001 through 0x0020 in ; individual masks 0x0040 through 0x8000 are unassigned. The registration policy is Specification Required . Review must verify a nonoverlapping mask and define epoch/collector compatibility. A change to the observation algorithm or a new operation profile requires explicit version or IE separation; a flag alone MUST NOT silently redefine existing counters.