Out-of-Path Measurement Methodology for Agent-Initiated Payment Rails

1.1. Relationship to other work

Several contemporaneous Internet-Drafts address agent-initiated payments at the protocol layer. [I-D.ryan-httpauth-payment] defines a "Payment" HTTP authentication scheme over the 402 status code of [RFC9110]. [I-D.laxsharma-pact] defines a contract and escrow layer for autonomous agent commerce, with bonded optimistic settlement and graded verification of delivered work. Neither specifies a measurement methodology, and this document does not compete with either: it measures rails, including rails that implement such protocols, and it defines no protocol of its own.¶

[I-D.contreras-bmwg-ai-agent-benchmarking] defines a benchmarking methodology for AI agents performing network operations tasks. This document addresses a different system under test, and adopts a comparable document structure.¶

[I-D.irtf-pearg-safe-internet-measurement] discusses considerations for performing measurement on the Internet safely; those considerations apply to any deployment of this methodology that observes traffic other than the measuring party's own.¶

6.1. The problem with a single definition of final

Settlement finality is not one quantity across rails. Each rail's ecosystem has a point at which participants conventionally treat a payment as good, and those points differ in kind: a number of block confirmations on one rail, a ledger close on another, a named commitment state on a third, a consensus decision on a fourth, the release of a cryptographic preimage on a fifth.¶

Imposing a single definition across such rails produces one of two failures. Choosing the strictest available definition measures rails against a bar that their own ecosystems do not use, and produces numbers that no participant acts on. Choosing a uniform nominal definition, such as a fixed number of confirmations, silently measures different trust properties on different rails while presenting them as the same quantity.¶

6.2. The requirement

A conforming measurement MUST define finality per rail at the reliance level that rail's ecosystem canonically uses, and MUST record that reliance level as data accompanying every measurement.¶

Each recorded reliance level MUST carry a finality class, which states the trust character of that reliance point. The finality class exists so that a reader can determine whether two rails' finality points are equivalent in kind before comparing their durations.¶

A conforming measurement MUST NOT present durations measured at reliance levels of different finality classes as a single ordered comparison without simultaneously presenting the finality classes. The classes are not a footnote to the comparison; they are a precondition for reading it.¶

A reliance level that is weaker than economic finality MUST be labelled as such in the recorded data. Where a rail's canonical reliance point is a soft commitment that a subsequent reorganisation could in principle undo, the record MUST say so, and MUST NOT describe the measurement as economic finality.¶

7.1. Authorization is not one thing

Rails expose a decision point before settlement at which the rail indicates that a payment may proceed. It is tempting to treat that point as one comparable quantity across rails, and to race rails against each other on the time taken to reach it. That treatment is a category error, and this section specifies the fix.¶

The decision points divide into at least two kinds:¶

Payment-validation:

The payer has already submitted something, and the rail evaluates it. The question answered is "is this payment valid?".¶

Challenge-issuance:

The payer has committed nothing, and the payee issues a payable challenge. The question answered is "what should be paid?".¶

These are different protocol steps in a different order, not two speeds of the same step.¶

That authorization is separable from settlement is long established in card-network practice, where authorization messages are distinct from the later clearing and settlement messages [ISO8583]. That precedent supports the separability this document relies on throughout; it does not itself draw the distinction between validation and issuance, which rests on the order-of-operations argument above and not on any prior standard.¶

7.2. The requirement

A conforming measurement MUST determine, for each rail, which authorization primitives that rail actually exposes, and MUST measure the rail on each primitive it exposes.¶

Measurements of a payment-validation primitive and measurements of a challenge-issuance primitive MUST be reported as separate quantities and MUST NOT be combined into a single comparison, ordering or score.¶

A rail that exposes only one of the two primitives is measured only on that one. Its absence from the other comparison is not a deficiency of the rail and MUST NOT be reported as one.¶

A rail that exposes both is measured on both, and yields two independent quantities that are not commensurable with each other.¶

The two kinds named here are the minimum, not the whole. Where a rail exposes an authorization primitive of some further kind, that kind MUST be measured and reported separately under a name of its own, and MUST NOT be combined with either of the two named here. The requirement is that primitives of different kinds are never raced; the enumeration of kinds is expected to grow.¶

Attaching a label to a mixed comparison is not a conforming alternative to splitting it. A label names a category error without removing it: the underlying model of a single ordered comparison still asserts that the items differ in degree along one axis when they differ in kind.¶

7.3. Where an authorization primitive cannot be separated from settlement

On some rails, paying is settling: the act that would signal acceptance is the same act that achieves finality. On such a rail there is no pre-settlement validation decision to measure, and a conforming measurement MUST NOT synthesise one by relabelling a settlement event as an authorization event. Such a rail is measured on the challenge-issuance primitive it does expose, and on settlement finality (Section 6), and it does not appear in the payment-validation comparison at all.¶

7.4. Terminology honesty

No rail known to the author publishes a metric under the name used in this section. Rails name these operations in their own terms. The framing in this document is therefore an analytical framing imposed over primitives that the rails do publish, and a conforming measurement SHOULD say so plainly rather than implying that it reports a figure the rail itself defines.¶

The separability of authorization from settlement is nonetheless genuine and observable on each rail that exposes both.¶

8.1. Why a single ordering is not a well-formed output

Within a single authorization comparison, rails may differ enormously in the work their primitive performs. Some complete the primitive without any network round-trip: the critical path is local computation, typically signature verification. Others cannot answer without at least one intrinsic round-trip to a backing service.¶

When the spread between these classes is large relative to the spread within them, a single ordering across both encodes class membership rather than rail quality. The resulting table looks like a ranking and functions as a restatement of which rails do local work.¶

8.2. The requirement

A conforming measurement MUST classify each measured primitive into a work class:¶

Local-complete:

No network round-trip to a backing service lies on the critical path of the primitive.¶

Network-dependent:

At least one intrinsic backing-service round-trip lies on the critical path of each call.¶

Comparisons MUST be made within a work class. A conforming measurement MUST NOT produce a single ordinal comparison spanning both work classes.¶

Where a cross-class statement is needed, it MUST be made through the decomposition tuple defined below, and not through an ordering.¶

8.3. The decomposition tuple

For each measured rail, a conforming measurement MUST report the tuple:¶

• the local compute floor, being the portion of the measurement attributable to computation performed by the measuring party;¶

• the backing-service component, being the portion attributable to round-trips to services the rail depends on;¶

• the end-to-end duration.¶

The end-to-end duration is the headline figure. A conforming measurement MUST NOT subtract the backing-service component from the end-to-end figure, because the round-trip to a backing service is part of the authorization work the rail performs, not overhead added by measurement. The decomposition is a secondary, protocol-design view, published alongside the headline and never in place of it.¶

8.4. Topology requirements

A single topology measures the measurement harness as much as it measures the rail. Accordingly:¶

• A conforming comparison involving a network-dependent primitive MUST draw on measurements from at least two network-distinct topologies. Each such measurement is a separate system under test (Section 5); the requirement is on the comparison, not on any single measurement.¶

• Each reported figure MUST carry the topology it was taken from, and MUST carry the minimum observed round-trip floor for that topology, so that a reader can separate the rail's behaviour from the route.¶

• Local-complete primitives are expected to be topology-invariant. Measuring them from the same topologies provides a control: if a local-complete figure varies materially across topologies, the harness is being measured and the measurement is not yet valid.¶

Where a rail exposes distinct work modes within one work class, each mode is reported separately with the work it performs disclosed, and the mode representing autonomous agent operation is the one reported as that rail's headline for the agentic case.¶

9.1. Two postures

A measuring party may sit in the payment path or outside it.¶

An in-path measurer operates a leg of the flow. It may hold keys, sign, submit, or serve one side of the exchange. Such a party can measure everything, and can also, in principle, affect the payment it is measuring. Its measurements are measurements of a system that includes the measurer.¶

An out-of-path measurer does not touch funds, keys, signing or broadcast. It learns about a payment that has already happened, either by reading a public ledger after the fact or by being told by a participant after the fact.¶

The distinction is long established in payment messaging, where a copy of a message taken in the flow before completion is a different artefact, with different capabilities and different risk, from a copy of a settlement record taken after the fact for reporting.¶

9.2. The requirement

A conforming out-of-path measurement MUST satisfy all of the following:¶

• The measuring component MUST NOT hold or have access to key material for any party to the payment.¶

• The measuring component MUST NOT sign, and MUST NOT construct, submit or broadcast any payment or transaction.¶

• The measuring component MUST use read-only interfaces to any settlement system it observes.¶

• The measuring component MUST NOT share code paths with any in-path component in a way that could give it signing capability. Where both an in-path harness and an out-of-path observer exist in the same project, the observer MUST NOT import the harness and the harness MUST NOT import the observer.¶

9.3. Mechanical enforcement

An out-of-path claim that rests on a written assertion is not verifiable by a reader, and degrades silently as code changes.¶

A conforming measurement therefore MUST enforce the properties above mechanically, in automated checks that run on every change, and MUST describe those checks in the published methodology. Suitable enforcement includes an import allowlist that excludes signing libraries and key material, a restriction to read-only remote-procedure-call methods, and a bidirectional check that the in-path and out-of-path components do not import each other.¶

A failing enforcement check means the out-of-path claim is false for that revision, and results produced by that revision MUST NOT be published as out-of-path.¶

9.4. What each posture can and cannot measure

An out-of-path measurer that reads a ledger after settlement cannot observe any event the payer performed privately, and in particular cannot observe the moment the payer began. It therefore cannot report any duration anchored at a payer event.¶

An out-of-path measurer that is told about a payment by a participant after the fact can report durations anchored at that participant's own events, because the participant observed them, but only those.¶

These limits are not deficiencies to be worked around. They are the reason the next section exists.¶

10.1. The requirement

Every reported duration MUST carry the clock it was measured on.¶

A duration anchored at an event the reporting party itself performed is on the payer clock. A duration anchored at the moment an observing process first saw a third-party event is on the observer clock. The second includes the observer's polling interval, its network path to whatever it polls, and its own scheduling; the first does not.¶

Measurements on different clocks MUST NOT be merged into one series, one average, or one comparison, even where they nominally describe the same transition. Where a data schema is used to record measurements, that schema SHOULD make the merge impossible to express rather than merely discouraged.¶

10.2. No synthesised durations

Where a measuring party did not observe the start of an interval, it MUST NOT report a duration for that interval.¶

This applies specifically to retrospective observation of a settlement record. Such a record commonly carries enough information to make a plausible duration computable by assumption. A conforming measurement MUST NOT compute it. A post-hoc observer cannot know the durations it did not observe and MUST NOT invent them, and a conforming schema MUST reject records that carry duration fields the observation mode could not have produced.¶

A retrospective observation that yields no durations is a complete measurement of the static facts it did observe, and conforms as such. Conformance in this document is defined per observation mode: a report conforms if it satisfies every requirement applicable to the mode it declares. The requirements in Section 13 that presuppose durations, or that presuppose the decomposition tuple, apply only to observation modes capable of producing them, and their absence from a retrospective observation is neither a defect nor a non-conformance. Every report MUST declare its observation mode, so that a reader can tell which requirements were in force.¶

10.3. Fee attribution

Where a rail's design causes a network fee to be borne by a party other than the payer, a conforming measurement MUST record the fee as the fact it is and MUST NOT attribute it to a party that did not bear it.¶

11.1. Comparison

Where a methodology compares rails pairwise, it MUST specify the estimator in advance. The method of paired comparisons [BT] is suitable for deriving a latent comparative strength from pairwise outcomes, and where it is used, ties MUST be handled by an explicit extension rather than by discarding tied observations.¶

Comparison MUST be confined within a work class (Section 8) and within an authorization primitive kind (Section 7).¶

11.2. Censoring

Rejections, timeouts and other non-completions MUST be treated explicitly as censored or competing outcomes. They MUST NOT be silently dropped from the sample, because dropping them biases the result toward the conditions under which a rail succeeds.¶

Where the data violate the assumptions of the chosen pairwise model, the methodology MUST specify the fallback analysis in advance, together with the data-driven trigger conditions that select it. Selecting a fallback after seeing that it produces a more agreeable result is not conforming.¶

11.3. Interval reporting

Point estimates MUST be accompanied by interval estimates, and the full interval MUST be reported alongside any summary figure derived from it. Where a proportion such as a success rate is reported, a score interval [WILSON] is suitable; the conservative bound for a success proportion is the lower bound, because it is the bound that does not flatter the measured system. Which bound is conservative depends on the quantity and on the direction in which being wrong would mislead: for a latency, the conservative bound is the upper one. A methodology MUST state which bound it treats as conservative for each reported quantity, and MUST NOT report a bound in place of the interval.¶

11.4. Distribution reporting

For any latency-like quantity, the central tendency alone is not an adequate summary, because a rail whose behaviour depends on a network path can have a median that moves materially as the sample grows while its tail tells the stable story.¶

A conforming measurement MUST report, for each measured rail and each measured quantity, at minimum: the median, an upper percentile, a further upper percentile characterising the tail, the sample size, the measurement timestamp, and the topology.¶

Any threshold grid used to report the probability of completing within a bound MUST be fixed in advance and MUST NOT be refitted per run.¶

12.1. The requirement

Where a methodology will be used to produce published comparisons, the design and the analysis plan MUST be fixed before any result is generated, and the fixed form MUST be recorded in a way that a third party can verify was fixed beforehand.¶

The record MUST identify the frozen artefacts by cryptographic digest, and MUST be committed to at least one anchor that the publishing party does not control.¶

Reproducibility requires that any deterministic component of the methodology, including any simulation or calibration pipeline, produce identical output from identical input, and that the expected digest of that output be part of the pre-registered record. Random number generation used in such a pipeline MUST be seeded deterministically and MUST be domain-separated per measured dimension, so that adding a dimension cannot perturb an existing one.¶

12.2. What anchors prove, and what they do not

An anchor establishes that a specific byte sequence existed at a specific time and was committed to by a specific signer. That is precedence, integrity and signer identity.¶

An anchor does not establish that the anchored method is correct, that its results are valid, or that its author originated any cryptographic primitive it uses. Primitives such as hash functions, signature schemes and transparency logs are commodity standards used as-is.¶

A conforming methodology SHOULD state this distinction explicitly in its own published record. Overstating what an anchor proves is a failure mode of pre-registration practice and undermines the credibility the anchor was intended to supply.¶

12.3. Worked example

The methodology described in this document was developed alongside two pre-registered measurement dimensions whose records are public and may be inspected as an instance of the requirements above: a settlement-finality pre-registration [OSF-FINALITY] and an authorization-latency pre-registration [OSF-AUTHLAT]. Each fixes its design and analysis plan, identifies its frozen artefacts by digest, and is committed to independent anchors. They are cited here as an example of the practice; what each record does and does not contain is stated below.¶

Competing interests. Both pre-registrations are the author's own. The author operates the measurement project from which this methodology was distilled, and the requirements stated in this document were derived from that project's two measurement dimensions, exercised to date on test networks and on traffic the project itself originated. The records contain the pre-registered designs, the digests of the frozen artefacts, and, for the authorization-latency record, a first-party test-network calibration record disclosed as pilot evidence; neither contains a result for any production rail. The author is therefore a prospective publisher under this methodology as well as its editor. Readers should weigh the requirements on their merits and against their own deployments rather than on the authority of the worked example.¶

13.1. Disclosure of limitations

A conforming report MUST disclose known limitations of its own measurements, and MUST do so as part of the pre-registered record rather than only after results are known.¶

This requirement exists to prevent silent caps: a limitation that is known to the measuring party, is material to interpretation, and is simply not mentioned. Limitations that MUST be disclosed where they apply include at least:¶

• measurement taken under load conditions unrepresentative of production, where the effect on distribution tails is expected to be material;¶

• measurement of a path that omits a step the production path would include;¶

• measurement on a test network whose production counterpart shares a design but not necessarily its empirical behaviour;¶

• any bound that is known to be optimistic or pessimistic rather than representative.¶

Where a methodology bounds its own coverage, by sampling, by capping retries, or by reporting only a subset, the bound MUST be stated. Silent truncation reads to a reader as complete coverage.¶

16.1. Normative References

[RFC2119]

Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/rfc/rfc2119>.

[RFC8174]

Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

16.2. Informative References

[BT]

Bradley, R. A. and M. E. Terry, "Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons", Biometrika 39(3/4), pp. 324-345, 1952.

[I-D.contreras-bmwg-ai-agent-benchmarking]

Contreras, L. M., "Benchmarking Methodology for AI Agents in Network Operations", Work in Progress, Internet-Draft, draft-contreras-bmwg-ai-agent-benchmarking-00, 6 July 2026, <https://datatracker.ietf.org/doc/html/draft-contreras-bmwg-ai-agent-benchmarking-00>.

[I-D.irtf-pearg-safe-internet-measurement]

Learmonth, I. R., Knodel, M., and G. Grover, "Considerations for Performing Safe Measurement on the Internet", Work in Progress, Internet-Draft, draft-irtf-pearg-safe-internet-measurement-14, 25 January 2026, <https://datatracker.ietf.org/doc/html/draft-irtf-pearg-safe-internet-measurement-14>.

[I-D.laxsharma-pact]

Sharma, L., "PACT: A Contract Layer for Autonomous Agent Commerce", Work in Progress, Internet-Draft, draft-laxsharma-pact-00, 27 July 2026, <https://datatracker.ietf.org/doc/html/draft-laxsharma-pact-00>.

[I-D.ryan-httpauth-payment]

Ryan, B., Moxey, J., Meagher, T., Weinstein, J., and S. Kaliski, "The "Payment" HTTP Authentication Scheme", Work in Progress, Internet-Draft, draft-ryan-httpauth-payment-01, 17 March 2026, <https://datatracker.ietf.org/doc/html/draft-ryan-httpauth-payment-01>.

[ISO8583]

International Organization for Standardization, "ISO 8583: Financial transaction card originated messages, Interchange message specifications", ISO 8583, 2003.

[OSF-AUTHLAT]

Blake, M., "PayBench authorization-latency benchmark: pre-registration", DOI 10.17605/OSF.IO/UFQG5, 2026, <https://doi.org/10.17605/OSF.IO/UFQG5>.

[OSF-FINALITY]

Blake, M., "PayBench settlement-finality benchmark: pre-registration", DOI 10.17605/OSF.IO/XGFUJ, 2026, <https://doi.org/10.17605/OSF.IO/XGFUJ>.

[RFC2544]

Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, DOI 10.17487/RFC2544, March 1999, <https://www.rfc-editor.org/rfc/rfc2544>.

[RFC9110]

Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, DOI 10.17487/RFC9110, June 2022, <https://www.rfc-editor.org/rfc/rfc9110>.

[WILSON]

Wilson, E. B., "Probable Inference, the Law of Succession, and Statistical Inference", JASA 22(158), pp. 209-212, 1927.