Computing-Aware Traffic Steering H. Wang Internet-Draft Q. Li Intended status: Informational Pengcheng Laboratory Expires: 28 February 2027 Y. Jiang Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory M. Xu Tsinghua University G. Muntean Dublin City University 27 August 2026 An Architecture for Open, Decentralized, and Scalable Large Language Model Inference draft-wang-cats-odsi-01 Abstract Large Language Model (LLM) inference is normally operated by one provider, even when the provider distributes execution across many sites. This document describes a different system model in which independently operated and mutually untrusted participants contribute compute, memory, model storage, and network capacity to one inference service. No single administrative entity is required to admit participants, select every execution path, hold the complete model, verify all results, or settle all resource contributions. This document defines the Open, Decentralized, and Scalable Inference (ODSI) architecture. It specifies the architectural roles, trust boundaries, named objects, protocol-independent interfaces, execution workflow, verification choices, timing model, and security and privacy requirements needed to construct a multi-operator inference overlay. It also identifies the protocol and operational choices that each ODSI deployment must specify so that independently developed participants can interoperate. ODSI is related to Computing-Aware Traffic Steering (CATS), but it does not extend the CATS single-provider model across trust domains. CATS mechanisms can be used within a participating domain or as an input to local path selection. Cross-domain membership, model governance, execution verification, and settlement are separate ODSI functions. This document is Informational and does not define a wire format, consensus algorithm, payment system, or new CATS metric. About This Document This note is to be removed before publishing as an RFC. Wang, et al. Expires 28 February 2027 [Page 1] Internet-Draft ODSI August 2026 Status information for this document may be found at https://datatracker.ietf.org/doc/draft-wang-cats-odsi/. Discussion of this document takes place on the Computing-Aware Traffic Steering Working Group mailing list (mailto:cats@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/cats/. Subscribe at https://www.ietf.org/mailman/listinfo/cats/. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 28 February 2027. Copyright Notice Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1. Goals . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3. Non-Goals . . . . . . . . . . . . . . . . . . . . . . . . 6 1.4. Relationship to CATS and IDN . . . . . . . . . . . . . . 6 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 7 3. System Model . . . . . . . . . . . . . . . . . . . . . . . . 9 Wang, et al. Expires 28 February 2027 [Page 2] Internet-Draft ODSI August 2026 3.1. Decentralization Properties . . . . . . . . . . . . . . . 9 3.2. Roles . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3. Network and Timing Assumptions . . . . . . . . . . . . . 10 3.4. Adversary and Failure Model . . . . . . . . . . . . . . . 11 3.5. Required Deployment Profile . . . . . . . . . . . . . . . 11 4. Named Objects and Execution Semantics . . . . . . . . . . . . 12 4.1. Model Manifest . . . . . . . . . . . . . . . . . . . . . 12 4.2. Execution Profile . . . . . . . . . . . . . . . . . . . . 12 4.3. Request, Step, and State Identity . . . . . . . . . . . . 13 4.4. Resource Profile . . . . . . . . . . . . . . . . . . . . 14 4.5. Commitments, Receipts, and Evidence . . . . . . . . . . . 14 5. ODSI Architecture . . . . . . . . . . . . . . . . . . . . . . 15 5.1. Functional Overview . . . . . . . . . . . . . . . . . . . 15 5.2. Protocol-Independent Interfaces . . . . . . . . . . . . . 16 5.3. Path Construction . . . . . . . . . . . . . . . . . . . . 16 6. Protocol-Independent Workflow . . . . . . . . . . . . . . . . 17 6.1. Participant Join and Resource Publication . . . . . . . . 17 6.2. Request and Path Establishment . . . . . . . . . . . . . 17 6.3. Inference Execution . . . . . . . . . . . . . . . . . . . 18 6.4. Failure and Path Repair . . . . . . . . . . . . . . . . . 18 6.5. Verification and Settlement . . . . . . . . . . . . . . . 18 7. Correctness and Verification . . . . . . . . . . . . . . . . 19 7.1. Correctness Definition . . . . . . . . . . . . . . . . . 19 7.2. Verification Methods . . . . . . . . . . . . . . . . . . 19 7.3. Evidence and Consequences . . . . . . . . . . . . . . . . 20 8. Timing, State, and Recovery . . . . . . . . . . . . . . . . . 20 8.1. Deadline Semantics . . . . . . . . . . . . . . . . . . . 20 8.2. Stateful Execution . . . . . . . . . . . . . . . . . . . 21 8.3. Churn and Availability . . . . . . . . . . . . . . . . . 21 9. Incentives, Sybil Resistance, and Governance . . . . . . . . 22 9.1. Incentive Scope . . . . . . . . . . . . . . . . . . . . . 22 9.2. Sybil and Collusion Resistance . . . . . . . . . . . . . 22 9.3. Settlement and Adjudication . . . . . . . . . . . . . . . 22 9.4. Model and Protocol Governance . . . . . . . . . . . . . . 23 10. CATS Applicability . . . . . . . . . . . . . . . . . . . . . 23 11. Scalability and Performance Considerations . . . . . . . . . 24 12. Manageability and Operational Considerations . . . . . . . . 25 13. Security Considerations . . . . . . . . . . . . . . . . . . . 26 13.1. Identity and Message Security . . . . . . . . . . . . . 26 13.2. Manifest and Software Supply Chain . . . . . . . . . . . 26 13.3. Resource Discovery and Path Manipulation . . . . . . . . 27 13.4. Execution Integrity . . . . . . . . . . . . . . . . . . 27 13.5. State and Replay Attacks . . . . . . . . . . . . . . . . 27 13.6. Denial of Service . . . . . . . . . . . . . . . . . . . 28 13.7. Economic and Governance Attacks . . . . . . . . . . . . 28 13.8. Residual Risk . . . . . . . . . . . . . . . . . . . . . 28 14. Privacy Considerations . . . . . . . . . . . . . . . . . . . 28 15. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29 Wang, et al. Expires 28 February 2027 [Page 3] Internet-Draft ODSI August 2026 16. Relationship to Existing Work . . . . . . . . . . . . . . . . 29 17. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 30 18. Informative References . . . . . . . . . . . . . . . . . . . 30 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 32 1. Introduction LLM inference is a stateful network service. A request selects a model and execution configuration, performs a prefill operation over the input, and then generates output tokens incrementally. Each generation step depends on model parameters and per-request state accumulated by earlier steps. Implementations commonly divide a model into stages and retain key-value (KV) caches near the stages that use them. Most distributed inference systems remain under one administrative entity. The entity admits resources, distributes model artifacts, selects execution nodes, monitors them, and decides which results to trust. The Intelligence Delivery Network (IDN) architecture [I-D.li-cats-idn] deliberately uses this provider-controlled model and applies CATS concepts to the placement and selection of distributed inference capabilities. ODSI considers a different environment. Compute nodes belong to independent entities, no prior contractual trust is assumed, membership can change, and nodes may be faulty, rational, or malicious. A model can be represented by content-addressed artifacts held by multiple participants, and a request can be executed by a temporary path assembled from their resources. The objective is technical decentralization of control and custody, i.e., no single administrative entity is required to control participant admission, model-artifact availability, execution-path selection, result verification, or contribution settlement. Existing techniques in peer-to-peer content distribution and decentralized ledgers illustrate different parts of this design space. Content-addressed pieces, peer discovery, and replication are useful for distributing model artifacts [BitTorrent]. Permissionless identity, Sybil resistance, and consistent settlement state are relevant to resource contribution [Bitcoin]. But neither of them solves inference correctness. A digest proves the integrity of an artifact but not that a participant evaluated a model stage correctly, and repeating an expensive, numerically variable computation is not equivalent to validating a small deterministic ledger transition. Wang, et al. Expires 28 February 2027 [Page 4] Internet-Draft ODSI August 2026 This document therefore separates artifact integrity, execution correctness, service timeliness, and economic settlement. Each has different evidence and trust requirements. 1.1. Goals ODSI has the following architectural goals: * allow independently operated nodes to advertise partial resources and participate without relying on one service provider; * identify model artifacts, execution semantics, requests, steps, and state sufficiently precisely for independent participants to interoperate; * construct state-aware execution paths using heterogeneous and changing resource information; * attribute protocol actions and make the limits of that attribution explicit; * support selectable verification methods with stated correctness, cost, latency, and trust properties; * contain failures and permit recovery without requiring unrelated requests to stop; and * permit, but not require, a settlement mechanism that accounts for independently supplied resources. 1.2. Scope This document defines a protocol-independent architecture. It describes the information that crosses component boundaries and the properties that a complete deployment profile needs to provide. Separate specifications can define encodings, transports, discovery mechanisms, verification methods, consensus protocols, or settlement systems. ODSI is an application-layer overlay. The underlay forwards packets without knowledge of models, tensors, token positions, deadlines, rewards, or execution paths. An ODSI implementation can use existing secure transports and can use CATS inside a provider domain, but this document does not modify Internet routing or transport protocols. Wang, et al. Expires 28 February 2027 [Page 5] Internet-Draft ODSI August 2026 1.3. Non-Goals This document does not: * standardize a token, ledger, consensus algorithm, proof system, trusted execution environment, or reputation formula; * assert that decentralization always reduces latency, cost, energy use, data movement, or concentration; * provide confidentiality merely by splitting a model or request among nodes; * define legal ownership or licensing of a model; * require that no participant ever stores a complete model; or * guarantee progress during an unbounded network partition. ODSI can remove the technical requirement for a single operator, but it cannot by itself decentralize model ownership, licensing, governance, or economic power. A deployment therefore needs separate policies for model authorization, protocol changes, participant governance, and compensation, as discussed in [RFC9518]. 1.4. Relationship to CATS and IDN CATS selects a suitable service contact instance using network and compute information in a single service-provider environment [I-D.ietf-cats-framework] [I-D.ietf-cats-usecases-requirements]. IDN adds inference-specific capability, artifact, state, and policy concepts while retaining a provider-controlled administrative domain [I-D.li-cats-idn]. ODSI differs in the trust and control model: Wang, et al. Expires 28 February 2027 [Page 6] Internet-Draft ODSI August 2026 +================+=====================+============================+ | Property | IDN | ODSI | +================+=====================+============================+ | Administrative | One provider- | Independently operated | | scope | controlled domain | participants | +----------------+---------------------+----------------------------+ | Admission | Provider policy | Open discovery plus | | | | profile-selected | | | | admission and Sybil | | | | resistance | +----------------+---------------------+----------------------------+ | Control | Provider control | No required global | | | plane | controller | +----------------+---------------------+----------------------------+ | Artifact | Provider managed | Content-addressed, | | custody | | replicated, and | | | | potentially partitioned | | | | across participants | +----------------+---------------------+----------------------------+ | Execution | Provider assurance | Explicit verification | | trust | and policy | profile for mutually | | | | untrusted nodes | +----------------+---------------------+----------------------------+ | Accounting | Provider internal | Optional cross-entity | | | | settlement | +----------------+---------------------+----------------------------+ | Failure model | Managed faults | Faults, churn, | | | | equivocation, | | | | collusion, and resource | | | | attacks | +----------------+---------------------+----------------------------+ Table 1 ODSI does not treat the global peer overlay as a CATS domain. A participant that controls multiple sites can use CATS to select a local resource, and CATS metrics [I-D.ietf-cats-metric-definition] can inform that local decision. ODSI then treats the selected endpoint as one participant in its overlay. CATS does not provide ODSI peer admission, cross-domain trust, model governance, execution proofs, or settlement. 2. Terminology The terms below have architectural meaning in this document. ARTIFACT A content-addressed object used for inference, such as Wang, et al. Expires 28 February 2027 [Page 7] Internet-Draft ODSI August 2026 model weights, a tokenizer, an adapter, executable code, or a configuration object. CLIENT The entity that creates an INFERENCE REQUEST and consumes its result. A CLIENT can also operate other ODSI roles. COORDINATION PLANE The non-latency-critical functions used for discovery, MODEL MANIFEST publication, resource advertisement, path construction, evidence handling, governance, and optional settlement The term denotes functions and interfaces, not a required centralized controller. DEADLINE A locally interpreted time bound associated with a request or step. A DEADLINE is not globally enforceable evidence unless the selected profile defines a trusted time and observation model. EXECUTION COMMITMENT A signed statement binding a PARTICIPANT to a request, EXECUTION PROFILE, execution unit, input commitment, predecessor, and locally interpreted timing terms. A commitment attributes a statement; it does not prove correct computation. EXECUTION PATH An ordered set of execution assignments that, together, evaluate an INFERENCE REQUEST. EXECUTION PROFILE A versioned description of the semantics required to evaluate and verify a model, including partitioning, tensor formats, numerical behavior, decoding, and the VERIFICATION PROFILE. EXECUTION RECEIPT A signed record of an attempted execution unit, including input and output commitments, predecessor information, observed timing, and state information required by the selected verification method. EXECUTION UNIT The smallest computation independently assigned and receipted by an ODSI deployment. It can contain one layer, a group of layers, an expert, or another graph segment. INFERENCE PLANE The latency-sensitive functions that transport request data and state, execute assigned units, return receipts, and report failures. INFERENCE REQUEST A request bound to a MODEL MANIFEST, EXECUTION PROFILE, input, decoding parameters, and client policy. MODEL MANIFEST A signed, content-addressed description of the artifacts, graph, versions, compatibility information, and governance metadata that identify an executable model. Wang, et al. Expires 28 February 2027 [Page 8] Internet-Draft ODSI August 2026 PARTICIPANT An independently operated entity that contributes compute, memory, artifact storage, or network capacity using one or more cryptographic identities. RESOURCE PROFILE A signed, time-limited description of resources and supported execution capabilities advertised by a PARTICIPANT. SETTLEMENT MECHANISM An optional mechanism that records accepted contributions and transfers or accounts for value. Settlement is outside the inference critical path. STATE HANDLE A reference to per-request or reusable state, including its type, version, location or retrieval information, integrity commitment, and access policy. VERIFICATION PROFILE The selected method and parameters for deciding whether an execution result is acceptable and whether evidence is sufficient for a consequence. 3. System Model 3.1. Decentralization Properties ODSI separates five properties that are sometimes conflated as model ownership: * resource ownership: compute and storage are supplied by independent entities; * artifact custody: no single participant is required to retain every artifact; * operational control: no single participant is required to admit all nodes or schedule all work; * verification control: no single participant's assertion is sufficient in profiles intended to tolerate malicious execution; and * governance control: model and protocol changes follow the governance mechanism named by the deployment profile. A deployment can decentralize these properties to different degrees. It needs to disclose which property remains centralized. ODSI does not label a system decentralized solely because it runs on geographically distributed nodes. Wang, et al. Expires 28 February 2027 [Page 9] Internet-Draft ODSI August 2026 3.2. Roles The architecture defines the following logical roles. One participant can perform several roles, and a role can be replicated: * clients create requests and specify acceptable service, privacy, cost, and verification policy; * artifact providers publish or retain model artifacts; * execution nodes evaluate execution units and retain associated state; * discovery nodes help locate manifests, artifacts, resources, and peers; * path constructors select candidate execution assignments; * verifiers evaluate receipts or results under a verification profile; * evidence adjudicators apply profile-defined rules to objective evidence; and * settlement nodes record accepted contributions when settlement is enabled. The last three roles are not automatically trustworthy. A deployment profile defines their selection, replication, independence assumptions, and decision rule. 3.3. Network and Timing Assumptions Participants communicate over an asynchronous packet network. Messages can be delayed, lost, duplicated, reordered, or replayed, and participants can become unreachable. ODSI does not assume a global upper bound on delay or globally synchronized clocks. Consequently, the architecture cannot guarantee both deterministic progress and safety under every partition or fault pattern [FLP]. Interactive deployments can operate under periods of practical synchrony, using observed delay distributions and local timeout policies. A timeout is evidence that an observer did not receive a result in time; it is not, by itself, proof that a remote node failed to compute or transmit the result. Profiles that penalize deadline violations need an explicit trusted observation, timestamping, or multi-observer rule. Wang, et al. Expires 28 February 2027 [Page 10] Internet-Draft ODSI August 2026 3.4. Adversary and Failure Model Participants can crash, disconnect, perform slowly, advertise false resources, return incorrect results, equivocate, replay old state, withhold artifacts, selectively abort, or collude. An adversary can create many identities, observe and modify traffic it controls, and attempt to isolate a participant from honest peers. The base architecture does not assume an honest global majority. A concrete profile that depends on an honest majority, honest stake fraction, trusted hardware operator, proof-system assumption, or independent verifier set needs to state that dependency and the consequence when it fails. "Majority" needs to name the measured resource, such as identities, stake, assigned executions, or verifier weight. Clients can also be malicious. They can submit resource-exhausting inputs, induce repeated path construction, dispute correct work, or attempt to extract model artifacts and other users' state. 3.5. Required Deployment Profile The architecture requies a versioned ODSI profile selects at least the following specifications before deployment: * identity, authentication, key rotation, and revocation mechanisms; * peer, manifest, artifact, and resource discovery mechanisms; * canonical encodings and digest algorithms for named objects; * internationalization, comparison, and display rules for human- readable fields, while keeping protocol identifiers unambiguous; * an EXECUTION PROFILE and compatible transport bindings; * a VERIFICATION PROFILE and its adversary assumptions; * a path-construction and admission policy; * failure, retry, state-recovery, and cancellation behavior; * privacy and transport-confidentiality behavior; * a governance mechanism for profile and model changes; and * Sybil resistance, evidence adjudication, and settlement behavior when those functions are enabled. Wang, et al. Expires 28 February 2027 [Page 11] Internet-Draft ODSI August 2026 This list is a conformance boundary for future specifications. Implementations using different unresolved choices are not assumed to interoperate. 4. Named Objects and Execution Semantics 4.1. Model Manifest A MODEL MANIFEST provides stable identity for the computation requested by a client. At minimum, it contains or references: * a model identifier and manifest version; * the content digest, size, encoding, and retrieval information for every required artifact; * a graph that maps execution-unit identifiers to artifacts and their dependencies; * tokenizer, adapter, runtime, safety-component, and configuration identifiers; * supported EXECUTION PROFILE identifiers; * compatibility and deprecation information; * publisher signatures and the governance rule for authorized updates; and * licensing or usage-policy references where applicable. Artifact digests detect corruption or substitution after a manifest is trusted. They do not establish that a model is safe, lawful, unbiased, useful, or free of malicious code. Trust in a manifest publisher and authorization of updates are profile-level decisions. An implementation can replicate or erasure-code artifacts, and no participant is required by this architecture to store a complete checkpoint. However, every execution unit needs timely access to its required artifacts. A profile needs an availability strategy that prevents a unique artifact holder from becoming a single point of failure. 4.2. Execution Profile An EXECUTION PROFILE removes ambiguity about what it means to execute a manifest. It defines: Wang, et al. Expires 28 February 2027 [Page 12] Internet-Draft ODSI August 2026 * execution-unit boundaries and their ordering; * tensor names, dimensions, element types, byte order, framing, and compression; * supported quantization and runtime variants; * state creation, lookup, migration, expiration, and invalidation semantics; * prefill, decode, batching, and cancellation behavior; * decoding parameters and the party responsible for stochastic sampling; * numerical equivalence rules; and * the identifier and parameters of the VERIFICATION PROFILE. Floating-point results can differ across accelerators, kernels, batching choices, and compiler versions. A profile can require bitwise-deterministic execution, define a tolerance and comparison procedure, verify a deterministic representation such as quantized outputs, or use a proof or attestation mechanism. A digest comparison alone only works when the profile requires identical bytes. Stochastic sampling needs special treatment. A profile can bind a seed and algorithm, move sampling to the client, or verify logits before sampling. Without such a rule, two honest executions can produce different output tokens. 4.3. Request, Step, and State Identity An INFERENCE REQUEST is bound to the digest of its MODEL MANIFEST and EXECUTION PROFILE. It also identifies the input commitment, requested operation, decoding parameters, client policy, and a nonce that prevents accidental request collision. Every assigned step is identified by the request identifier, token or prefill position, execution-unit identifier, attempt number, and predecessor commitment. This tuple prevents a valid output for one position, model version, or retry from being substituted into another. Wang, et al. Expires 28 February 2027 [Page 13] Internet-Draft ODSI August 2026 A STATE HANDLE identifies the state type, producing step, manifest and profile versions, integrity commitment, access scope, expiration policy, and available locations. Possession of a state handle does not itself authorize access to the state. 4.4. Resource Profile A RESOURCE PROFILE describes what a participant offers rather than exposing every hardware detail. It can include: * supported execution units, profiles, runtimes, and artifact residency; * accelerator and memory classes; * capacity, queue, throughput, and observed execution-time ranges; * network reachability and locality information; * state capacity and migration support; * verification or attestation capabilities; * administrative, jurisdictional, and privacy attributes; and * validity interval, sequence number, and participant signature. Fast-changing values are claims, not facts. Path construction needs to account for measurement source, freshness, uncertainty, and the cost of verification. Resource profiles use application-level information; raw values are not automatically suitable for distribution as CATS metrics. 4.5. Commitments, Receipts, and Evidence Before accepting an execution assignment, a node can issue an EXECUTION COMMITMENT binding its identity to the request, execution unit, input commitment, predecessor, execution profile, attempt, and agreed service terms. After execution, the node produces an EXECUTION RECEIPT containing the corresponding identifiers, output commitment or output, resulting state commitment, observed local timing, and verification-specific material. Receipts are linked to predecessor receipts so that omission, replay, and equivocation can be investigated. Wang, et al. Expires 28 February 2027 [Page 14] Internet-Draft ODSI August 2026 Signatures provide origin authentication and integrity for these statements. They do not prove that advertised resources existed, that computation occurred, that an output is correct, or that a deadline was objectively missed. Those conclusions require evidence defined by the selected profile. 5. ODSI Architecture 5.1. Functional Overview The architecture separates latency-sensitive inference from slower coordination and optional settlement: +---------------------+ | Client | +----------+----------+ | Request, result, evidence policy | +------------------------v------------------------+ | Inference Plane | | | | Exec A ---> Exec B ---> Exec C ---> ... | | unit 0 units 1-3 unit 4 | | activations, state handles, receipts | +------------------------+------------------------+ | events and observations | +------------------------v------------------------+ | Coordination Plane | | manifest/artifact discovery | resource discovery| | path construction | verification | | evidence adjudication | governance | +------------------------+------------------------+ | accepted contribution | +------------------------v------------------------+ | Optional Settlement Mechanism | +-------------------------------------------------+ The diagram shows functional relationships, not central services. Each coordination function can be distributed or replicated according to the deployment profile. Wang, et al. Expires 28 February 2027 [Page 15] Internet-Draft ODSI August 2026 5.2. Protocol-Independent Interfaces Future ODSI protocol specifications need to cover the following interfaces: * manifest and artifact publication, retrieval, validation, and withdrawal; * peer and resource discovery, including freshness and anti-replay behavior; * execution solicitation, offer, selection, and commitment; * activation and state transport with request and predecessor binding; * execution receipt and failure reporting; * verification request, evidence exchange, and decision reporting; * cancellation, retry, replacement, and state recovery; and * contribution submission to an optional settlement mechanism. These interfaces can use different protocols. A deployment profile identifies the binding between them and prevents downgrade to weaker identity, privacy, or verification behavior. 5.3. Path Construction A path constructor selects an ordered set of assignments compatible with the manifest, execution profile, resource profiles, client policy, and current observations. Path construction can be performed by a client, a set of brokers, an auction, a distributed algorithm, or another profile-defined mechanism. ODSI does not require a globally optimal path or a single scheduler. A candidate path needs to account for compute time, queueing, artifact availability, activation transfer, state affinity, verification overhead, failure probability, diversity, privacy policy, and cost. Selecting each unit independently can create a path whose slowest boundary dominates the token rate. Implementations therefore commonly assign contiguous units and favor stable paths across token steps. Wang, et al. Expires 28 February 2027 [Page 16] Internet-Draft ODSI August 2026 If the verification profile relies on redundant execution, path construction also enforces its independence or diversity assumptions. Merely choosing two identities is insufficient when both can be controlled by one entity. 6. Protocol-Independent Workflow 6.1. Participant Join and Resource Publication * First, a participant creates or obtains a profile-compatible identity and discovers bootstrap peers through the selected discovery mechanism. * The participant then obtains the active ODSI profile and the manifests it intends to support. It validates their governance and signatures before retrieving artifacts. * The participant validates artifact digests, prepares compatible runtimes, and publishes a signed, expiring RESOURCE PROFILE. * Finally, discovery nodes distribute or index the profile according to the profile's privacy, freshness, and Sybil-resistance rules. Joining discovery does not automatically grant assignments. Path constructors apply admission, diversity, reputation, collateral, attestation, or challenge policies defined by the deployment profile. 6.2. Request and Path Establishment * First, a client selects a MODEL MANIFEST, EXECUTION PROFILE, and client policy. The policy includes acceptable verification, confidentiality, latency, cost, and jurisdiction constraints. * A path constructor then obtains candidate resource profiles and recent observations. It builds one or more feasible paths and reserves resources if the selected protocol supports reservation. * Selected execution nodes validate the request and predecessor information, then return commitments or rejection reasons. * Finally, the client or path constructor activates a path only after the commitments required by its verification and failure policy are available. Wang, et al. Expires 28 February 2027 [Page 17] Internet-Draft ODSI August 2026 6.3. Inference Execution * First, the initial assigned node receives a request-bound input and verifies its manifest, profile, step, attempt, and predecessor fields. * It then evaluates its assigned execution unit, retains or emits state as specified, and sends the next activation together with a receipt or receipt reference. * Each successor rejects inputs that are stale, duplicated, incompatible, unauthorized, or not linked to the expected predecessor. * Finally, the final node returns the profile-defined result and receipt chain to the client. Verification work can occur inline, asynchronously, or both. An asynchronous verifier cannot retract an output already acted upon by a client. Applications that require prevention rather than later detection need inline verification or a trusted execution path and must include its latency in the service objective. 6.4. Failure and Path Repair When a local timeout or explicit failure occurs, the path constructor can retry the same node, use a warm replica, reassign downstream work, migrate state, reconstruct state from an earlier checkpoint, or restart the request. The selected action depends on remaining client time, state size, artifact availability, and verification policy. Every retry receives a new attempt identifier. Late results from an earlier attempt cannot be inserted into the repaired path. A repaired path records the last accepted predecessor so verifiers can distinguish a valid retry from equivocation. Path repair is not always transparent. Loss of unique KV state can require replay of all preceding tokens, and loss of a unique model artifact can make progress impossible. Profiles intended for interactive service need replication or checkpoint rules that bound these risks under their stated failure assumptions. 6.5. Verification and Settlement Receipts and verification material are evaluated according to the VERIFICATION PROFILE. An accepted verification decision identifies the evidence, decision rule, participating verifiers, and confidence or finality semantics. Wang, et al. Expires 28 February 2027 [Page 18] Internet-Draft ODSI August 2026 Only accepted contributions are submitted for settlement. A settlement mechanism can aggregate per-step receipts rather than place every inference event in a global consensus log. Disputed contributions remain pending until the profile-defined adjudication rule resolves or expires them. 7. Correctness and Verification 7.1. Correctness Definition For ODSI, an execution result is correct only relative to a specific manifest and execution profile. Correctness includes: * the expected execution unit consumed the expected predecessor output; * required artifacts and state versions were used; * the output satisfies the profile's numerical-equivalence rule; * execution did not violate a profile-defined policy that is part of correctness; and * the result belongs to the identified request, position, and attempt. Availability and deadline adherence are service properties, not computational correctness. A correct late result and a timely incorrect result need different handling. 7.2. Verification Methods No single verification method is suitable for all models and deployments. A VERIFICATION PROFILE can use one or more of the following: * redundant execution compares independently produced results; it adds compute cost and requires a decision rule for numerical differences and collusion; * challenge-based recomputation samples steps or subcomputations; it reduces average verification cost but detects cheating probabilistically and needs unpredictable challenges; * trusted execution evidence can bind code and artifacts to a hardware-backed environment; it introduces hardware, manufacturer, freshness, side-channel, and revocation assumptions; Wang, et al. Expires 28 February 2027 [Page 19] Internet-Draft ODSI August 2026 * cryptographic validity proofs can provide strong verification for supported computations; they introduce prover cost, circuit or model-binding complexity, setup assumptions, and potentially substantial latency; and * client or verifier recomputation provides a direct reference under deterministic semantics but can eliminate the resource advantage that motivated outsourcing. A profile documents which faults its method detects, its false-accept and false-reject behavior, independence assumptions, and whether a decision is immediate, probabilistic, or eventual. 7.3. Evidence and Consequences Objective evidence can include conflicting signed commitments, two incompatible receipts for the same step and attempt, an invalid artifact digest, or verification material that fails a deterministic check. Signed timing assertions from one party are not objective proof of remote lateness. Consequences need to be proportional to evidence. A profile can distinguish: * service failure, which affects selection or payment but is not proven misconduct; * attributable protocol violation, such as equivocation or replay; * verified incorrect execution under the selected verification method; and * inconclusive disagreement, which can trigger further verification without punishment. This distinction prevents network delay, verifier disagreement, or honest numerical variation from automatically causing a penalty. 8. Timing, State, and Recovery 8.1. Deadline Semantics The client defines an end-to-end service objective, such as time to first token or time per output token. A path constructor can allocate local budgets to execution and transfer steps. These budgets guide scheduling and recovery but do not create globally comparable timestamps. Wang, et al. Expires 28 February 2027 [Page 20] Internet-Draft ODSI August 2026 Remaining slack is an estimate derived from client-observed time, predicted compute and transfer time, and verification overhead. Nodes can report local durations, while receivers can measure arrival times. A profile that needs cross-node timing evidence specifies clock synchronization, error bounds, trusted timestamp sources, and observation rules. ODSI cannot guarantee an upper latency bound over an asynchronous public network. Implementations can report percentile objectives and failure probabilities for a measured operating region. 8.2. Stateful Execution KV caches and other request state can be much larger than an individual activation. Stable assignment across decode steps avoids repeated state movement, while reassignment can require migration or replay. A state policy defines: * which node owns the authoritative current version; * whether and when replicas or checkpoints are created; * how state is encrypted and authorized; * how concurrent attempts are fenced; * how rollback and stale-state use are detected; and * when state is deleted. State reconstruction needs the exact manifest, profile, request prefix, and accepted receipt history. If these are unavailable, restarting from a claimed state can compromise correctness. 8.3. Churn and Availability Open membership makes churn normal. Availability depends on replication of artifacts, diversity of compatible execution nodes, repair capacity, and the amount of non-replicated request state. Reputation or economic penalties do not restore unavailable data and cannot substitute for redundancy. Profiles need explicit minimum replication or recoverability policies for critical artifacts and state. They also need admission control so that apparent resource growth does not overload discovery, verification, or path-construction functions. Wang, et al. Expires 28 February 2027 [Page 21] Internet-Draft ODSI August 2026 9. Incentives, Sybil Resistance, and Governance 9.1. Incentive Scope Independent resource owners incur costs and may require compensation. ODSI therefore supports contribution accounting, but it does not require a particular currency or assume that payment alone causes honest behavior. An accounting rule identifies the unit being purchased, the evidence required for acceptance, cancellation behavior, verification cost, and who pays for retries. Paying solely for submitted outputs encourages spam; paying solely for accepted outputs can expose honest nodes to malicious clients or faulty verifiers. These trade-offs are profile and deployment policy. 9.2. Sybil and Collusion Resistance Cryptographic identities make actions linkable but do not make entities unique. A profile can use collateral, proof of resource, rate limits, admission attestations, diversity constraints, historical performance, or combinations of these mechanisms. Each choice changes openness and concentration risk. Stake makes identities costly only relative to the value and liquidity of the stake and the credibility of enforcement. It does not prove resource ownership, prevent collusion, or establish correct execution. Reputation can also be manipulated through self-dealing, selective behavior, identity retirement, and reputation farming. 9.3. Settlement and Adjudication If a deployment uses a ledger or replicated state machine, its consensus and finality assumptions are part of the ODSI profile. Byzantine consensus techniques illustrate how replicated parties can agree under bounded adversary assumptions [Byzantine], but ODSI does not assume that one such technique is universally suitable. Penalties are applied only by an authorized settlement or governance mechanism and only for profile-defined evidence. Missed local deadlines should normally affect payment or future selection rather than trigger punitive slashing unless the profile provides objective timing evidence. Wang, et al. Expires 28 February 2027 [Page 22] Internet-Draft ODSI August 2026 9.4. Model and Protocol Governance Decentralized resource contribution does not determine who can publish a model version, change safety policy, update a verification method, or repair a vulnerability. A deployment profile names the governance rule for: * manifest publication and revocation; * protocol and profile upgrades; * emergency suspension; * verifier and adjudicator changes; and * incompatible forks. Clients need to bind requests to explicit versions rather than an ambiguous "latest" model. Participants can support several governance domains without treating them as one inference service. 10. CATS Applicability ODSI and CATS operate at different scopes but can be composed. Within a participant's provider-controlled network, a CATS Path Selector can select among service contact instances that provide an ODSI execution capability. Network and compute conditions can help select the local instance. The result is then exposed to the ODSI overlay through one participant identity and resource profile. The following conceptual mapping can be useful inside such a domain: * an ODSI execution endpoint can be realized as a CATS service instance; * a compatible execution-unit capability can contribute to identification of the offered service; * CATS network and compute information can inform the participant's local endpoint selection; and * the participant advertises a service-oriented summary to ODSI rather than exporting raw CATS state across domains. Wang, et al. Expires 28 February 2027 [Page 23] Internet-Draft ODSI August 2026 This composition preserves the CATS single-provider boundary. ODSI does not require a CATS Path Selector to understand global stake, reputation, model governance, or receipt adjudication. It also does not require CATS to steer every layer boundary or expose rapidly changing application state to the network. Before defining ODSI-specific CATS metrics, deployment experience is needed to determine which information is stable, comparable, safe to disclose, and useful for traffic steering. Model identifiers, tensor details, and settlement attributes are application metadata and are not presumed to be CATS metrics. 11. Scalability and Performance Considerations For an execution path with n units, token-step latency includes the sum of unit execution time, queueing time, inter-unit transfer time, state operations, and inline verification. Throughput is constrained by the slowest repeatedly used stage and by pipeline utilization. Adding participants increases the candidate resource pool but does not necessarily reduce latency or increase useful capacity. Important scaling limits include: * activation size and the number of wide-area stage boundaries; * model-artifact loading and warm-up time; * KV-cache growth and recovery cost; * stragglers and correlated failures; * discovery and resource-advertisement churn; * verification and redundant-execution overhead; * settlement event volume; and * concentration around nodes with rare artifacts, high bandwidth, or strong reputation. Every transport binding needs congestion control appropriate to the selected transport. Deadline pressure does not justify unbounded retransmission, duplicate execution traffic, or activation bursts. Path repair and redundant verification need to share congestion and backpressure information with ordinary inference traffic so that recovery from one failure does not cause a wider overload. Wang, et al. Expires 28 February 2027 [Page 24] Internet-Draft ODSI August 2026 Research systems have demonstrated collaborative inference across multiple parties [Petals] and adaptive pipelines over heterogeneous unreliable devices [SWARM]. These results motivate the architecture but do not establish performance for public-Internet deployment or adversary model. An ODSI evaluation should report model and execution profiles, participant count and diversity, network distributions, activation and state volume, time to first token, time per output token, throughput, tail latency, repair time, verification cost, false decisions, and behavior under churn and adversarial actions. Comparisons should include a provider-controlled distributed baseline. Claims of scalability, bounded recovery, or cost reduction need to identify the measured operating region. 12. Manageability and Operational Considerations Even without one global operator, each participant needs local management and the overlay needs interoperable observations. Implementations should expose privacy-preserving counters and failure reasons for: * manifest, artifact, and profile validation; * peer discovery and resource-profile freshness; * assignment acceptance, rejection, cancellation, and retry; * compute, queue, transfer, state, and verification duration; * state creation, replication, migration, expiration, and loss; * receipt-chain gaps, equivocation, and verification outcomes; and * settlement submission and finality state when enabled. Failure reports need stable categories so that a path constructor can distinguish incompatibility, overload, policy rejection, missing artifact, missing state, transport failure, invalid predecessor, and verification failure. Detailed diagnostics can reveal topology, capacity, model holdings, or user behavior and therefore require access control and retention limits. Wang, et al. Expires 28 February 2027 [Page 25] Internet-Draft ODSI August 2026 Participants need procedures for key compromise, vulnerable artifacts, verifier failure, profile rollback, and emergency manifest revocation. Because coordination is decentralized, revocation propagation can be delayed or partitioned. Clients should treat the freshness of governance and revocation state as part of path acceptance. 13. Security Considerations This architecture assumes hostile participants and networks. A concrete protocol specification is expected to follow the threat- analysis guidance in [RFC3552] and to state how it addresses the threats below. 13.1. Identity and Message Security Self-generated keys provide stable pseudonyms, not real-world identity, uniqueness, authorization, or trust. Protocol bindings need mutual authentication where required, integrity protection, replay protection, context binding, algorithm agility, key rotation, and revocation. Every signed object needs a domain separator and version so that a valid signature in one context cannot authorize another action. Traffic between execution nodes can contain prompts, activations, state, results, and receipts. It requires confidentiality and integrity protection against on-path attackers unless a deployment explicitly accepts exposure. Transport security does not protect data from an execution node that legitimately receives plaintext. 13.2. Manifest and Software Supply Chain A malicious or compromised manifest publisher can distribute backdoored weights, unsafe runtime code, incompatible tokenizers, or weakened verification settings. Artifact digests only preserve the published bytes. Profiles need publisher authorization, review or transparency policy, rollback protection, revocation, dependency validation, and isolation of untrusted runtime components. Content-addressed retrieval can expose which models a participant supports. An attacker can withhold rare artifacts, advertise unavailable pieces, or flood storage with low-value artifacts. Replication policy, quotas, proof of possession where appropriate, and retrieval diversity reduce these risks. Wang, et al. Expires 28 February 2027 [Page 26] Internet-Draft ODSI August 2026 13.3. Resource Discovery and Path Manipulation Attackers can create Sybil identities, advertise false performance, replay fresh-looking profiles, eclipse clients, bias peer sampling, or attract sensitive requests before failing or observing them. Signed advertisements do not establish truth. Profiles need freshness limits, rate controls, independent observations, peer diversity, and resistance to a single discovery source. Path constructors can censor participants, leak request policy, select colluding verifiers, or create paths that concentrate information. Clients should be able to validate path constraints and, where the threat model requires it, obtain paths from independent sources or construct paths locally. 13.4. Execution Integrity An execution node can use wrong artifacts or state, skip computation, fabricate output, return stale output, equivocate, or selectively corrupt requests. Commitments and receipts support attribution but do not prove computation. The VERIFICATION PROFILE is the primary mitigation and needs to address numerical nondeterminism, verifier collusion, adaptive cheating, and false accusation. Redundant execution fails when replicas collude or share a common faulty implementation. Trusted execution can fail through compromised hardware, stale attestation, side channels, or vendor control. Proof systems can contain implementation errors or bind to the wrong model. The profile needs operational response for failure of its verification root. 13.5. State and Replay Attacks State can be rolled back, substituted between users, replayed across attempts, or retained after expiration. Request-, manifest-, profile-, position-, and predecessor-binding prevent many substitutions. Concurrent retries need fencing so that two valid branches do not both become authoritative. State needs access control, integrity protection, lifecycle enforcement, and secure deletion appropriate to the deployment. Wang, et al. Expires 28 February 2027 [Page 27] Internet-Draft ODSI August 2026 13.6. Denial of Service Clients can submit long contexts, request unsupported profiles, repeatedly trigger repair, or create expensive disputes. Participants can accept work and withhold results, overload successors with large activations, or flood discovery and verification systems. Implementations need admission control, authenticated quotas, bounded parsing and allocation, request-size limits, cancellation, backpressure, and separation between untrusted input processing and scarce accelerator resources. Settlement can amplify denial of service if every event requires global consensus. Aggregation, channels, batching, and minimum-value policies can reduce event volume, but their trust and failure behavior must be documented. 13.7. Economic and Governance Attacks Stake and reputation introduce bribery, collusion, self-dealing, identity farming, verifier capture, censorship, and concentration risks. A party can behave honestly while building reputation and cheat only on selected valuable requests. Governance participants can authorize a malicious model or verification downgrade. Economic parameters and governance are therefore security-critical configuration, not merely business policy. Penalizing ambiguous failures creates an attack in which an adversary delays traffic or supplies inconsistent observations to cause loss to an honest node. Punitive consequences require stronger evidence than non-payment or path-selection decisions. 13.8. Residual Risk No base ODSI mechanism guarantees correct, confidential, available, or timely inference. Those properties depend on the selected deployment profile and its assumptions. Applications with safety- critical consequences should not act on optimistic results before the verification required by their risk model completes. 14. Privacy Considerations ODSI increases the number of administrative entities that can observe parts of an inference request. Prompts, tokens, activations, logits, embeddings, KV caches, model-selection information, timing, path structure, payment records, and resource advertisements can reveal user content or behavior. Wang, et al. Expires 28 February 2027 [Page 28] Internet-Draft ODSI August 2026 Splitting a model does not provide a general confidentiality guarantee. Intermediate representations can retain information about inputs, and a participant can combine observations across requests. Likewise, distributing execution closer to a user does not improve privacy if more independent parties receive the data. A deployment profile needs to specify: * which roles receive plaintext input, activations, state, and output; * end-to-end and hop-by-hop encryption boundaries; * participant-selection and jurisdiction constraints; * unlinkability or pseudonym-rotation behavior and its interaction with reputation; * logging, evidence, and settlement retention; * state isolation and deletion; and * whether verification reveals inputs or intermediate values. Data minimization can favor larger contiguous execution units so fewer parties observe intermediate values. Trusted execution, secure multiparty computation, homomorphic techniques, or zero-knowledge proofs can change exposure, but each adds assumptions and cost and is not provided by this architecture. Public settlement records can permanently link participants, clients, timing, and workload volume. Profiles should avoid placing request content or stable user identifiers in globally replicated records and should disclose unavoidable metadata leakage. 15. IANA Considerations This document has no IANA actions. 16. Relationship to Existing Work IDN [I-D.li-cats-idn] distributes and selects inference capabilities within one provider-controlled domain. ODSI uses a different trust model and complements rather than replaces IDN. An IDN deployment can contribute an endpoint to ODSI, while remaining internally provider controlled. Wang, et al. Expires 28 February 2027 [Page 29] Internet-Draft ODSI August 2026 CATS [I-D.ietf-cats-framework] combines network and compute information when steering traffic to service instances. ODSI can use CATS inside a participant domain but keeps cross-domain peer and execution coordination at the application layer. Peer-to-peer systems demonstrate content-addressed distribution and incentive mechanisms [BitTorrent]. Decentralized ledgers demonstrate replicated settlement under explicit consensus assumptions [Bitcoin]. ODSI borrows architectural lessons but does not require either protocol. Collaborative model-serving systems demonstrate that multiple parties can collectively host large models [Petals]. Work on unreliable heterogeneous pipelines explores adaptive placement under churn [SWARM]. ODSI focuses on the Internet architecture, trust boundaries, interoperable objects, and evidence requirements that arise when such participants do not share one operator. 17. Acknowledgments The authors would like to thank colleagues and reviewers in the community who provided feedback on the early version of this draft. 18. Informative References [Bitcoin] Nakamoto, S., "Bitcoin: A Peer-to-Peer Electronic Cash System", 2008, . [BitTorrent] Cohen, B., "Incentives Build Robustness in BitTorrent", Workshop on Economics of Peer-to-Peer Systems, May 2003, . [Byzantine] Castro, M. and B. Liskov, "Practical Byzantine Fault Tolerance", Third Symposium on Operating Systems Design and Implementation, February 1999, . [FLP] Fischer, M. J., Lynch, N. A., and M. S. Paterson, "Impossibility of Distributed Consensus with One Faulty Process", Journal of the ACM, Volume 32, Issue 2, April 1985, . [I-D.ietf-cats-framework] Li, C., Du, Z., Boucadair, M., Contreras, L. M., and J. Drake, "A Framework for Computing-Aware Traffic Steering Wang, et al. Expires 28 February 2027 [Page 30] Internet-Draft ODSI August 2026 (CATS)", Work in Progress, Internet-Draft, draft-ietf- cats-framework-24, 2 April 2026, . [I-D.ietf-cats-metric-definition] Yao, K., Li, C., Contreras, L. M., Ros-Giralt, J., and G. Zeng, "CATS Metrics Definition", Work in Progress, Internet-Draft, draft-ietf-cats-metric-definition-10, 22 June 2026, . [I-D.ietf-cats-usecases-requirements] Yao, K., Contreras, L. M., Shi, H., Zhang, S., and Q. An, "Computing-Aware Traffic Steering (CATS) Problem Statement, Use Cases, and Requirements", Work in Progress, Internet-Draft, draft-ietf-cats-usecases-requirements-14, 2 February 2026, . [I-D.li-cats-idn] Li, Q., Wang, H., Jiang, Y., Xu, M., and G. Muntean, "A Framework of Intelligence Delivery Network (IDN) for Deep Learning Inference", Work in Progress, Internet-Draft, draft-li-cats-idn-01, 26 August 2026, . [Petals] Borzunov, A., Baranchuk, D., Dettmers, T., Ryabinin, M., Belkada, Y., Chumachenko, A., Samygin, P., and C. Raffel, "Petals: Collaborative Inference and Fine-tuning of Large Models", Proceedings of ACL 2023: System Demonstrations, pp. 558-568, July 2023, . [RFC3552] Rescorla, E. and B. Korver, "Guidelines for Writing RFC Text on Security Considerations", BCP 72, RFC 3552, DOI 10.17487/RFC3552, July 2003, . [RFC9518] Nottingham, M., "Centralization, Decentralization, and Internet Standards", RFC 9518, DOI 10.17487/RFC9518, December 2023, . Wang, et al. Expires 28 February 2027 [Page 31] Internet-Draft ODSI August 2026 [SWARM] Ryabinin, M., Dettmers, T., Diskin, M., and A. Borzunov, "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient", Proceedings of the 40th International Conference on Machine Learning, PMLR 202, July 2023, . Authors' Addresses Hanling Wang Pengcheng Laboratory Email: wanghl03@pcl.ac.cn Qing Li Pengcheng Laboratory Email: liq@pcl.ac.cn Yong Jiang Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory Email: jiangy@sz.tsinghua.edu.cn Mingwei Xu Tsinghua University Email: xumw@tsinghua.edu.cn Gabriel-Miro Muntean Dublin City University Email: gabriel.muntean@dcu.ie Wang, et al. Expires 28 February 2027 [Page 32]