Network Working Group D. O. Watts Internet-Draft GoodShyt Group Inc. Intended status: Informational 21 October 2026 Expires: 24 April 2027 Protocol Conformance Test Evidence Records: A Reproducibility Profile draft-watts-protocol-test-evidence-profile-00 Abstract This document proposes a narrow record profile for describing software protocol-conformance test runs, including the tested artifact, test suite, implementation, execution environment, individual case outcomes, digests, and known limitations. It does not define a new signature scheme, attestation architecture, transparency service, or general-purpose test-output language. Implementations are encouraged to reuse existing mechanisms such as SCITT statements, RATS terminology, COSE, JSON canonicalization, and existing test-report formats where suitable. This individual draft candidate has not been submitted to or endorsed by the IETF. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 24 April 2027. Copyright Notice Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights Watts Expires 24 April 2027 [Page 1] Internet-Draft Protocol Test Evidence Profile October 2026 and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Conventions and Terminology . . . . . . . . . . . . . . . . . 3 3. Scope and Non-Goals . . . . . . . . . . . . . . . . . . . . . 3 4. Evidence Record Model . . . . . . . . . . . . . . . . . . . . 3 4.1. Subject and Test Suite . . . . . . . . . . . . . . . . . 4 4.2. Environment and Run . . . . . . . . . . . . . . . . . . . 4 4.3. Case-Level Outcomes . . . . . . . . . . . . . . . . . . . 4 4.4. Provenance and Limitations . . . . . . . . . . . . . . . 4 5. Canonicalization and Integrity . . . . . . . . . . . . . . . 4 6. Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 7. Interoperability Considerations . . . . . . . . . . . . . . . 5 8. Security Considerations . . . . . . . . . . . . . . . . . . . 5 9. Privacy Considerations . . . . . . . . . . . . . . . . . . . 6 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 11. Status and Standards-Track Caveat . . . . . . . . . . . . . . 6 12. Normative References . . . . . . . . . . . . . . . . . . . . 6 13. Informative References . . . . . . . . . . . . . . . . . . . 6 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 7 1. Introduction Software protocol implementations are evaluated by test harnesses, continuous-integration jobs, conformance suites, and interoperability labs. Reports can be difficult to compare when they omit artifact digests, test-suite versions, runtime environments, case-level status, or provenance linking results to source inputs. This document defines a small, extensible JSON record profile for protocol-conformance test evidence. A record states what its producer asserts happened during a run. It is not proof that the harness was trustworthy, the test suite complete, or the implementation secure. Existing work covers substantial adjacent requirements. SCITT defines transparency for signed statements; RATS defines evidence- appraisal roles and terminology; COSE and JSON Canonicalization Scheme provide existing security/canonicalization building blocks; and in-toto supports authenticated metadata with type-specific predicates. Implementers SHOULD first evaluate whether existing formats meet their needs. Watts Expires 24 April 2027 [Page 2] Internet-Draft Protocol Test Evidence Profile October 2026 2. Conventions and Terminology The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL are to be interpreted as described in BCP 14 (RFC 2119 and RFC 8174) when, and only when, they appear in all capitals. Evidence record A structured statement describing a test run and its reported outcomes. Subject artifact The implementation, binary, source snapshot, endpoint, or object evaluated by the test run. Test suite The named and versioned collection of cases executed by a harness. Producer The system that creates the record. Consumer A system or person that parses and evaluates the record. 3. Scope and Non-Goals The profile describes test-run inputs, environment, case-level outcomes, artifact digests, provenance, and limitations. This document does not specify test design, pass/fail computation, key provisioning, signing, transparency receipts, or trust decisions. Those functions SHOULD use established mechanisms. This document does not establish a new evidence ledger. Deployments requiring transparency SHOULD evaluate SCITT (RFC 9943); deployments requiring remote-attestation semantics SHOULD evaluate RATS (RFC 9334) where applicable. 4. Evidence Record Model A record is a UTF-8 JSON object. Producers MUST include schema_version, record_id, subject, test_suite, implementation, environment, run, cases, result_summary, and provenance. Consumers SHOULD preserve unknown extension members when transforming or re- emitting records. Watts Expires 24 April 2027 [Page 3] Internet-Draft Protocol Test Evidence Profile October 2026 4.1. Subject and Test Suite The subject SHOULD identify the artifact by stable name and cryptographic digest. The test_suite SHOULD identify suite name, version, and digest. Digests MUST identify exact bytes or a clearly specified canonical representation; a label alone is not a stable artifact identifier. 4.2. Environment and Run The environment SHOULD identify relevant runtime, operating system, architecture, dependencies, and hardware. The run MUST include status and SHOULD include timestamps, exit code, case count, and assertion count when available. Missing values MUST NOT be fabricated. 4.3. Case-Level Outcomes Each case MUST have a stable case_id and status of pass, fail, skip, error, or not-run. Expected and observed values MAY be included. Logs SHOULD be referenced by digest or controlled URI rather than embedded when they may contain secrets or personal data. 4.4. Provenance and Limitations The provenance member SHOULD link source revisions, test-harness digests, build identifiers, and producer identity. A limitations array is RECOMMENDED and SHOULD describe untested platforms, incomplete coverage, simulated inputs, and other material constraints. 5. Canonicalization and Integrity JSON does not inherently provide a unique byte representation for semantically equivalent objects. If JSON is signed or hashed, implementations SHOULD use an established canonicalization procedure such as RFC 8785 and MUST document which bytes are covered. This profile defines no signature format. Signed statements SHOULD reuse an established mechanism such as COSE (RFC 9052) or an existing attestation envelope. Where transparency is required, deployments SHOULD evaluate SCITT (RFC 9943). A signature authenticates an assertion under the selected trust model; it does not prove the assertion true. Watts Expires 24 April 2027 [Page 4] Internet-Draft Protocol Test Evidence Profile October 2026 6. Example The following example is illustrative, unsigned, and has no transparency receipt. { "schema_version": "0.1.0-draft", "record_id": "RUN-EXAMPLE-001", "subject": {"name": "example-implementation", "digest_algorithm": "sha-256", "digest": ""}, "test_suite": {"name": "example-suite", "version": "1.0", "digest_algorithm": "sha-256", "digest": ""}, "implementation": {"name": "example-implementation", "version": "1.0"}, "environment": {"runtime": "example-runtime", "platform": "example-platform"}, "run": {"status": "passed", "case_count": 2, "assertion_count": 4, "exit_code": 0}, "cases": [{"case_id": "CASE-001", "status": "pass"}, {"case_id": "CASE-002", "status": "pass"}], "result_summary": {"passed": 2, "failed": 0, "skipped": 0}, "provenance": {"producer": "example-harness", "source_revision": ""}, "limitations": ["Illustrative example; not a measurement"] } 7. Interoperability Considerations Producers and consumers SHOULD test this profile against existing test-report formats and attestation predicates before adoption. Consumers MUST distinguish an unknown extension from a verified security property. Conformance vectors SHOULD cover pass, fail, skip, error, not-run, missing digest, and incomplete provenance. Before standardization, compare the proposed field set with in-toto predicates, TAP, JUnit-style reports, and SCITT statement payloads. If an existing format expresses the required information without ambiguity, reusing it is preferred. 8. Security Considerations An evidence record is an assertion by its producer. A compromised or dishonest producer can report false results, omit failures, or associate results with the wrong artifact. Consumers SHOULD verify digests independently, authenticate the producer where needed, and apply local trust policy. Records can be replayed or presented out of context. Deployments SHOULD use unique run identifiers, trustworthy timestamps where available, immutable source revisions, and transparency/attestation mechanisms where justified. A digest alone does not establish freshness, authorship, correctness, or completeness. Watts Expires 24 April 2027 [Page 5] Internet-Draft Protocol Test Evidence Profile October 2026 Logs and environment metadata can reveal secrets, hostnames, tokens, identifiers, vulnerabilities, or confidential details. Producers MUST NOT include secrets in public records and SHOULD minimize sensitive fields. This profile does not guarantee post-quantum security. 9. Privacy Considerations Environment metadata may include personal or indirectly identifying information. Producers SHOULD collect only what is necessary, redact direct identifiers, and review log references before publication. Hashing a low-entropy identifier does not necessarily anonymize it. 10. IANA Considerations This document requests no IANA actions. 11. Status and Standards-Track Caveat This individual draft candidate has not been submitted to the IETF, has no assigned Datatracker identifier, and represents no IETF consensus or endorsement. The prior-art review is preliminary. SCITT (RFC 9943), RATS (RFC 9334), in-toto attestations, and existing test-report formats overlap with parts of this proposal. Community feedback and an interoperability prototype are prerequisites for deciding whether a separate profile is useful. 12. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", RFC 8174, May 2017, . [RFC8259] Bray, T., "The JavaScript Object Notation (JSON) Data Interchange Format", RFC 8259, December 2017, . [RFC8785] Rundgren, A., Jordan, B., and S. Erdtman, "JSON Canonicalization Scheme (JCS)", RFC 8785, June 2020, . 13. Informative References Watts Expires 24 April 2027 [Page 6] Internet-Draft Protocol Test Evidence Profile October 2026 [RFC9334] Birkholz, H., Thaler, D., Richardson, M., Smith, N., and W. Pan, "Remote ATtestation procedureS (RATS) Architecture", RFC 9334, January 2023, . [RFC9943] Birkholz, H., Delignat-Lavaud, A., Fournet, C., Deshpande, Y., and S. Lasker, "An Architecture for Trustworthy and Transparent Digital Supply Chains", RFC 9943, June 2026, . [RFC9052] Schaad, J., "CBOR Object Signing and Encryption (COSE): Structures and Process", RFC 9052, August 2022, . [INTOTO] Maintainers, I., "in-toto Attestation Framework Specification", . [TAP14] Schlueter, I. Z., "Test Anything Protocol Version 14 Specification", . [XML2RFC] Hoffman, P., "The xml2rfc Version 3 Vocabulary", RFC 7991, December 2016, . Author's Address Deonté O’dell Watts GoodShyt Group Inc. Email: deonte@goodshyt.fun URI: https://orcid.org/0009-0005-8586-3650 Watts Expires 24 April 2027 [Page 7]