Protocol Conformance Test Evidence Records: A Reproducibility Profile

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 24 April 2027.¶

Copyright Notice

Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

Software protocol implementations are evaluated by test harnesses, continuous-integration jobs, conformance suites, and interoperability labs. Reports can be difficult to compare when they omit artifact digests, test-suite versions, runtime environments, case-level status, or provenance linking results to source inputs.¶

This document defines a small, extensible JSON record profile for protocol-conformance test evidence. A record states what its producer asserts happened during a run. It is not proof that the harness was trustworthy, the test suite complete, or the implementation secure.¶

Existing work covers substantial adjacent requirements. SCITT defines transparency for signed statements; RATS defines evidence-appraisal roles and terminology; COSE and JSON Canonicalization Scheme provide existing security/canonicalization building blocks; and in-toto supports authenticated metadata with type-specific predicates. Implementers SHOULD first evaluate whether existing formats meet their needs.¶

2. Conventions and Terminology

The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL are to be interpreted as described in BCP 14 (RFC 2119 and RFC 8174) when, and only when, they appear in all capitals.¶

Evidence record

A structured statement describing a test run and its reported outcomes.¶

Subject artifact

The implementation, binary, source snapshot, endpoint, or object evaluated by the test run.¶

Test suite

The named and versioned collection of cases executed by a harness.¶

Producer

The system that creates the record.¶

Consumer

A system or person that parses and evaluates the record.¶

3. Scope and Non-Goals

The profile describes test-run inputs, environment, case-level outcomes, artifact digests, provenance, and limitations.¶

This document does not specify test design, pass/fail computation, key provisioning, signing, transparency receipts, or trust decisions. Those functions SHOULD use established mechanisms.¶

This document does not establish a new evidence ledger. Deployments requiring transparency SHOULD evaluate SCITT (RFC 9943); deployments requiring remote-attestation semantics SHOULD evaluate RATS (RFC 9334) where applicable.¶

4. Evidence Record Model

A record is a UTF-8 JSON object. Producers MUST include schema_version, record_id, subject, test_suite, implementation, environment, run, cases, result_summary, and provenance. Consumers SHOULD preserve unknown extension members when transforming or re-emitting records.¶

5. Canonicalization and Integrity

JSON does not inherently provide a unique byte representation for semantically equivalent objects. If JSON is signed or hashed, implementations SHOULD use an established canonicalization procedure such as RFC 8785 and MUST document which bytes are covered.¶

This profile defines no signature format. Signed statements SHOULD reuse an established mechanism such as COSE (RFC 9052) or an existing attestation envelope. Where transparency is required, deployments SHOULD evaluate SCITT (RFC 9943). A signature authenticates an assertion under the selected trust model; it does not prove the assertion true.¶

6. Example

The following example is illustrative, unsigned, and has no transparency receipt.¶

{
  "schema_version": "0.1.0-draft",
  "record_id": "RUN-EXAMPLE-001",
  "subject": {"name": "example-implementation", "digest_algorithm": "sha-256", "digest": "<artifact-digest>"},
  "test_suite": {"name": "example-suite", "version": "1.0", "digest_algorithm": "sha-256", "digest": "<suite-digest>"},
  "implementation": {"name": "example-implementation", "version": "1.0"},
  "environment": {"runtime": "example-runtime", "platform": "example-platform"},
  "run": {"status": "passed", "case_count": 2, "assertion_count": 4, "exit_code": 0},
  "cases": [{"case_id": "CASE-001", "status": "pass"}, {"case_id": "CASE-002", "status": "pass"}],
  "result_summary": {"passed": 2, "failed": 0, "skipped": 0},
  "provenance": {"producer": "example-harness", "source_revision": "<revision-id>"},
  "limitations": ["Illustrative example; not a measurement"]
}

7. Interoperability Considerations

Producers and consumers SHOULD test this profile against existing test-report formats and attestation predicates before adoption. Consumers MUST distinguish an unknown extension from a verified security property. Conformance vectors SHOULD cover pass, fail, skip, error, not-run, missing digest, and incomplete provenance.¶

Before standardization, compare the proposed field set with in-toto predicates, TAP, JUnit-style reports, and SCITT statement payloads. If an existing format expresses the required information without ambiguity, reusing it is preferred.¶

8. Security Considerations

An evidence record is an assertion by its producer. A compromised or dishonest producer can report false results, omit failures, or associate results with the wrong artifact. Consumers SHOULD verify digests independently, authenticate the producer where needed, and apply local trust policy.¶

Records can be replayed or presented out of context. Deployments SHOULD use unique run identifiers, trustworthy timestamps where available, immutable source revisions, and transparency/attestation mechanisms where justified. A digest alone does not establish freshness, authorship, correctness, or completeness.¶

Logs and environment metadata can reveal secrets, hostnames, tokens, identifiers, vulnerabilities, or confidential details. Producers MUST NOT include secrets in public records and SHOULD minimize sensitive fields. This profile does not guarantee post-quantum security.¶

9. Privacy Considerations

Environment metadata may include personal or indirectly identifying information. Producers SHOULD collect only what is necessary, redact direct identifiers, and review log references before publication. Hashing a low-entropy identifier does not necessarily anonymize it.¶

10. IANA Considerations

This document requests no IANA actions.¶

11. Status and Standards-Track Caveat

This individual draft candidate has not been submitted to the IETF, has no assigned Datatracker identifier, and represents no IETF consensus or endorsement. The prior-art review is preliminary. SCITT (RFC 9943), RATS (RFC 9334), in-toto attestations, and existing test-report formats overlap with parts of this proposal. Community feedback and an interoperability prototype are prerequisites for deciding whether a separate profile is useful.¶

Author's Address