Benchmarking Methodology for ASPA Verification

5.2.1. ASPA Dataset Size and Structure

The ASPA dataset provided to the DUT is characterized by the number of CAS entries and the associated provider authorizations.¶

At minimum, the following parameters MUST be reported:
- ASPA Record Count: The number of distinct CAS entries in the ASPA dataset.
- Provider Authorization Count: The total number of distinct CAS-to-PAS authorization relationships.
- Provider-Set Cardinality: The number of authorized providers associated with each CAS.¶

These parameters MUST be distinguished because ASPA datasets containing the same number of CAS entries may differ significantly in the number and distribution of provider authorizations. For a dataset containing M CAS entries, the provider authorization count can be expressed as:¶

P = sum(|S_i|), for i = 1, ..., M

where S_i denotes the set of distinct authorized provider ASes for CAS i, excluding the special AS 0 value.¶

An ASPA entry containing only AS 0 represents an explicit statement that the CAS has no transit providers, rather than an authorization of AS 0 as a provider. Such entries MUST be distinguished from CAS entries for which no ASPA information is available.¶

When varying ASPA dataset size, the tester SHOULD distinguish between increasing the number of CAS entries and increasing the number of provider authorizations per CAS.¶

The provider-set cardinality distribution SHOULD be documented using statistical characteristics such as minimum, maximum, mean, and median values.¶

For tests intended to isolate the effect of provider-set cardinality, the number of CAS entries and other relevant workload characteristics SHOULD remain fixed.¶

The tester MAY use datasets derived from operational ASPA information or synthetically generated datasets. Synthetic datasets SHOULD be used when precise control of dataset characteristics is required.¶

5.2.2. ASPA Coverage

ASPA coverage characterizes the availability of provider authorization information for ASes involved in the verification workload.¶

For benchmarking purposes, ASPA coverage is defined relative to a specified reference AS population derived from the BGP AS_PATHs and verification procedures used in the test.¶

Let A_W denote the set of distinct ASNs that may be used as the customer argument in provider authorization checks for the specified workload and verification directions.¶

Let A_D denote the set of CASes for which ASPA information is available in the dataset. ASPA coverage is defined as:¶

C_ASPA = (|A_W intersect A_D| / |A_W|) * 100%

The reference population A_W MUST remain unchanged when comparing different ASPA coverage levels for the same workload. ASPA records unrelated to the reference population MUST NOT contribute to the coverage numerator. If the reference population is empty, ASPA coverage is undefined and MUST be reported as not applicable.¶

The tester SHOULD evaluate multiple ASPA coverage levels to characterize how verification performance changes with the availability of authorization information. Illustrative coverage levels include 0%, 25%, 50%, 75%, and 100%. Other levels MAY be used depending on the test objective. When constructing datasets with different coverage levels, the selection of covered CASes MUST be reproducible. Changing ASPA coverage may change the expected verification results of the test routes. Therefore, the distribution of Valid, Invalid, and Unknown results MUST be determined and reported for each coverage level. An ASPA entry containing only AS 0 counts as an available ASPA entry for coverage purposes. It MUST NOT be treated as equivalent to the absence of ASPA information.¶

5.2.3. AS_PATH Length and Structure

AS_PATH length is a primary characteristic of ASPA verification workloads.¶

The ASPA verification procedures operate on the COMPRESSED_AS_PATH, obtained by removing consecutive duplicate ASNs from the AS_PATH.¶

Therefore, the benchmarking workload MUST distinguish between:
- Raw AS_PATH Length: The number of ASNs in the original AS_PATH.
- Compressed AS_PATH Length: The number of ASNs remaining after consecutive duplicate ASNs have been removed.¶

Unless otherwise specified, AS_PATH length in the benchmarking procedures refers to compressed AS_PATH length. The tester SHOULD evaluate multiple compressed AS_PATH lengths to characterize their impact on verification performance. For example, compressed path lengths of 2, 4, 8, 16, and 32 ASNs MAY be used as controlled test values. These values are illustrative rather than mandatory. When evaluating AS_PATH length sensitivity, the tester SHOULD maintain consistent ASPA dataset characteristics, verification direction, and verification-result distribution where feasible.¶

In addition to path length, the structure of provider authorization relationships along the AS_PATH may influence verification processing. The ASPA verification procedures distinguish between an up-ramp and a down-ramp based on the authorization relationships between consecutive ASes. Therefore, test workloads SHOULD include different authorization relationship patterns consistent with the applicable verification procedures.¶

For tests intended to isolate the effect of AS_PATH prepending, the tester SHOULD construct paths with different raw AS_PATH lengths but identical compressed AS_PATHs. This allows the processing overhead associated with AS_PATH representation and compression to be distinguished from the effects of compressed path length. The raw and compressed AS_PATH length distributions MUST be reported for each applicable test.¶

5.2.4. Provider Authorization Patterns

ASPA verification evaluates the authorization relationships between ASes appearing in an AS_PATH. The provider authorization function defined in [I-D.ietf-sidrops-aspa-verification] produces three possible results: - Provider+. - Not Provider+. - No Attestation.¶

The distribution and position of these results along an AS_PATH may influence the verification outcome and the processing required by an implementation. Therefore, the benchmarking methodology SHOULD consider the following authorization patterns:
- Fully Authorized Paths: Paths for which the relevant provider authorization checks return Provider+ and the complete verification procedure produces Valid.
- Paths with Missing Attestations: Paths containing authorization checks that return No Attestation, potentially resulting in an Unknown verification outcome.
- Paths with Unauthorized Relationships: Paths containing authorization checks that return Not Provider+, potentially resulting in an Invalid verification outcome.¶

The terms above describe workload characteristics and do not replace the verification results defined in [I-D.ietf-sidrops-aspa-verification]. The tester SHOULD construct workloads in which the positions of missing attestations and unauthorized relationships are controlled. For example, a test MAY place the first relevant No Attestation or Not Provider+ result at different positions along an AS_PATH while keeping the compressed path length unchanged.¶

The position MUST be described relative to the path orientation and authorization checks defined by the applicable upstream or downstream verification procedure. Such tests may help characterize the sensitivity of an implementation to authorization relationship patterns and early termination opportunities.¶

However, the final verification result MUST always be determined using the complete ASPA verification procedure. A single No Attestation or Not Provider+ result MUST NOT be assumed to produce a particular final verification result independently of the remaining path structure.¶

5.2.5. Verification Direction and Outcome Distribution

ASPA verification applies different procedures depending on the relationship between the receiving router and its BGP neighbor.
The benchmarking methodology distinguishes between:
- Upstream Verification: Applied to routes received from Customers or Peers, and to the applicable Route Server and RS-client cases.
- Downstream Verification: Applied to routes received from Providers.¶

The tester SHOULD evaluate upstream and downstream verification separately before introducing workloads involving multiple verification directions.
The configured neighbor relationship and the applied verification procedure MUST be reported.
For each test workload, the expected ASPA verification results MUST be determined according to [I-D.ietf-sidrops-aspa-verification]. The verification outcome distribution describes the proportion of routes expected to produce each result:¶

D=(p_{\mathrm{Valid}},p_{\mathrm{Invalid}},p_{\mathrm{Unknown}})¶

where: p_{\mathrm{Valid}}+p_{\mathrm{Invalid}}+p_{\mathrm{Unknown}}=1¶

The tester SHOULD include workloads dominated by individual verification results, as well as mixed workloads containing specified proportions of Valid, Invalid, and Unknown routes.
The verification outcome distribution MUST be reported for each test.
Where relevant, the outcome distribution across distinct AS_PATHs SHOULD also be reported separately from the route-weighted distribution.
Not all combinations of ASPA coverage, AS_PATH structure, verification direction, and desired outcome distribution are necessarily achievable.
The tester MUST ensure that the generated workload is consistent with the applicable ASPA verification procedures and MUST document any constraints that prevent the construction of a requested workload.¶

5.2.6. BGP Route and Unique AS_PATH Populations

The number of BGP routes and the number of distinct AS_PATHs are separate workload parameters. Multiple routes may share the same AS_PATH, while other workloads may contain a large number of distinct AS_PATHs. The benchmarking methodology MUST distinguish between: - Total BGP route count.
- Unique AS_PATH count.
- Number of routes sharing each AS_PATH.
- Number of distinct ASNs represented in the workload.¶

These characteristics may influence the processing and memory requirements of ASPA verification implementations. For example, an implementation may reuse previously computed verification results or maintain shared representations for routes with identical AS_PATHs. The benchmarking methodology does not prescribe such optimizations. However, workloads SHOULD be constructed to allow their performance effects to be observed. When evaluating the effect of unique AS_PATH count, the total number of BGP routes SHOULD remain fixed where feasible. Conversely, when evaluating the effect of BGP route count, the number and characteristics of distinct AS_PATHs SHOULD remain fixed where feasible. The tester MUST specify whether AS_PATH uniqueness is determined using raw or compressed AS_PATHs.¶

5.2.7. ASPA Update Characteristics

ASPA updates may modify provider authorization information without changing the BGP routes currently stored by the DUT. The benchmarking methodology distinguishes three ASPA update operations:
- Addition: Introduction of an ASPA record for a previously uncovered CAS.
- Replacement: Modification of the complete provider set associated with an existing CAS.
- Withdrawal: Removal of an existing ASPA record for a CAS.
The update workload MUST conform to the applicable RTR protocol semantics.¶

For each update transaction, the following characteristics SHOULD be controlled and reported:
- Number of changed CAS entries.
- Number of provider authorizations added or removed.
- Number of ASPA records announced or withdrawn.
- Update transaction size.
- Update transaction frequency.
- Distribution of updates across CAS entries.¶

When evaluating ASPA churn, the tester MUST specify whether the update rate refers to changed CAS records, individual provider authorization changes, or complete RTR update transactions. These measures MUST NOT be treated as interchangeable. The tester SHOULD evaluate both individual ASPA updates and batches containing multiple changes. For revalidation-related benchmarks, the tester SHOULD also distinguish: - The number of existing routes containing ASes relevant to the modified ASPA information.
- The number of routes whose expected verification results change.
- The number of distinct AS_PATHs whose expected verification results change.¶

These quantities may differ because an ASPA update can affect multiple AS_PATHs, and multiple routes may share the same AS_PATH. The affected route population MUST be determined using the expected verification results before and after the ASPA update.¶

5.2.8. BGP Update Characteristics

The BGP update workload determines the rate and pattern at which routes requiring ASPA verification are presented to the DUT. The benchmarking methodology SHOULD control the following parameters:
- Number of BGP announcements and withdrawals.
- BGP UPDATE message rate.
- Route announcement and withdrawal rate.
- Number of distinct AS_PATHs introduced per unit time.
- Distribution of verification results among updated routes.
- Burst size and inter-arrival intervals.¶

The number of BGP UPDATE messages and the number of routes represented by those messages MUST be distinguished when calculating processing rates. The tester SHOULD evaluate both constant-rate and bursty BGP update workloads. For concurrent ASPA and BGP update tests, the timing relationship between the two update sources MUST be specified. The RTR and BGP traffic generators SHOULD provide synchronized or otherwise correlated timestamps sufficient to reproduce the intended update sequence.¶

5.3.1. Static Verification Workloads

Static verification workloads evaluate AS_PATH verification performance while the ASPA dataset remains unchanged. The tester first installs a predefined ASPA dataset and allows the DUT to reach steady state. BGP routes are then introduced with controlled AS_PATH characteristics and expected verification results. These workloads are intended to characterize the performance effects of:
- Compressed AS_PATH length.
- ASPA coverage.
- Provider-set cardinality.
- Provider authorization patterns.
- Upstream and downstream verification procedures.
- Verification outcome distributions.
- Unique AS_PATH count and route sharing.¶

When evaluating an individual parameter, the tester SHOULD maintain the remaining workload characteristics as consistently as possible. If changes to one parameter necessarily affect other parameters, such dependencies MUST be documented.¶

5.3.2. ASPA Update and Revalidation Workloads

ASPA update and revalidation workloads evaluate DUT behavior when ASPA information changes while the BGP route population remains stable. The tester MUST establish a predefined ASPA dataset and BGP route population before triggering the ASPA update. The update workload SHOULD include controlled changes that produce known verification-state transitions. Examples include:
- Valid to Invalid.
- Invalid to Valid.
- Unknown to Valid.
- Unknown to Invalid.
- Valid to Unknown.
- Invalid to Unknown.¶

These transitions MUST be constructed consistently with the applicable ASPA verification procedures. The tester SHOULD independently control the number of changed ASPA records, the number of affected BGP routes, and the number of distinct affected AS_PATHs where feasible. Workloads SHOULD include both concentrated updates, in which a small number of CAS changes affect many routes, and distributed updates, in which changes to multiple CAS entries affect different route populations. These workloads are intended to characterize ASPA update processing and route revalidation performance independently of new BGP announcements.¶

5.3.3. Concurrent Update Workloads

Concurrent update workloads evaluate ASPA verification performance when ASPA information and BGP routing information change during overlapping intervals. The tester SHOULD specify independent ASPA and BGP update sequences. The following characteristics MUST be documented:
- ASPA update rate.
- BGP update rate.
- Duration of the concurrent workload.
- Timing relationship between ASPA and BGP updates.
- Number of routes affected by each workload.
- Expected verification and routing states after the workload completes.¶

Concurrent workloads SHOULD include both sustained update rates and controlled bursts. When comparing concurrent workloads with isolated workloads, the tester SHOULD use equivalent ASPA and BGP update sequences to characterize the additional processing effects of simultaneous activity.¶

5.3.4. Recovery Workloads

Recovery workloads evaluate ASPA verification behavior following RTR session interruptions, cache resets, or other controlled recovery events. The tester MUST establish a known baseline ASPA dataset and BGP route population before triggering the recovery event. Recovery workloads SHOULD distinguish between cases in which the ASPA dataset remains unchanged and cases in which the dataset changes during the interruption. The tester SHOULD document whether previously received ASPA information remains available during recovery and identify the applicable RTR timing parameters. Recovery measurements SHOULD include the time required to restore the expected ASPA dataset, verification results, and BGP routing state.¶

5.4.1. ASPA Synchronization Time

ASPA synchronization time characterizes the time required for the DUT to receive and process an initial ASPA dataset. The following measurements are distinguished: RTR Data Transfer Time: The interval between the arrival of the first PDU of the measured RTR response at the DUT and the completion of the corresponding End of Data PDU. ASPA Dataset Application Time: The instant at which the complete received ASPA dataset becomes available for ASPA verification. End-to-End ASPA Synchronization Time: The interval between the beginning of the measured RTR response at the DUT and the point at which the complete ASPA dataset is available for verification. The completion of an RTR transaction does not necessarily indicate the completion of internal ASPA processing. Therefore, RTR data transfer time MUST NOT be reported as equivalent to end-to-end ASPA synchronization time. When ASPA dataset application cannot be directly observed, the tester MUST report only the measurements supported by the available observation points. The report MUST identify whether the measurement includes RTR session establishment, cache response generation, or other protocol operations.¶

5.4.2. ASPA Update Processing Latency

ASPA update processing latency characterizes the time required for changes in ASPA information to become effective at the DUT. For a controlled update transaction, the measurement begins when the corresponding RTR update response starts arriving at the DUT. The measurement ends when the updated ASPA information becomes available for verification, as determined by the selected observation method. The tester SHOULD distinguish: - RTR update transfer time. - End-to-end ASPA update processing latency. - Post-transfer processing latency, when independently observable. The measurement MUST specify the number and types of ASPA changes included in the transaction. If the DUT applies ASPA information before the End of Data PDU is received, the tester MUST account for that behavior and MUST NOT assume that all processing begins after transfer completion. Measurements involving subsequent route revalidation MUST be reported separately from ASPA data processing latency.¶

5.4.3. AS_PATH Verification Latency

AS_PATH verification latency characterizes the time required for the DUT to determine the ASPA verification result of a BGP route. Where suitable internal instrumentation is available, verification latency MAY be measured as the interval between the beginning and completion of the ASPA verification procedure for the route. Such measurements MUST identify the corresponding instrumentation points. When direct verification latency cannot be observed, the tester MAY measure externally observable verification-related route-processing latency. This measurement begins with the arrival of the relevant BGP UPDATE at the DUT and ends at a defined observable event indicating that the route has reached the expected verification-related state. The externally observable interval may include BGP message parsing, routing policy processing, queueing, and other control-plane operations. Therefore, it MUST NOT be reported as the execution time of the ASPA verification algorithm. The tester SHOULD report latency distributions rather than only aggregate averages when sufficient samples are available.¶

5.4.4. Verification Throughput

Verification throughput characterizes the rate at which the DUT processes BGP routes requiring ASPA verification under a specified workload. Where internal verification operation counts are available, verification throughput MAY be reported in verification operations per second. Otherwise, the tester SHOULD use externally observable route-processing throughput. For externally observable throughput measurements, the workload MUST be constructed so that the relevant routes reach identifiable expected states. The tester SHOULD determine the maximum sustainable processing rate for which the DUT continues to process the workload without persistent backlog growth or unsuccessful route processing. The measurement MUST distinguish: - BGP UPDATE messages per second. - Route announcements processed per second. - Distinct AS_PATHs processed per second. - Internal ASPA verification operations per second, when measurable. These quantities MUST NOT be used interchangeably. The benchmark MUST identify which quantity is being measured and the conditions used to determine successful processing.¶

5.4.5. Route Revalidation Time

Route revalidation time characterizes the time required for previously received BGP routes to reach their expected verification states following an ASPA update. The benchmark MUST define the affected route population before initiating the update. Two measurements are distinguished: End-to-End Route Revalidation Time: The interval between the beginning of the corresponding RTR update response at the DUT and completion of the expected verification-state changes for all affected routes. Post-Application Route Revalidation Time: The interval between the point at which the updated ASPA information becomes available for verification and completion of the expected verification-state changes. The second measurement MAY be reported only when both observation points are available. The tester MUST identify how completion of route revalidation is determined. The report MUST include: - Number of changed ASPA records. - Number of affected routes. - Number of distinct affected AS_PATHs. - Expected verification-state transitions. - Route revalidation time. When a test includes routes whose verification results remain unchanged, their number SHOULD be reported separately from the population whose results change.¶

5.4.6. BGP Convergence Time

BGP convergence time characterizes the interval required for the DUT to reach the expected stable routing state following a specified event. Two categories are considered: ASPA-triggered BGP Convergence: BGP routing-state changes caused by an update to ASPA information. BGP-triggered Convergence with ASPA Verification: BGP convergence following a controlled BGP routing event while ASPA verification is enabled. For ASPA-triggered convergence, the measurement begins at the specified RTR update event and ends when the expected BGP routing state has been reached. For BGP-triggered convergence, the measurement begins at the specified BGP event and ends when the expected stable routing state has been reached. The tester MUST document the routing policies applied to ASPA verification results. A change in ASPA verification result does not necessarily cause a BGP routing-state change. Therefore, ASPA-triggered BGP convergence time MUST be measured only for test conditions in which the configured routing policy produces a defined and observable routing-state transition. When comparing ASPA-enabled and ASPA-disabled configurations, the benchmark SHOULD use equivalent BGP workloads and expected routing outcomes. Any differences in routing policy or expected convergence behavior MUST be documented.¶

5.4.7. Resource Utilization

Resource utilization characterizes the CPU and memory resources consumed by ASPA verification under specified workload conditions. At minimum, the following metrics SHOULD be collected: - Total CPU utilization. - Total memory consumption. - Peak CPU utilization during the test. - Peak memory consumption during the test. Where available, the tester SHOULD also collect: - CPU utilization of routing processes. - Memory consumption of routing processes. - Memory used to store ASPA information. - ASPA-related processing queue or backlog information. Resource-utilization measurements SHOULD be collected during both steady-state and transient workload conditions. The measurement interval, data source, and scope of the reported resource utilization MUST be documented. For tests comparing configurations with ASPA verification enabled and disabled, resource utilization SHOULD be measured under equivalent BGP workload conditions.¶

5.4.8. Scalability and Recovery Measurements

ASPA scalability is characterized by the maximum workload size that the DUT can process while satisfying predefined successful-operation criteria. Relevant scalability dimensions include: - ASPA record count. - Provider authorization count. - Provider-set cardinality. - BGP route population. - Unique AS_PATH count. - Compressed AS_PATH length. - ASPA update rate. - Number of routes affected by ASPA updates. The tester MUST specify the scalability dimension being evaluated and the criteria used to determine successful operation. These criteria MAY include successful completion of ASPA synchronization, correct verification results, stable BGP sessions, bounded processing latency, and the absence of persistent processing backlogs. A reported scalability limit MUST identify whether it represents an observed DUT limitation, a predefined performance threshold, or the maximum workload supported by the test equipment. Recovery-related measurements SHOULD include RTR reconnection time, ASPA resynchronization time, and the time required to restore the expected verification and routing states. The recovery scenario, initial conditions, and applicable RTR timing parameters MUST be documented.¶

6.1.1. Initial ASPA Synchronization

This test characterizes ASPA processing performance when the DUT loads a complete ASPA dataset from an RTR cache.¶

Test Conditions:¶

The DUT MUST be configured with ASPA verification enabled and connected to the RTR Cache/Emulator described in Section 4.3.¶

The tester MUST prepare an ASPA dataset with specified CAS and provider authorization counts.¶

The test MUST begin from a known RTR synchronization state. The tester MUST ensure that previously installed ASPA information does not unintentionally affect the measurements.¶

The test MAY be conducted with or without an existing BGP route population. When BGP routes are present, their number and AS_PATH characteristics MUST be documented.¶

Procedure:¶

1. Configure the DUT and RTR Cache/Emulator according to the selected test parameters.¶

2. Prepare an ASPA dataset with the specified CAS count, provider authorization count, and provider-set cardinality distribution.¶

3. Establish the required BGP sessions and baseline route population, if applicable.¶

4. Initiate a full RTR synchronization using the applicable Reset Query procedure.¶

5. Record the arrival time of the Cache Response PDU at the DUT-facing interface.¶

6. Record the arrival time of the corresponding End of Data PDU.¶

7. Determine when the received ASPA dataset becomes available for verification, if a suitable observation mechanism is available.¶

8. Record CPU and memory utilization during synchronization.¶

9. Verify that the expected ASPA information and verification states are available after synchronization.¶

10. Restore the initial conditions and repeat the test.¶

The tester SHOULD repeat the procedure using different ASPA dataset sizes.¶

The number of CAS entries and the total number of provider authorizations SHOULD be varied independently where feasible.¶

Measurements:¶

• RTR data transfer time.¶

• End-to-end ASPA synchronization time, when observable.¶

• ASPA application completion time, when observable.¶

• Peak CPU utilization.¶

• Peak memory consumption.¶

• Number of successfully installed ASPA records, when observable.¶

• Synchronization failures.¶

The completion of the End of Data PDU indicates completion of the corresponding RTR data transfer but does not necessarily indicate that all ASPA information has become effective for route verification.¶

Therefore, RTR data transfer time MUST NOT be reported as ASPA processing completion time.¶

When direct ASPA application completion cannot be observed, the tester MAY measure the time required for predefined routing-state changes resulting from the newly synchronized ASPA information. Such measurements MUST be reported separately as externally observable routing-state response times.¶

Reporting:¶

The report MUST include the ASPA dataset characteristics, RTR protocol version, measurement observation points, and synchronization results.¶

When other RTR payload types are present, their quantities MUST be reported. Non-ASPA payload populations SHOULD remain unchanged across test runs intended to isolate ASPA processing performance.¶

6.1.2. Incremental ASPA Update Processing

This test measures DUT processing performance when an existing ASPA dataset is modified.¶

Test Conditions:¶

The DUT MUST initially contain a predefined ASPA dataset and have reached steady state.¶

The RTR Cache/Emulator MUST support controlled ASPA additions, replacements, and withdrawals.¶

Each update transaction MUST conform to the applicable RTR protocol semantics.¶

In particular, a replacement announcement for an existing CAS MUST contain the complete new provider set.¶

Procedure:¶

1. Load the baseline ASPA dataset and establish steady state.¶

2. Prepare a controlled ASPA update transaction containing a specified number of changes.¶

3. Select one of the following update operations:¶

   • Addition of new CAS entries.¶

   • Replacement of existing CAS provider sets.¶

   • Withdrawal of existing CAS entries.¶

4. Make the updated dataset available through the RTR Cache/Emulator.¶

5. Initiate the corresponding RTR update exchange using the applicable protocol procedures.¶

6. Record the arrival time of the Cache Response PDU at the DUT-facing interface.¶

7. Record the arrival time of the corresponding End of Data PDU.¶

8. Determine when the updated ASPA information becomes effective, if directly observable.¶

9. Record processing latency and resource utilization.¶

10. Verify that the resulting ASPA dataset matches the expected state.¶

11. Restore the baseline dataset and repeat the test.¶

The tester SHOULD evaluate additions, replacements, and withdrawals separately before using mixed update transactions.¶

The tester SHOULD vary the number of changed CAS entries and provider authorizations across test runs.¶

Measurements:¶

• RTR update transfer time.¶

• ASPA update processing latency, when observable.¶

• Externally observable routing-state response time, when applicable.¶

• Number of changed CAS entries.¶

• Number of provider authorizations added or removed.¶

• CPU and memory utilization.¶

• Update processing failures.¶

Reporting:¶

Results SHOULD characterize update processing performance as a function of the number of changed CAS entries and provider authorizations.¶

The report MUST distinguish RTR transfer completion, ASPA data application, and subsequent route revalidation.¶

When an ASPA update triggers route revalidation, the corresponding revalidation performance SHOULD be measured using Section 6.3.¶

6.2.1. Baseline AS_PATH Verification

This test establishes baseline measurements for BGP route processing with ASPA verification enabled.¶

Test Conditions:¶

The DUT MUST be configured with ASPA verification enabled.¶

The tester MUST prepare an ASPA dataset and a BGP workload with known verification results.¶

The workload MUST specify:¶

• Number of BGP routes.¶

• Number of unique AS_PATHs.¶

• Compressed AS_PATH length distribution.¶

• ASPA coverage.¶

• Verification direction.¶

• Verification outcome distribution.¶

• BGP route announcement rate.¶

Procedure:¶

1. Load the predefined ASPA dataset into the DUT.¶

2. Establish the BGP sessions required for the selected verification direction.¶

3. Confirm completion of ASPA synchronization and establish steady state.¶

4. Prepare BGP routes with predefined AS_PATHs and expected verification results.¶

5. Start the Measurement System.¶

6. Announce the test routes at the specified rate.¶

7. Record the relevant BGP input events and observable verification or routing-state changes.¶

8. Measure verification-related processing latency and route-processing throughput using the observation methods defined in Section 5.4.¶

9. Verify that the DUT produces the expected verification results.¶

10. Withdraw the test routes, restore the initial state, and repeat the test.¶

The tester SHOULD conduct a corresponding measurement with ASPA verification disabled.¶

The ASPA-enabled and ASPA-disabled configurations SHOULD use equivalent BGP workloads and routing policies that preserve the same expected route-selection outcomes when the objective is to isolate ASPA processing overhead.¶

Measurements:¶

• Internal ASPA verification latency, when directly measurable.¶

• Externally observable verification-related route-processing latency.¶

• Route-processing throughput.¶

• CPU utilization.¶

• Memory consumption.¶

• Processing overhead relative to the ASPA-disabled baseline.¶

Reporting:¶

The report MUST identify the measurement method and distinguish internal verification latency from externally observable processing latency.¶

Results SHOULD include latency distributions and throughput measurements under the specified workload.¶

6.2.2. AS_PATH Length Sensitivity

This test evaluates how verification performance varies with compressed AS_PATH length.¶

Procedure:¶

1. Prepare multiple BGP workloads with different compressed AS_PATH lengths.¶

2. Construct corresponding ASPA datasets that produce known verification results.¶

3. Maintain the number of advertised routes and other relevant workload characteristics as consistently as possible.¶

4. Load the ASPA dataset corresponding to the first workload.¶

5. Establish steady state.¶

6. Announce the test routes at a fixed rate.¶

7. Measure verification-related processing latency, throughput, and resource utilization.¶

8. Restore the initial state.¶

9. Repeat the procedure for each selected AS_PATH length.¶

The tester SHOULD evaluate upstream and downstream verification separately.¶

Where feasible, the verification outcome distribution SHOULD remain consistent across the tested path lengths.¶

The tester MAY also construct workloads with different raw AS_PATH lengths but identical compressed AS_PATHs to characterize the additional processing associated with AS_PATH prepending.¶

Measurements:¶

• Verification-related processing latency.¶

• Route-processing throughput.¶

• CPU and memory utilization.¶

Reporting:¶

Results SHOULD be presented against compressed AS_PATH length.¶

Raw AS_PATH length MUST also be reported when AS_PATH prepending is present.¶

Differences in ASPA dataset characteristics or verification outcome distributions across test conditions MUST be documented.¶

6.2.3. ASPA Coverage Sensitivity

This test evaluates the effect of ASPA coverage on verification performance.¶

Procedure:¶

1. Prepare a fixed population of BGP routes and AS_PATHs.¶

2. Determine the reference AS population used to calculate ASPA coverage, as defined in Section 5.2.2.¶

3. Generate ASPA datasets with different coverage levels.¶

4. Determine the expected verification results for each dataset.¶

5. Load the first ASPA dataset and establish steady state.¶

6. Announce the predefined BGP workload.¶

7. Measure verification-related latency, throughput, and resource utilization.¶

8. Restore the initial state.¶

9. Repeat the procedure for each coverage level.¶

The BGP route population, AS_PATH characteristics, and verification direction SHOULD remain unchanged across coverage levels.¶

Because changes in ASPA coverage may affect the verification outcome distribution, the expected and observed outcome distributions MUST be reported for each dataset.¶

Measurements:¶

• Verification-related processing latency.¶

• Route-processing throughput.¶

• CPU and memory utilization.¶

• Verification outcome distribution.¶

Reporting:¶

Results SHOULD characterize verification performance as a function of ASPA coverage.¶

The report MUST identify the reference AS population and the method used to select covered CASes.¶

6.2.4. Provider-Set Cardinality Sensitivity

This test evaluates the influence of provider-set cardinality on ASPA verification performance.¶

Procedure:¶

1. Prepare ASPA datasets containing the same number of CAS entries but different provider-set cardinality distributions.¶

2. Construct BGP workloads that exercise the corresponding provider authorization relationships.¶

3. Determine the expected verification results for each workload.¶

4. Load the first ASPA dataset and establish steady state.¶

5. Announce the test routes at a fixed rate.¶

6. Measure verification-related latency, throughput, and resource utilization.¶

7. Restore the baseline state.¶

8. Repeat the procedure for the remaining provider-set cardinality distributions.¶

The tester SHOULD maintain equivalent BGP route counts, compressed AS_PATH length distributions, and verification directions across test runs.¶

When increasing provider-set cardinality also increases the total provider authorization count, the resulting dataset-size difference MUST be documented.¶

Measurements:¶

• Verification-related processing latency.¶

• Route-processing throughput.¶

• CPU and memory utilization.¶

Reporting:¶

Results SHOULD characterize verification performance as a function of provider-set cardinality.¶

The report MUST include the provider-set cardinality distribution and total provider authorization count for each dataset.¶

6.2.5. Provider Authorization Pattern Sensitivity

This test evaluates whether the distribution and position of provider authorization results along an AS_PATH influence verification performance.¶

The purpose is to characterize implementation performance without requiring a particular internal verification algorithm or optimization.¶

Procedure:¶

1. Select a fixed compressed AS_PATH length and verification direction.¶

2. Construct multiple ASPA datasets and corresponding BGP workloads with different provider authorization patterns.¶

3. Control the positions of Provider+, Not Provider+, and No Attestation results along the AS_PATH.¶

4. Determine the expected final verification result for each workload using [I-D.ietf-sidrops-aspa-verification].¶

5. Maintain equivalent route counts and compressed AS_PATH lengths across test conditions.¶

6. Load the selected ASPA dataset and establish steady state.¶

7. Announce the corresponding BGP workload.¶

8. Measure verification-related processing latency, throughput, and resource utilization.¶

9. Restore the baseline state.¶

10. Repeat the procedure for each authorization pattern.¶

The tester SHOULD include patterns in which the first relevant Not Provider+ or No Attestation result occurs at different positions along the path.¶

Where feasible, the workloads SHOULD maintain the same final verification outcome while varying the authorization pattern.¶

The positions of authorization results MUST be defined consistently with the AS_PATH orientation and the applicable upstream or downstream verification procedure.¶

The tester MUST NOT assume that an individual provider authorization result alone determines the final AS_PATH verification outcome.¶

Measurements:¶

• Verification-related processing latency.¶

• Route-processing throughput.¶

• CPU and memory utilization.¶

Reporting:¶

The report MUST describe the authorization patterns and expected verification results.¶

Results SHOULD characterize performance differences across the tested authorization patterns.¶

6.2.6. Upstream and Downstream Verification

This test evaluates verification performance under the upstream and downstream procedures defined in [I-D.ietf-sidrops-aspa-verification].¶

Procedure:¶

1. Configure an eBGP session with a neighbor relationship that invokes upstream verification.¶

2. Prepare an ASPA dataset and BGP workload with known verification results.¶

3. Load the dataset and establish steady state.¶

4. Announce the BGP workload.¶

5. Measure verification-related latency, throughput, and resource utilization.¶

6. Restore the baseline state.¶

7. Configure an eBGP session with a neighbor relationship that invokes downstream verification.¶

8. Prepare and load the corresponding ASPA dataset and BGP workload.¶

9. Repeat the measurements.¶

The tester SHOULD maintain equivalent route counts, compressed AS_PATH length distributions, and BGP update rates across the two scenarios.¶

Because upstream and downstream verification use different procedures, their provider authorization patterns and expected outcomes MUST be constructed separately.¶

Measurements:¶

• Verification-related processing latency.¶

• Route-processing throughput.¶

• CPU and memory utilization.¶

Reporting:¶

The report MUST identify the neighbor relationship, verification direction, AS_PATH characteristics, ASPA dataset characteristics, and verification outcome distribution.¶

Any workload differences between the two verification directions MUST be documented.¶

6.2.7. Sustained Verification Throughput

This test determines the maximum sustainable BGP route-processing rate with ASPA verification enabled.¶

Test Conditions:¶

The DUT MUST contain a stable ASPA dataset.¶

The BGP Route Generator MUST be capable of producing a sufficiently large population of routes with known verification results.¶

Before the test, the tester MUST define the conditions for successful sustained processing.¶

Procedure:¶

1. Prepare a predefined ASPA dataset and BGP route population.¶

2. Load the ASPA dataset and establish steady state.¶

3. Configure the BGP Route Generator with an initial announcement rate.¶

4. Start announcing routes at the selected rate for a predefined measurement interval.¶

5. Record the offered route rate and the rate at which routes reach their expected observable states.¶

6. Measure processing latency and resource utilization.¶

7. Determine whether the DUT satisfies the predefined successful-operation criteria.¶

8. Restore the initial state and repeat at a higher announcement rate.¶

9. Continue until the maximum planned rate is reached or the DUT can no longer sustain the workload.¶

The successful-operation criteria MUST include a defined condition for determining that the offered workload has been processed.¶

Where processing backlog information is available, the tester SHOULD use it to determine whether the backlog remains bounded.¶

Where backlog information is unavailable, the tester SHOULD use externally observable processing completion and a predefined maximum completion delay.¶

The tester MUST ensure that the BGP generator does not become the limiting factor.¶

Measurements:¶

• Offered BGP route announcement rate.¶

• Successfully processed route rate.¶

• Verification-related processing latency.¶

• CPU and memory utilization.¶

• Processing backlog, when observable.¶

• Processing failures.¶

Reporting:¶

The maximum sustainable processing rate MUST be reported together with the workload characteristics and successful-operation criteria.¶

The report MUST distinguish offered throughput from successfully processed throughput.¶

The result MUST be identified as route-processing throughput unless internal ASPA verification operations are measured directly.¶

6.3.1. Verification-State Transition

This test characterizes revalidation performance for different verification-state transitions.¶

Test Conditions:¶

The DUT MUST contain a predefined ASPA dataset and BGP route population.¶

The tester MUST prepare an ASPA update that changes the expected verification results of a known set of existing routes.¶

The existing BGP routes MUST remain unchanged during the test.¶

Procedure:¶

1. Load the baseline ASPA dataset.¶

2. Advertise the predefined BGP route population.¶

3. Confirm the expected initial verification states.¶

4. Establish steady state.¶

5. Prepare an ASPA update that produces a specified verification-state transition.¶

6. Trigger the update through the RTR Cache/Emulator.¶

7. Record the corresponding RTR input events.¶

8. Determine when the updated ASPA information becomes effective, if observable.¶

9. Monitor the affected routes using the selected verification-state or routing-state observation mechanism.¶

10. Determine when the expected post-update states have been reached.¶

11. Record the corresponding response time and resource utilization.¶

12. Restore the baseline dataset and repeat the test.¶

The tester SHOULD evaluate the following transitions where they can be constructed consistently with the applicable verification procedures:¶

• Valid to Invalid.¶

• Invalid to Valid.¶

• Unknown to Valid.¶

• Unknown to Invalid.¶

• Valid to Unknown.¶

• Invalid to Unknown.¶

Each transition type SHOULD be evaluated separately before mixed-transition workloads are introduced.¶

When the selected routing policy makes Invalid routes ineligible, the absence of a route from the Loc-RIB or from a monitoring session MUST NOT, by itself, be interpreted as proof that ASPA revalidation has completed.¶

If the verification-state transition cannot be directly observed, the tester MAY construct an associated routing-state transition that is externally observable. The resulting measurement MUST be labeled accordingly.¶

Measurements:¶

• End-to-end route revalidation time, when directly observable.¶

• Post-ASPA-application revalidation time, when directly observable.¶

• Externally observable routing-state response time, when applicable.¶

• Number of changed ASPA records.¶

• Number of affected routes.¶

• Number of distinct affected AS_PATHs.¶

• Verification-state transition type.¶

• CPU and memory utilization.¶

• Revalidation or routing-state completion failures.¶

Reporting:¶

Results SHOULD characterize revalidation performance for each verification-state transition type.¶

The report MUST identify the observation method and distinguish direct revalidation measurements from indirect routing-state measurements.¶

6.3.2. Affected Route Population Sensitivity

This test evaluates the effect of the number of affected BGP routes on ASPA-triggered revalidation performance.¶

Procedure:¶

1. Prepare a baseline ASPA dataset and BGP route population with known verification results.¶

2. Construct an ASPA update that changes the expected verification states of a controlled subset of routes.¶

3. Select an initial affected route population size.¶

4. Load the baseline ASPA dataset and advertise the corresponding routes.¶

5. Establish steady state.¶

6. Trigger the ASPA update.¶

7. Measure the time required for the affected routes to reach their expected observable post-update states.¶

8. Record CPU and memory utilization.¶

9. Restore the initial state.¶

10. Repeat with increasing affected route population sizes.¶

The tester SHOULD keep the number of changed ASPA records constant while varying the affected route population.¶

The compressed AS_PATH length distribution, verification direction, and verification-state transition distribution SHOULD remain equivalent across test conditions where feasible.¶

Measurements:¶

• Number of affected routes.¶

• Number of distinct affected AS_PATHs.¶

• Route revalidation time, when directly observable.¶

• Externally observable routing-state response time, when applicable.¶

• CPU and memory utilization.¶

Reporting:¶

Results SHOULD be presented against the affected route population size.¶

The report MUST include the number of distinct affected AS_PATHs and the observation method used.¶

6.3.3. Unique AS_PATH Sensitivity

This test evaluates how the number of distinct AS_PATHs influences ASPA-triggered revalidation performance.¶

Procedure:¶

1. Prepare multiple BGP workloads containing the same total number of affected routes but different numbers of distinct AS_PATHs.¶

2. Construct corresponding ASPA datasets and updates that produce known verification-state changes.¶

3. Maintain equivalent route counts and verification-state transition distributions across workloads.¶

4. Load the first ASPA dataset and advertise the corresponding routes.¶

5. Establish steady state.¶

6. Trigger the ASPA update.¶

7. Measure revalidation-related response time and resource utilization.¶

8. Restore the baseline state.¶

9. Repeat using the remaining workloads.¶

The tester SHOULD consider workloads in which:¶

• Many routes share a small number of distinct AS_PATHs.¶

• Routes are distributed across a moderate number of distinct AS_PATHs.¶

• Most routes have distinct AS_PATHs.¶

Where feasible, the compressed AS_PATH length distribution and authorization patterns SHOULD remain equivalent across the tested workloads.¶

Measurements:¶

• Number of affected routes.¶

• Number of distinct affected AS_PATHs.¶

• Route revalidation time, when directly observable.¶

• Externally observable routing-state response time, when applicable.¶

• CPU and memory utilization.¶

Reporting:¶

Results SHOULD characterize revalidation-related performance as a function of distinct AS_PATH count.¶

The report MUST include the distribution of routes across distinct AS_PATHs.¶

This test does not require the DUT to implement verification-result caching, shared path representations, or any other specific optimization.¶

6.4.1. ASPA-triggered BGP Convergence

This test evaluates BGP convergence caused by changes in ASPA verification results.¶

Test Conditions:¶

The DUT MUST be configured with an ASPA-aware routing policy that produces a deterministic and observable routing-state change following the selected ASPA update.¶

The tester MUST prepare a stable BGP route population containing routes whose eligibility or selection changes as a result of the update.¶

The initial and expected final routing states MUST be determined before testing.¶

Procedure:¶

1. Configure ASPA verification and the selected routing policy.¶

2. Load the baseline ASPA dataset.¶

3. Establish the BGP sessions and advertise the predefined routes.¶

4. Confirm the expected initial route-selection state.¶

5. Establish steady state.¶

6. Prepare an ASPA update that changes the eligibility or selection of a known set of routes.¶

7. Trigger the ASPA update.¶

8. Record the relevant RTR input events.¶

9. Monitor the routing states using the selected observation mechanism.¶

10. Determine when the expected stable post-update routing state has been reached.¶

11. Record convergence time and resource utilization.¶

12. Restore the initial state and repeat the test.¶

The tester SHOULD evaluate different numbers of affected prefixes and routes.¶

The test MAY include:¶

• A previously selected route becoming ineligible.¶

• A previously ineligible route becoming eligible.¶

• A change in the selected route among multiple candidates.¶

The selected routing policy MUST remain unchanged across comparable test runs.¶

Measurements:¶

• End-to-end ASPA-triggered BGP convergence time.¶

• Post-ASPA-application convergence time, when observable.¶

• Number of affected routes and prefixes.¶

• CPU and memory utilization.¶

• Convergence failures.¶

Reporting:¶

The report MUST describe the routing policy, initial routing state, expected final state, and convergence observation points.¶

A verification-state change that does not produce a routing-state transition MUST NOT be reported as a BGP convergence event.¶

6.4.2. BGP-triggered Convergence with and without ASPA Verification

This test compares BGP convergence performance with ASPA verification disabled and enabled.¶

Procedure:¶

1. Prepare a predefined BGP topology and route population.¶

2. Configure the DUT with ASPA verification disabled.¶

3. Establish the initial routing state.¶

4. Trigger a controlled BGP routing event, such as:¶

   • Announcement of a predefined route population.¶

   • Withdrawal of selected routes.¶

   • Replacement of selected AS_PATHs.¶

   • Controlled BGP peer failure.¶

5. Measure the time required to reach the expected stable routing state.¶

6. Record CPU and memory utilization.¶

7. Restore the initial state.¶

8. Enable ASPA verification and load the predefined ASPA dataset.¶

9. Confirm that the expected initial routing state is equivalent to that used in the ASPA-disabled test.¶

10. Repeat the same BGP event.¶

11. Measure convergence time and resource utilization.¶

12. Compare the results.¶

To isolate the processing overhead of ASPA verification, the expected route-selection outcomes SHOULD remain equivalent between the two configurations.¶

The measurement start and completion conditions MUST be equivalent.¶

Measurements:¶

• BGP convergence time without ASPA verification.¶

• BGP convergence time with ASPA verification.¶

• Absolute convergence-time difference.¶

• Relative convergence-time difference.¶

• CPU and memory utilization.¶

The relative convergence-time difference MAY be calculated as:¶

Relative Change = ((T_ASPA - T_Baseline) / T_Baseline) * 100%

where T_ASPA and T_Baseline are measured under equivalent test conditions.¶

Reporting:¶

The report MUST identify the BGP event, routing configuration, workload characteristics, and convergence criteria.¶

Results SHOULD include absolute convergence times, relative differences, and measurement variability.¶

6.5.1. ASPA Dataset Scalability

This test evaluates DUT performance as the ASPA dataset size increases.¶

Procedure:¶

1. Generate ASPA datasets with progressively increasing CAS counts.¶

2. Maintain a controlled provider-set cardinality distribution.¶

3. Prepare a fixed BGP workload, where applicable.¶

4. Load the first ASPA dataset and establish steady state.¶

5. Measure synchronization performance, verification-related processing performance, and resource utilization.¶

6. Restore the initial state.¶

7. Repeat using larger ASPA datasets.¶

8. Continue until the maximum planned dataset size is reached or the DUT fails to satisfy the predefined successful-operation criteria.¶

The tester SHOULD conduct a separate series of measurements in which the provider authorization count increases while CAS count remains fixed.¶

The tester SHOULD distinguish ASPA records referenced by the current BGP workload from records unrelated to the current AS_PATH population.¶

Measurements:¶

• ASPA synchronization time.¶

• Verification-related processing latency.¶

• CPU and memory utilization.¶

• Successfully installed ASPA records, when observable.¶

• Processing failures.¶

Reporting:¶

Results SHOULD be presented against ASPA record count and provider authorization count.¶

The report MUST distinguish DUT limitations from test equipment limitations.¶

6.5.2. BGP Route and AS_PATH Scalability

This test evaluates the impact of BGP route population and distinct AS_PATH count on ASPA verification performance.¶

Procedure:¶

1. Prepare a fixed ASPA dataset.¶

2. Generate BGP workloads with increasing route counts.¶

3. Define the number and distribution of distinct AS_PATHs for each workload.¶

4. Load the ASPA dataset and establish steady state.¶

5. Announce the selected BGP workload.¶

6. Measure processing throughput, latency, and resource utilization.¶

7. Restore the initial state.¶

8. Repeat using larger workloads.¶

The tester SHOULD evaluate route count and distinct AS_PATH count as separate scalability dimensions.¶

When isolating route-count effects, the number and characteristics of distinct AS_PATHs SHOULD remain fixed.¶

When isolating unique-AS_PATH effects, the total route count SHOULD remain fixed.¶

Measurements:¶

• Route-processing throughput.¶

• Verification-related processing latency.¶

• CPU and memory utilization.¶

• Processing failures.¶

Reporting:¶

The report MUST include route count, unique AS_PATH count, and the distribution of routes across distinct AS_PATHs.¶

Results SHOULD characterize performance changes across the tested workload sizes.¶

6.5.3. Scalability Limit Determination

This test identifies the highest workload level at which the DUT satisfies predefined successful-operation criteria.¶

Procedure:¶

1. Select a scalability parameter and define its test range.¶

2. Establish successful-operation criteria before testing.¶

3. Execute the corresponding benchmark at the initial parameter value.¶

4. Increase the selected parameter while maintaining other relevant workload characteristics.¶

5. Determine whether the DUT satisfies the successful-operation criteria.¶

6. Continue until an unsuccessful condition occurs or the maximum planned test value is reached.¶

7. Record the highest successfully tested value.¶

8. Repeat measurements near the observed limit where feasible.¶

Successful-operation criteria SHOULD include:¶

• Successful ASPA synchronization, when applicable.¶

• Expected ASPA verification results.¶

• Stable BGP sessions.¶

• Completion of the specified processing workload.¶

• Absence of persistent processing backlogs.¶

• Absence of resource exhaustion or processing failures.¶

Any latency or resource-utilization thresholds used to define successful operation MUST be specified before testing.¶

Reporting:¶

The report MUST identify the tested scalability parameter, successful-operation criteria, highest successful test value, and observed failure conditions.¶

A result MUST NOT be described as the maximum DUT capacity when the test is limited by the workload generator or the maximum planned test size.¶

6.6.1. Sustained ASPA Churn

This test evaluates the DUT under a sustained sequence of ASPA dataset changes.¶

Test Conditions:¶

The DUT MUST contain a predefined ASPA dataset and BGP route population.¶

The RTR Cache/Emulator MUST support reproducible ASPA update sequences containing additions, replacements, and withdrawals.¶

The update workload MUST conform to the applicable RTR protocol procedures and timing constraints.¶

Procedure:¶

1. Load the baseline ASPA dataset and BGP route population.¶

2. Confirm the expected initial verification states.¶

3. Establish steady state.¶

4. Prepare an ASPA update sequence with a specified rate and duration.¶

5. Start the update sequence using the applicable RTR protocol procedures.¶

6. Record the rate of changes generated by the ASPA data source and the rate of changes delivered to the DUT.¶

7. Monitor ASPA processing, route revalidation behavior, and resource utilization throughout the test.¶

8. Stop generating new updates after the predefined test duration.¶

9. Determine whether the DUT processes all delivered updates and reaches the expected final verification state.¶

10. Record the time required to reach the final stable state.¶

11. Restore the baseline conditions and repeat at different update rates.¶

The tester SHOULD evaluate:¶

• Repeated additions and withdrawals.¶

• Replacements of existing provider sets.¶

• Updates distributed across many CAS entries.¶

• Repeated updates affecting a small CAS population.¶

The ASPA dataset size SHOULD remain approximately stable when the objective is to isolate update-frequency effects.¶

The tester MUST distinguish changes generated by the ASPA data source from RTR update transactions actually delivered to the DUT.¶

Multiple source-side dataset changes may be represented by a smaller number of RTR update transactions. These quantities MUST NOT be treated as interchangeable.¶

Measurements:¶

• Generated ASPA change rate.¶

• Delivered ASPA update rate.¶

• ASPA update processing latency, when observable.¶

• Route revalidation time, when observable.¶

• CPU and memory utilization.¶

• Processing backlog, when observable.¶

• Time to reach the final stable state.¶

• Processing failures.¶

Reporting:¶

Results SHOULD characterize processing performance and resource utilization against the delivered ASPA update rate.¶

The report MUST describe the update sequence, update types, test duration, and actual RTR transactions delivered.¶

6.6.2. Burst ASPA Updates

This test evaluates the DUT when a large number of ASPA changes are delivered within a short interval.¶

Procedure:¶

1. Establish the baseline ASPA dataset and BGP route population.¶

2. Confirm steady state.¶

3. Prepare a controlled ASPA update transaction containing a specified number of changed CAS entries.¶

4. Trigger the corresponding RTR update exchange.¶

5. Record the RTR response timing.¶

6. Measure ASPA update processing performance using the available observation points.¶

7. Measure subsequent route revalidation behavior, when applicable.¶

8. Record peak resource utilization.¶

9. Verify that the DUT reaches the expected final state.¶

10. Restore the baseline conditions and repeat with different burst sizes.¶

The tester SHOULD distinguish between:¶

• A single RTR response containing many ASPA changes.¶

• Multiple consecutive RTR update transactions.¶

These patterns MUST NOT be treated as equivalent.¶

Measurements:¶

• Number of changed CAS entries.¶

• Number of affected provider authorizations.¶

• RTR update transfer time.¶

• ASPA update processing latency, when observable.¶

• Route revalidation time, when observable.¶

• Peak CPU and memory utilization.¶

• Processing failures.¶

Reporting:¶

Results SHOULD be presented against update burst size.¶

The report MUST identify whether the burst consists of one RTR transaction or multiple transactions.¶

6.6.3. Concurrent ASPA and BGP Updates

This test evaluates DUT performance when ASPA information and BGP routing information change during overlapping intervals.¶

Test Conditions:¶

The tester MUST prepare independent, reproducible ASPA and BGP update sequences.¶

The expected final ASPA dataset, verification results, and BGP routing states MUST be determined before testing.¶

Procedure:¶

1. Load the baseline ASPA dataset.¶

2. Establish the baseline BGP route population.¶

3. Confirm the expected initial verification and routing states.¶

4. Prepare the ASPA update sequence.¶

5. Prepare the BGP announcement and withdrawal sequence.¶

6. Establish steady state.¶

7. Start the two update sequences according to a predefined timing relationship.¶

8. Record ASPA and BGP input events and corresponding observable processing behavior.¶

9. Measure processing latency, throughput, and resource utilization during the test.¶

10. Stop both update sequences after their specified durations.¶

11. Determine whether the DUT reaches the expected final ASPA and BGP states.¶

12. Restore the baseline conditions and repeat using different combinations of update rates.¶

The tester SHOULD evaluate:¶

• ASPA updates beginning before BGP updates.¶

• BGP updates beginning before ASPA updates.¶

• ASPA and BGP updates beginning within the same controlled interval.¶

The tester SHOULD also conduct corresponding ASPA-only and BGP-only tests using equivalent update sequences.¶

These isolated measurements provide a baseline for evaluating the additional effects of concurrent workloads.¶

Measurements:¶

• ASPA update rate.¶

• BGP route announcement and withdrawal rates.¶

• ASPA processing latency, when observable.¶

• BGP route-processing throughput.¶

• Route revalidation time, when observable.¶

• BGP convergence time, when applicable.¶

• CPU and memory utilization.¶

• Processing backlog and failures.¶

Reporting:¶

The report MUST identify the timing relationship between ASPA and BGP updates.¶

Results SHOULD compare concurrent workload performance with the corresponding isolated workload measurements.¶

6.8.1. RTR Session Interruption and Reconnection

This test evaluates DUT recovery following a temporary RTR session interruption.¶

Test Conditions:¶

The DUT MUST initially contain a predefined ASPA dataset and BGP route population.¶

The RTR session MUST be established, and the expected verification states MUST be confirmed before the interruption.¶

Procedure:¶

1. Establish the RTR session and load the baseline ASPA dataset.¶

2. Establish the baseline BGP route population.¶

3. Confirm the expected verification states.¶

4. Record the configured RTR timing parameters.¶

5. Establish steady state.¶

6. Trigger a controlled RTR session interruption.¶

7. Record the interruption time.¶

8. Observe whether previously received ASPA information remains available during the interruption.¶

9. Restore RTR connectivity after a predefined interval.¶

10. Measure the time required to re-establish the RTR session and complete any necessary synchronization.¶

11. Determine when the expected verification and routing states have been restored.¶

12. Record resource utilization and recovery failures.¶

13. Restore the baseline conditions and repeat.¶

The tester SHOULD distinguish interruptions shorter than the configured Expire Interval from interruptions that cause the previously received data to expire.¶

The benchmark MUST account for the applicable RTR data-retention and expiration requirements.¶

Measurements:¶

• RTR session reconnection time.¶

• ASPA resynchronization time.¶

• Time to restore the expected verification state, when observable.¶

• Externally observable routing-state recovery time, when applicable.¶

• CPU and memory utilization.¶

• Recovery failures.¶

Reporting:¶

The report MUST include the interruption duration, RTR timing parameters, and observed ASPA data-retention behavior.¶

6.8.2. Cache Reset and Full ASPA Resynchronization

This test evaluates the DUT when the RTR cache cannot provide the requested incremental update and a full synchronization is required.¶

Procedure:¶

1. Establish a stable RTR session and load the baseline ASPA dataset.¶

2. Establish the baseline BGP route population.¶

3. Confirm the expected verification and routing states.¶

4. Configure the RTR Cache/Emulator so that the DUT requests an incremental update that the cache cannot provide.¶

5. Cause the cache to respond with a Cache Reset PDU according to the applicable RTR protocol procedures.¶

6. Observe the DUT's recovery behavior and subsequent RTR exchanges.¶

7. If the DUT proceeds with full synchronization, record the corresponding Reset Query, Cache Response, payload PDUs, and End of Data PDU.¶

8. Measure the time required to complete the RTR synchronization.¶

9. Determine when the expected ASPA verification and routing states have been restored, using the available observation mechanisms.¶

10. Record resource utilization and recovery failures.¶

11. Restore the baseline conditions and repeat.¶

The tester SHOULD evaluate scenarios in which the restored ASPA dataset is unchanged and scenarios in which it differs from the original dataset.¶

When the dataset changes, the expected verification-state transitions MUST be determined before testing.¶

A Cache Reset PDU does not, by itself, imply immediate invalidation or removal of all previously received ASPA information. The DUT's handling of retained data and subsequent recovery MUST be interpreted according to the applicable RTR specification.¶

Measurements:¶

• Cache reset recovery time.¶

• Full RTR data transfer time.¶

• ASPA resynchronization time, when observable.¶

• Route revalidation time, when observable.¶

• BGP convergence time, when applicable.¶

• CPU and memory utilization.¶

• Recovery failures.¶

Reporting:¶

The report MUST identify the triggering condition, RTR protocol exchanges, ASPA dataset characteristics, and observation points used to determine recovery completion.¶

6.8.3. RTR Cache Failover

This optional test evaluates DUT behavior when ASPA information is obtained from an alternative RTR cache following the unavailability of a previously used cache.¶

Test Conditions:¶

The test environment MUST provide at least two RTR caches or emulators.¶

The datasets and configurations of the caches MUST be documented.¶

The DUT MUST support a cache configuration that allows the selected failover scenario to be exercised.¶

Procedure:¶

1. Configure the DUT with multiple RTR caches according to its supported cache selection behavior.¶

2. Establish the baseline ASPA dataset and BGP route population.¶

3. Confirm the expected verification and routing states.¶

4. Establish steady state.¶

5. Trigger a controlled failure of the cache selected for the test.¶

6. Observe the DUT's recovery behavior and RTR protocol exchanges.¶

7. Determine whether and when the DUT begins obtaining ASPA information from an alternative cache.¶

8. Measure the time required to restore the expected ASPA verification state, when observable.¶

9. Record any associated route revalidation or BGP routing-state changes.¶

10. Restore the baseline conditions and repeat.¶

The tester SHOULD distinguish between scenarios in which the caches contain equivalent ASPA datasets and scenarios in which their datasets differ.¶

The test MUST account for the DUT's supported cache selection behavior and the applicable RTR protocol semantics.¶

Measurements:¶

• RTR recovery time.¶

• ASPA resynchronization time.¶

• Time to restore the expected verification state, when observable.¶

• Externally observable routing-state recovery time, when applicable.¶

• CPU and memory utilization.¶

• Recovery failures.¶

Reporting:¶

The report MUST identify the cache configuration, dataset characteristics, relevant timing parameters, and observed recovery behavior.¶

If the DUT does not support the selected cache failover scenario, the test MUST be reported as not applicable rather than as a performance failure.¶