<?xml version='1.0' encoding='utf-8'?>

<!DOCTYPE rfc [
  <!ENTITY RFC5040 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5040.xml">
  <!ENTITY RFC5041 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5041.xml">
  <!ENTITY RFC5042 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5042.xml">
  <!ENTITY RFC5043 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5043.xml">
  <!ENTITY RFC5044 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.5044.xml">
  <!ENTITY RFC6580 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.6580.xml">
  <!ENTITY RFC6581 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.6581.xml">
  <!ENTITY RFC6703 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.6703.xml">  
  <!ENTITY RFC7306 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.7306.xml">
]>


<?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?>
<?rfc toc="yes"?>
<?rfc sortrefs="yes"?>
<?rfc symrefs="yes"?>
<?rfc comments="yes"?>

<rfc xmlns:xi="http://www.w3.org/2001/XInclude" ipr="trust200902" docName="draft-kcrh-hpwan-state-of-art-04" category="info" obsoletes="" updates="" submissionType="IETF" xml:lang="en" tocInclude="true" sortRefs="true" symRefs="true" version="3">

  <front>
    <title abbrev="HP-WAN STATE OF ART">Current State of the Art for High Performance Wide Area Networks</title>
    <seriesInfo name="Internet-Draft" value="draft-kcrh-hpwan-state-of-art-04"/>

    <author initials="D." surname="King" fullname="Daniel King">
      <organization>Lancaster University</organization>
      <address>
        <email>d.king@lancaster.ac.uk</email>
      </address>
    </author>

    <author initials="T." surname="Chown" fullname="Tim Chown">
      <organization>Jisc</organization>
      <address>
      <email>tim.chown@jisc.ac.uk</email>
      </address>
    </author>

    <author initials="C." surname="Rapier" fullname="Chris Rapier">
      <organization>Pittsburgh Supercomputing Center</organization>
      <address>
        <email>rapier@psc.edu</email>
      </address>
    </author>

    <author initials="D" surname="Huang" fullname="Daniel Huang">
      <organization>ZTE Corporation</organization>
      <address>
      <email>huang.guangping@zte.com.cn</email>
      </address>
    </author>

    <author initials="K." surname="Yao" fullname="Kehan Yao">
      <organization>China Mobile</organization>
      <address>
        <email>yaokehan@chinamobile.com</email>
      </address>
    </author>

    <date year="2026" month="November" day="2"/>

    <workgroup></workgroup>

    <keyword>HP-WAN</keyword>

    <abstract>

      <t>High Performance Wide Area Networks (HP-WANs) provide 
         infrastructure for the modern global Research and Education (R&amp;E) community, 
         facilitating collaboration across national and international boundaries. 
         These networks include global education and research networks, such as GÉANT, Internet2, Janet, ESnet, CANARIE, CERNET, and others, and also refer to large-scale commercial dedicated networks built by hyperscalers and operators.
         They are designed to support the transfer of data generated by scientific 
         research, high-performance computing, distributed AI-training, and large-scale simulations.</t>

      <t>This document provides an overview of the terminology and techniques used for 
         existing HP-WANs. It also explores the technological advancements, operational tools, 
         and future directions for HP-WANs, including their use in 
         scientific research, AI training, and large-scale R&amp;E data analysis.</t>

    </abstract>

  </front>

  <middle>

    <section anchor="INTRO" numbered="true" toc="default">
      <name>Introduction</name>

      <t>High Performance Wide Area Networks (HP-WANs) are the backbone of global Research 
         and Education (R&amp;E) infrastructure. They support large data transfers
         and scientific collaborations worldwide. These networks 
         are designed to meet the demanding requirements of data-intensive research fields, 
         including high-energy physics, climate modelling, genomics, and artificial 
         intelligence.</t>
         
      <t>HP-WANs have evolved with the growth in scientific datasets and international
         research collaboration. General-purpose WAN services did not always meet the
         capacity requirements of research institutions. As datasets grew from terabytes
         to petabytes, these institutions needed networks that could carry the data
         between sites.</t>  
                
      <t>Specialised (inter)National R&amp;E Networks (NRENs), such as ESnet in the United States, GÉANT in Europe, Janet in the UK, and CERNET in China, have evolved to support the unique needs of the scientific community, while also 
         carrying more generalised and less demanding research and education traffic. These networks are designed by their operators to provide high bandwidth, low latency, reliability, and security for
         applications like real-time data analysis, distributed computing, and remote instrumentation.</t>   
         
      <t>This evolution has been made possible by the design decisions made by NREN operators 
         and campus operators who build the localised network and systems infrastructures. HP-WANs require an end-to-end engineering approach:
         NREN operators provide capacity, while local campus operators engineer their network architectures and data transfer and compute systems to use that capacity effectively, e.g., by adopting the Science DMZ approach described in
         this document.</t>

      <t>Today, HP-WANs are foundational to the research community and are leading the way in
         demonstrating how advanced networking technologies can be applied to other sectors. 
         They serve as testbeds for networking innovations that eventually trickle down to 
         broader commercial applications. Emerging technologies such as quantum computing and 
         the Internet of Agents (IoA) may place further demands on these networks.</t>  
 
      <t>This document explores the current state of the art in HP-WANs, examining the 
         technological advancements, operational challenges, and emerging trends shaping the 
         future of networks built for research, education, massive data analysis, and 
         collaborative AI training at scale and speed.</t>  

       <section anchor="BACK" numbered="true" toc="default">
        <name>Background</name>

        <t>High Performance Wide Area Networks (HP-WANs) evolved as specialised networks initially designed to facilitate scientific research requiring high-speed data transfer, high reliability, and minimal latency. ESnet, Janet, GÉANT, and CERNET emerged in response to the increasing data volumes generated by scientific and educational institutions, extending the capabilities of traditional WANs.</t>
     
        <t>HP-WAN is an end-to-end capability, architected through collaboration among NREN operators, campus IT operators, and research communities. These networks have become integral to research and education, supporting distributed scientific collaborations, large-scale simulations, and intensive data analysis. Their capabilities have been continually enhanced to meet rising demands, laying foundations for future networking technologies.</t>

       </section> 

    </section> 


    <section anchor="TERM" numbered="true" toc="default">
        <name>Terminology</name>

        <t>This section provides terminology related to high-performance WANs.</t>

        <dl newline="false" spacing="normal">

          <dt>CERN:</dt>
          <dd>The European Organization for Nuclear Research, housing the Large Hadron Collider (LHC).</dd>

          <dt>High Performance Computing (HPC):</dt>
          <dd>Is a general term for computing with a high level
              of performance. Often high performance computing specifically refers to 
              running jobs which are very parallel, often running on hundreds or even 
              thousands of cores.</dd>

          <dt>High Performance Wide Area Network (HP-WAN):</dt>
          <dd>A type of Wide Area Network (WAN) designed specifically to meet the high-speed, 
              low-latency, and high-capacity needs of scientific research, education, and 
              data-intensive applications. These networks connect research institutions, 
              universities, and data centres across large geographical areas.</dd>

          <dt>Infiniband:</dt>
          <dd>Traditionally, a localised data interconnect used by many high performance computing (HPC) 
              systems providing high bandwidth and low latency.</dd> 

          <dt>National Research and Education Network (NREN):</dt>
          <dd>A specialised network operated to support the research and education community within a 
              specific country or region. NRENs provide high-speed connectivity and other 
              services tailored to the needs of academic and research institutions.</dd>

          <dt>Remote direct memory access (RDMA):</dt>
          <dd>Enables one networked node to access another 
              networked node's memory without involving either computer's operating system 
              or interrupting either node's processing. This helps minimise latency and maximise 
              throughput, reducing memory bandwidth bottlenecks.</dd>

          <dt>RDMA over Converged Ethernet (RoCE):</dt>
          <dd>Traditionally, a network protocol which allows remote direct 
              memory access (RDMA) over a local Ethernet network. There are multiple RoCE versions. 
              RoCE v1 <xref target="UEC"/> is an Ethernet link-layer protocol and allows communication between any
              two hosts in the same Ethernet broadcast domain. RoCE v2 is an internet layer 
              protocol which means that RoCE v2 packets can be routed.</dd>

          <dt>Worldwide LHC Computing Grid (WLCG):</dt>
          <dd>Is a global network of over 170 computing centres across more than 40 countries, 
              designed to process, store, and analyse the vast amounts of data generated by the 
              Large Hadron Collider (LHC) at CERN.</dd>

          <dt>perfSONAR:</dt>
          <dd>A network performance monitoring toolkit designed to provide end-to-end 
              performance measurement and monitoring across multi-domain network 
              infrastructures.</dd>

          <dt>Science DMZ:</dt>
          <dd>A model for deployment of infrastructure at a site (campus) to optimise 
              the performance of data transfers in and out of data transfer nodes 
              (DTNs) at the site; see https://fasterdata.es.net/science-dmz/. Elements 
              of the model include the local network architecture, tuning of DTNs, 
              selection of data transfer software, efficient implementation of 
              security policies, and persistent monitoring.</dd>


        </dl>
    </section>

    <section anchor="UC" numbered="true" toc="default">
      <name>Example Use Cases for HP-WANs</name>

        <t>HP-WAN development has become synonymous with large-scale research and experimentation, big data, and AI. HPC, and therefore HP-WAN, is 
          driven by the continuous need to move large data between HPC facilities, facilitating the following industries:</t>

      <ul spacing="normal">
        <li>High-Energy Physics Research, e.g., the Large Hadron Collider (LHC)</li>
        <li>Climate Modelling</li>
        <li>Radioastronomy, e.g., the Square Kilometre Array (SKA) project</li>
        <li>Healthcare, Genomics and Life Sciences</li>
        <li>AI training</li>
        <li>Data Backup</li>


      </ul>          

    <t>The data rates required by HPC applications vary significantly based on the application type and data scale.</t> 
        
    <t>Scientific simulations, such as climate modelling and molecular dynamics, typically demand data rates from 
       10 Gbps to over 100 Gbps due to the large volumes of data processed and moved between nodes and storage systems.</t>

       <t>In high-energy physics, such as experiments at CERN, data rates can reach hundreds of gigabits per second, with 
        aggregate peaks between sites currently exceeding 1 Tbps and predicted to rise to 10 Tbps during intensive data processing.</t>

    <t>Healthcare, Genomics, and Life Sciences might typically operate at rates between 1 Gbps and 40 Gbps. These applications 
       require high throughput to handle large datasets efficiently, often through parallel data streams.</t>

    <t>AI learning and tasks, particularly those involving deep learning, require data
      rates from 10 Gbps to 100 Gbps to ensure efficient data movement
      and keep GPUs and other accelerators fully utilised.</t>

    <t>These varying data rates underscore the high demands of HPC applications, which are expected to grow as the field 
       evolves and datasets become larger, along with the growing need to move large datasets between sites (including data centres).</t>
          
    </section>


    <section anchor="HP-WAN" numbered="true" toc="default">
      <name>Current Technologies Used in HP-WANs: Key Components</name>

      <t>HP-WANs are specialised wide area networks designed to connect
         supercomputers, scientific instruments, data repositories, and other high-performance computing resources across sites. They enable collaboration on computational tasks that require significant processing power, memory, 
         and data storage, and they support large-scale scientific research, complex simulations, and 
         data-intensive tasks that exceed the capabilities of standard enterprise networks.</t> 
         
      <t>The following sub-sections outline typical characteristics and requirements for HP-WANs. 
         These technical requirements ensure wide-area interconnects can
         meet the demands of distributed HPC environments, enabling researchers and 
         scientists to collaborate effectively worldwide.</t>

       <section anchor="ARCH" numbered="true" toc="default">
           <name>Architectural Elements</name>

           <t>Not all HP-WAN deployments rely on backbone controllers. In many NRENs, capacity is provisioned ahead of demand on private dedicated infrastructure, with no QoS or bandwidth-on-demand systems in the core.</t>
           
           <t>Some HP-WAN providers may choose, or need, to use specific resource and/or network controllers to deliver that functionality.</t>
           
           <t>Resource Controllers monitor and configure individual network resources, such as routers and switches, to manage their utilisation and performance.</t>

           <t>Network Controllers maintain global visibility of network topology, resource availability, and status, essential for path computation, resource reservation, and dynamic reconfiguration to meet stringent performance demands.</t>

           <t>End-to-End Orchestration translates user and application requirements into network operations for automated, policy-driven management.</t>
       </section>


       <section anchor="TOPO" numbered="true" toc="default">
        <name>Topology</name>
       
         <t>HPC networks can be broadly categorised into intra-site networks, which connect components
            within a single HPC site, such as a data centre, and inter-site networks, which link 
            multiple HPC sites across different geographical locations. The intra-site (HP-WAN) networks typically 
            use high-speed, low-latency non-Internet interconnects like InfiniBand or high-speed Ethernet. In 
            contrast, inter-site networks rely on dedicated,
            high-capacity wide area networks (WANs) to support
            distributed computing and data sharing at regional and
            global scales.</t>
            
         <t>Each NREN operator, e.g., Jisc in the case of Janet in the UK, will build and operate the 
            NREN infrastructure for its research and education users. This may typically take the form 
            of a well-provisioned backbone, with regional access networks extending to the end sites 
            (campuses, research organisations, etc). The NREN demarcation is typically at the campus 
            edge. In some countries the regional networks are operated separately.</t>
                       
         <t>The NRENs then typically have interconnects to other NRENs, forming a worldwide RE 
            network infrastructure. In Europe, GÉANT provides connectivity between the European NRENs 
            and then wider connectivity to the rest of the world. NRENs will also have other interconnects 
            to non-RE networks, e.g., via one or more national IXs, direct peerings to content providers 
            (including the big cloud providers) and then "catch-all" commodity connectivity via one or 
            more Tier 1 ISPs.</t>

         <t>Dedicated infrastructure is commonly used in HPC environments where performance, security, 
            and reliability are paramount. In these cases, the network infrastructure is built 
            exclusively for HPC applications, including dedicated fibre-optic connections, private 
            data centres, and specialised network transport like RDMA over Converged Ethernet (RoCE) 
            and InfiniBand nodes. The primary benefits of dedicated
            infrastructure include optimised performance for HPC tasks,
            stronger security by preventing unauthorised access, and
            consistent reliability by avoiding congestion or performance 
            issues caused by other network traffic.</t>
            
         <t>Usually, the responsibility for networking within an end site or campus lies
            with that organisation (e.g., a university IT department),
            while an HPC facility may have dedicated (separate) staff to
            operate it. With the additional administrative domains of the NRENs and 
            inter-NREN backbones like GÉANT, end-to-end traffic may pass through many networks operated by 
            different organisations. To achieve optimal end-to-end performance, everyone on the end-to-end path needs to implement best 
            practices.</t>

       </section> 

       <section anchor="BW" numbered="true" toc="default">
        <name>Bandwidth and Latency</name>

         <t>The technical requirements for wide area interconnects between HPC sites are stringent, 
            given the unique demands of distributed high-performance computing. High bandwidth is a 
            primary requirement, as these interconnects must support the rapid transfer of large 
            datasets between sites without data movement becoming a bottleneck in 
            computational workflows. HPC data flows might typically consume 1 Gbit/s to beyond 400 Gbit/s.</t>
            
         <t>Low latency is needed for applications that require real-time or near-real-time
            data processing. Latency requirements for inter-DC locations will be in the
            low-millisecond range.</t>

       </section> 

       <section anchor="PROTO" numbered="true" toc="default">
        <name>Localised Data Movement</name>
                           
         <t>Network-intensive applications like networked storage or cluster computing need 
            a network infrastructure with high bandwidth and low latency.</t>
                  
         <t>These interconnects may need to support specialised communication protocols 
            designed for HPC environments, such as Remote Direct Memory Access (RDMA) <xref target="RFC5040"/> and <xref target="RFC7306"/>, which 
            optimises the performance of distributed HPC applications by reducing overhead and 
            improving data transfer efficiency.</t>
            
         <t>InfiniBand (IB) is another computer networking communications standard used in 
            high-performance computing that features very high throughput and very low latency. 
            InfiniBand is also used as a direct or switched interconnect
            between servers and storage systems, as well as between
            storage systems.</t>
            
          <t>The advantages of RDMA and IB over other network application programming interfaces
             include lower latency, lower CPU load, and higher effective bandwidth. The downside of these specialised 
             protocols is that all interfaces and nodes must support the
             technique along the end-to-end path.</t>
        
         <t>iWARP is a computer networking protocol that implements remote direct memory access (RDMA) 
            for efficient data transfer over Internet Protocol networks. Several IETF techniques 
            are used for iWARP:</t>
         
           <ul spacing="normal">
           <li><xref target="RFC5040"/> A Remote Direct Memory Access Protocol Specification is layered over Direct Data Placement Protocol (DDP). It defines how RDMA Send, Read, and Write operations are encoded using DDP into headers on the network.</li>
           <li><xref target="RFC5041"/> Direct Data Placement over Reliable Transports is layered over MPA/TCP or SCTP. It defines how received data can be placed directly into upper-layer protocol receive buffers without intermediate buffers.</li>
           <li><xref target="RFC5042"/> Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security analyses security issues related to iWARP DDP and RDMAP protocol layers.</li>                
            <li><xref target="RFC5043"/> Stream Control Transmission Protocol (SCTP) Direct Data Placement (DDP) Adaptation defines an adaptation layer that enables DDP over SCTP.</li>
           <li><xref target="RFC5044"/> Marker PDU Aligned Framing for TCP Specification defines an adaptation layer that enables preservation of DDP-level protocol record boundaries layered over the TCP reliable connected byte stream.</li>       
           <li><xref target="RFC6580"/> IANA Registries for the Remote Direct Data Placement (RDDP) Protocol defines IANA registries for Remote Direct Data Placement (RDDP) error codes, operation codes, and function codes.</li>                
           <li><xref target="RFC6581"/> Enhanced Remote Direct Memory Access (RDMA) Connection Establishment fixes shortcomings with iWARP connection setup.</li>
           <li><xref target="RFC7306"/> Remote Direct Memory Access (RDMA) Protocol Extensions extends <xref target="RFC5040"/> with atomic operations and RDMA Write with Immediate Data.</li>        
         </ul>        
                 
       </section> 

       <section anchor="RTG" numbered="true" toc="default">
        <name>Forwarding Optimisation</name>

          <t>The scaling of HPC applications, especially across a WAN between multiple sites, 
             requires the ability to route massive traffic volumes. Specifically, this requires network 
            infrastructure to provide several routing and forwarding characteristics, which are detailed below.</t>

         <ul spacing="normal">
           <li>Low entropy: Compared to traditional data centre workloads, the number and the diversity
               of flows for workloads and flow patterns are usually repetitive and predictable.</li>
           <li>Burstiness: Flows usually exhibit the "on and off" nature in the time granularity of milliseconds.</li>
           <li>Jumbo frames: Ethernet frames larger than the standard maximum transmission unit (MTU) size of 1,500 bytes, 
               typically carrying payloads of up to 9,000 bytes. Using jumbo frames can significantly enhance network 
               efficiency and reduce CPU overhead.</li>                
           <li>Elephant flows: In each burst, the intensity of a flow can reach the line rate of NICs.</li>
         </ul>     

         <t>Efficiently handling these elephant flows is important in HPC as they can otherwise 
            saturate network links, leading to congestion and reduced performance for other network traffic. Strategies 
            to manage elephant flows effectively, such as prioritising these flows or segmenting network traffic, help 
            maintain overall network performance and prevent large data transfers from hindering the execution of 
            other critical tasks within the HPC environment.</t>

         <t>HPC transport options include IP (UDP and TCP) and emerging mechanisms such as QUIC. However, 
            each transport technology provides strengths and weaknesses. In all cases, the goal is to transfer large data sets at high throughput 
            with low latency, jitter, and packet loss.</t>

       </section> 

       <section anchor="REL" numbered="true" toc="default">
        <name>Reliability and High Availability</name>

         <t>Packet loss and path failures can interrupt HPC data transfers or increase
            their completion time. Operators need to assess recovery behaviour as well
            as steady-state throughput, particularly for long-running jobs.</t>
            
         <t>Where the transport uses retransmission, operators can monitor packet loss
            and retransmissions to identify impaired paths. Testing path failover under
            load helps establish how quickly transfers resume and whether the alternate
            path has sufficient capacity.</t>            
            
         <t>High availability and redundancy support continuous operation during long-running
            HPC tasks. Networks also need security measures, including encryption and
            secure access controls, to protect sensitive or classified data in transit.</t>

       </section> 

       <section anchor="QOS" numbered="true" toc="default">
        <name>Quality of Service</name>

          <t>Depending on provisioning and resource contention, the network may need to support Quality of Service (QoS) mechanisms to prioritise
            traffic so critical HPC tasks receive the necessary bandwidth and low-latency performance.</t>          

         <t>Applications may need a way to request specific bandwidth or latency guarantees
            for high-priority tasks. These guarantees are hard to provide over shared multi-domain infrastructure.</t>
            
         <t>Differentiated Services (Diffserv) offers a flexible method to manage traffic prioritisation without the 
            need for an explicit request-and-grant process. Diffserv
            marks packets with different priority levels, allowing the
            network to prioritise and protect capacity for critical
            tasks. 
            This approach may be useful in HPC environments where dynamic traffic patterns require adaptive resource management.</t>

       </section> 

       <section anchor="CC" numbered="true" toc="default">
        <name>Congestion Control</name>
            
         <t>Congestion control adjusts the offered load to the capacity available along
            a path. Operators need to distinguish congestion from other causes of packet
            loss and check whether sustained transfers build queues or leave capacity
            unused. Where several transfers share a bottleneck, throughput also depends
            on how they share that capacity.</t>
            
          <t>Depending on the transport technology used in the HPC environment, several congestion control
             schemes may be used:</t>

         <ul spacing="normal">
           <li>InfiniBand Congestion Control</li>
           <li>RDMA-based Data Center Quantized Congestion Notification (DCQCN)</li>
           <li>TCP-based Bottleneck Bandwidth and Round-Trip Time (BBRv3)</li>
           <li>Explicit Congestion Protocol (XCP)</li>
         </ul>   

    
       </section> 
       
       <section anchor="PERF" numbered="true" toc="default">
        <name>Performance Monitoring</name>

         <t>End-to-end performance measurement and monitoring across multi-domains and network infrastructures are 
            important in HPC environments. They provide a way to diagnose and troubleshoot network performance 
            issues that can affect data-intensive applications and distributed
            computing tasks common in HPC.</t>
            
          <t>perfSONAR is a network measurement toolkit commonly used in this space. It is designed to provide federated coverage 
             of network paths. It provides an interface for scheduling
             measurements, storing data, and generating visualisations.</t>
            
         <t>Data transfer applications should log their transfers such that monitoring tools can assess their end-to-end, disk-to-disk performance. 
            e.g., FTS does this for WLCG data transfers.</t>

       </section> 

       <section anchor="SCAL" numbered="true" toc="default">
        <name>Scalability</name>

         <t>The network needs to accommodate additional sites or increased capacity as
            computational needs grow, without significant reconfiguration. Interoperability
            across the different hardware, software, and protocols used at HPC sites is also necessary.</t>

       </section> 

       <section anchor="NRG" numbered="true" toc="default">
        <name>Sustainability and Energy Efficiency</name>

        <t>The scale of HP-WANs makes energy consumption an operational consideration. Network component efficiency and power management affect both operating costs and the environmental impact of data transfers.</t>

        <t>Operators can consider reducing power consumption during periods of lower utilisation, subject to performance and availability requirements. Energy use needs to be measured to assess the effect of such changes.</t>
 
       </section>



       <section anchor="SCHD" numbered="true" toc="default">
        <name>Resource Scheduling</name>

          <t>Some HP-WAN environments may also require service and resource scheduling to coordinate data transfer windows, reserved capacity, and compute availability across administrative domains. This is especially relevant when large transfers are planned in advance and must align with application workflows.</t>

          <t>HPC and AI workload schedulers commonly know job timing, placement, data-movement, and completion requirements. Network reservation and orchestration platforms need a common way to consume the network-relevant subset of that information without adopting each scheduler's native data model. The emerging workload-to-network models described in <xref target="HPC-AI-MODELS"/> address this gap.</t>

       </section> 
       
    </section>


    <section anchor="EXMP" numbered="true" toc="default">
      <name>Examples of HP-WANs</name>

      <t>The following sub-sections highlight examples of HP-WANs and
          their technical specifications.</t>

       <section anchor="GEANT" numbered="true" toc="default">
        <name>GÉANT</name>

        <t>The GÉANT network is a pan-European data network dedicated to research and education, providing 
           high-speed, high-capacity connectivity across Europe, between European NRENs and to other worldwide NRENs. 
           It is an essential infrastructure for HPC applications, enabling collaboration and data sharing among research institutions,
           universities, and HPC centres across the continent and beyond.</t>
           
        <t>The core of GÉANT operates at speeds of up to 600 Gbps, using Dense Wavelength Division Multiplexing
           (DWDM) technology. This provides connectivity suitable for HPC applications, particularly those 
           involving large-scale simulations, scientific research, and real-time data processing. Reliability is 
           provided by using multiple optical underlay paths for data to travel between GÉANT nodes. The alternate paths support continued operation 
           following a path failure.</t>   
        
        <t>The GÉANT network integrates perfSONAR for real-time network performance monitoring and reporting of IP performance metrics <xref target="RFC6703"/>, allowing HPC users 
           to detect and troubleshoot potential issues that could impact data transfer and overall performance. These measurements help operators assess whether 
           the network meets HPC application requirements.</t>
        
        <t>GÉANT provides specialised services for specific HPC projects, such as the LHC Optical Private Network (LHCOPN) 
           and LHC Open Network Environment (LHCONE), which are critical for supporting the data-intensive needs of the 
           Large Hadron Collider (LHC) at CERN. These services offer dedicated, high-bandwidth connections that are 
           optimised for the massive data flows generated by LHC experiments.</t>   
           
        <t>The GÉANT network connects over 50 million users across more than 10,000 institutions in 40 countries. This 
           extensive reach supports a wide range of HPC applications by connecting 
           geographically dispersed research facilities. Beyond Europe, GÉANT connects to other major research and 
           education networks, including Internet2 in the United States and CANARIE in Canada, allowing for global 
           HPC collaborations and data exchanges.</t>   

       </section> 

       <section anchor="Janet" numbered="true" toc="default">
        <name>Janet</name>

        <t>The Janet network is the UK NREN, operated by Jisc. First established in 1984, backbone links now run at up 
           to 800Gbps, with a growing number of sites connected at 100Gbps, in some cases with multiple 100G links. A 
           typical university site will have multiple 10G links.</t>
           
        <t>Janet connects to other RE networks via a 400G resilient link to GÉANT. It has a presence in multiple IXes, 
           predominantly LINX, connects/peers directly to many content and cloud providers, and has commodity connectivity 
           via Tier1 ISPs.  The total aggregate external capacity is around 4-5 Tbit/s.</t> 
        
        <t>Some private, dedicated links are used by Janet sites, e.g., the CERN to RAL (UK Tier 1 site) LHCOPN 
           link, which is a dedicated 200G path. While LHCONE is a L3VPN that may be on its own circuit or as an overlay on general RE paths. Otherwise, 
           Jisc seeks to provide sufficient capacity to its science communities ahead of demand, and works with them (via its network performance team) 
           to provide advice and guidance on how sites can optimise their use of the Janet network, e.g., by following Science DMZ principles.</t>

       </section> 

      <section anchor="Effingo" numbered="true" toc="default">
        <name>Google Effingo</name>

        <t>Google Effingo is a managed data copy service that transfers data between Google's clusters.
           It supports Google's internal services and moves data on behalf of Google Cloud customers.
           The service uses Google's private WAN for large-scale data movement.</t>
           
        <t>Effingo separates copy request orchestration from data transfer. It adjusts transfer parallelism
           to the available resources and shares capacity between competing copies. Workloads include
           batch transfers, replication, and latency-sensitive copies.</t>
           
        <t>Google's IETF 121 presentation describes operation across more than 100 data centre sites and
           a daily transfer volume of about one exabyte <xref target="EFFINGO"/>.</t>

       </section>


       <section anchor="ESNET" numbered="true" toc="default">
        <name>Energy Sciences Network</name>

        <t>The Energy Sciences Network (ESnet), established in 1986, serves scientific research within the United States and is operated by the U.S. Department of Energy (DOE). It connects national laboratories, supercomputing centres, universities, and research institutions for data-intensive applications and distributed HPC tasks.</t>

        <t>ESnet uses a fibre-optic backbone and dynamic circuit provisioning services to provide high-capacity, low-latency connectivity. Data transfer rates range from tens to hundreds of gigabits per second for scientific workflows such as high-energy physics experiments, climate modelling, and genomic research.</t>
        
        <t>The On-Demand Secure Circuits and Advance Reservation System (OSCARS) allows researchers to reserve guaranteed-bandwidth paths for their projects. SENSE provides intent-driven orchestration and automated network-resource management.</t>
        
        <t>ESnet uses monitoring and diagnostic tools such as perfSONAR to measure end-to-end performance across institutional boundaries. Operators can use these measurements to identify performance bottlenecks.</t>
        
        <t>ESnet interconnects with GÉANT, Janet, Internet2, and CANARIE for international research data exchange.</t>        

       </section> 

       <section anchor="I2" numbered="true" toc="default">
        <name>Internet2</name>

        <t>Internet2 is a networking consortium established in 1996 to serve the United States research and education community. Its infrastructure supports collaboration among educational institutions, government laboratories, and industry partners.</t>

        <t>Its optical backbone supports multi-terabit speeds and low-latency connections. Internet2 also supports dynamic bandwidth allocation, network monitoring, and federated identity management.</t>
        
       </section> 

       <section anchor="CN" numbered="true" toc="default">
        <name>CANARIE</name>

        <t>Established in 1993, CANARIE connects universities, research centres, healthcare institutions, and government laboratories across Canada. Canada's national research and education network also has international interconnections with GÉANT, Internet2, and ESnet.</t>
 
        <t>CANARIE operates a high-capacity fibre-optic backbone with dedicated high-speed links, automated resource allocation, and integrated identity and access management. Its Digital Accelerator for Innovation and Research (DAIR) service supports cloud-based research and development.</t>
        
       </section> 
 
        <section anchor="CERNET" numbered="true" toc="default">
        <name>CERNET</name>

        <t>CERNET (China Education and Research Network) links universities, research institutions and HPC centres across China. It provides high-speed, secure connectivity for research data sharing and HPC applications, with connections to international research networks.</t>
 
        <t>CERNET's backbone operates at up to 100 Gbps (with ongoing upgrades) using DWDM and SDN for HPC workloads such as scientific simulations, climate modelling and high-throughput data processing. Its redundant topology provides alternate paths to limit disruption following a failure.</t>
        
        <t>CERNET offers customised services for major national projects, including dedicated high-bandwidth connections for the China Spallation Neutron Source (CSNS) and national supercomputing centres (e.g., Sunway TaihuLight). These support data-intensive workloads and data exchange.</t>

        <t>CERNET covers 31 Chinese provinces and connects over 200 million users across more than 2,000 universities, institutes, and schools. International peers include GÉANT (Europe) and Internet2 (U.S.).</t>

       </section> 
       
        <section anchor="CM" numbered="true" toc="default">
        <name>China Mobile Cloud Dedicated Network</name>

        <t>China Mobile Cloud Dedicated Network is a high-performance dedicated service for enterprises, governments, and industrial users. It has built a large-scale SRv6 policy-based cloud dedicated network, offering secure, low-latency connectivity between on-premises data centres, branches, and its public cloud, supporting applications such as real-time industrial control and large-scale data analytics.</t>
       
        <t>It covers 31 provinces and over 300 cities, with nearly 50 Tbps of inter-provincial bandwidth and more than 800 SRv6 policy-enabled devices. Its backbone uses Segment Routing (SR); some backbone links support 400 Gbps OTN to handle high-bandwidth demands. The network uses a multi-path redundant architecture.</t>
       
        <t>It integrates a management platform for real-time monitoring of latency, jitter, and bandwidth, and connects tens of thousands of enterprise users domestically to ease inter-office collaboration and cloud migration. Internationally, it links to global cloud providers and overseas data centres for secure cross-border data transmission.</t>
        </section> 
 
    </section>
    
    <section anchor="FUT" numbered="true" toc="default">
      <name>Emerging Trends and Future Directions</name>

        <t>Scientific research, distributed AI, and industrial data analytics place
           further demands on HP-WAN capacity and service scheduling. The following
           sections discuss approaches to coordinating applications and network resources.</t>

        <t>While these capabilities may not yet be widely deployed on RE infrastructures, the NREN operator community is clearly interested in tracking their development and readiness for production deployment.</t>

        <section anchor="NETCONT" numbered="true" toc="default">
        <name>Integrated Resource and Network Control</name>

         <t>Resource controllers and network controllers can coordinate capacity
            reservations with application schedules. Sharing resource availability and
            timing requirements allows them to allocate bandwidth when an application
            needs it and release it when the transfer completes.</t>
     
         <t>For example, a typical trend for HP-WANs is to satisfy the needs of distributed or decentralised AI, using so-called "scale-across" networks between multiple large data centres over long distances. The integration of network resources and applications may need to consider underlying resources, such as the optical transport layer <xref target="HIC-OTN"/>, routing policies configured by network controllers, and service requests. This integration requires coordination among the optical resource management platform, the network controller, and the service orchestrator.</t>
     
       </section>

        <section anchor="HPC-AI-MODELS" numbered="true" toc="default">
        <name>HPC/AI Workload-to-Network Models</name>

          <t>The analysis of current HP-WAN technologies and operational practices identified a standards gap between the platform-specific information held by HPC and AI workload schedulers and the network mechanisms used to realise an HP-WAN service. A set of complementary models has been proposed to provide a common information flow from scheduler and job metadata, through technology-independent service intent, to network tunnel realisation.</t>

          <t>The HPC/AI Scheduler Job Metadata Model <xref target="I-D.xkk-rtgwg-hpc-scheduler-job-metadata"/> normalises the network-relevant job, workload, scheduler, tenant, timing, and task metadata available from heterogeneous workload managers and orchestration platforms.</t>

          <t>The HPC/AI Service Intent Model <xref target="I-D.xkk-rtgwg-hpc-service-intent"/> uses that context to express endpoints, communication patterns, timing, performance, data-movement, policy, and admission requirements as a technology-independent desired network outcome.</t>

          <t>The HPC/AI Service Intent Tunnel Realization Model <xref target="I-D.xkk-teas-hpc-tunnel-realization"/> correlates an admitted service intent with its network realisation state, including tunnels, paths, policies, resources, lifecycle, protection, admission outcome, and performance monitoring. Together, the models form the following information flow:</t>

          <artwork><![CDATA[
scheduler/job metadata -> HP-WAN service intent -> tunnel realization
          ]]></artwork>

          <t>Operational platforms such as ESnet's SENSE and OSCARS, and comparable systems operated by other NRENs, illustrate the scheduling, reservation, and orchestration environments in which common models could be used. Such platforms could consume, produce, or map to these models to exchange network-relevant workload information and correlate requested service outcomes with their realisation. This does not imply that any named platform has implemented these models or demonstrated interoperability with them.</t>

       </section>

        <section anchor="IBN" numbered="true" toc="default">
        <name>Intent-Based Networking and Automation</name>

         <t>Intent-based networking (IBN) and automation technologies are increasingly used to manage and orchestrate HP-WANs. IBN allows network administrators to define desired network states or outcomes, with automated systems translating these intents into actionable network configurations. Platforms such as ESnet's SENSE demonstrate intent-driven orchestration for resource allocation.</t>
        
       </section> 

        <section anchor="SIG" numbered="true" toc="default">
        <name>Network Signalling</name>

         <t>As the scale and complexity of HP-WAN deployments grow, efficient signalling mechanisms become increasingly critical, especially when running HP-WAN services over shared public infrastructure.</t>
 
         <t>Applications may want to signal their desired bandwidth to the network, enabling more precise rate negotiation and collaborative congestion control to achieve a targeted completion time for the data transfer.</t>

         <t>Therefore, efficient and scalable signalling approaches are vital for dynamic resource allocation in HP-WAN environments. Effective protocols must support rapid dissemination of resource states and swift propagation of requests between network components, minimising latency and overhead.</t>

         <t>Signalling mechanisms need to be extensible across different technologies, with low overhead and timely delivery of requests and resource updates. They also need to handle failures reliably.</t>
                
       </section> 

    </section>


    <section anchor="IANA" numbered="true" toc="default">
      <name>IANA Considerations</name>

        <t>This document makes no requests for action by IANA.</t>

    </section>


    <section anchor="SEC" numbered="true" toc="default">
      <name>Security Considerations</name>

      <t>HPC networks, particularly those connecting data centres, need to protect the integrity,
         confidentiality, and availability of sensitive data and computational resources.
         Security requirements depend on the data being processed, for example in defence
         and pharmaceutical research or climate science.</t>

      <t>This document surveys existing networks and techniques and does not
         define a protocol or data model. It introduces no security
         considerations beyond those of the technologies it describes.</t>

     </section>


    <section anchor="ACK" numbered="true" toc="default">
      <name>Acknowledgements</name>

      <t>This document was partly motivated by discussions on the IETF hp-wan@ietf.org mailing list.</t> 
         
      <t>The authors thank Gorry Fairhurst and Zahed Sarker for their reviews and suggestions.</t>

    </section>

   <section anchor="contributors" numbered="false" toc="default">
      <name>Contributors</name>

      <t>The following authors contributed significantly to this document:</t>
      <t>Nicholas Race, Lancaster University, United Kingdom. Email: n.race@lancaster.ac.uk</t>
    </section>


  </middle>

  <back>

    <references>
    
      <name>Informative References</name>

      <xi:include href="https://bib.ietf.org/public/rfc/bibxml3/reference.I-D.xkk-rtgwg-hpc-scheduler-job-metadata.xml"/>
      <xi:include href="https://bib.ietf.org/public/rfc/bibxml3/reference.I-D.xkk-rtgwg-hpc-service-intent.xml"/>
      <xi:include href="https://bib.ietf.org/public/rfc/bibxml3/reference.I-D.xkk-teas-hpc-tunnel-realization.xml"/>
      &RFC5040;
      &RFC5041;
      &RFC5042;
      &RFC5043;
      &RFC5044;
      &RFC6580;      
      &RFC6581;
      &RFC6703;     
      &RFC7306;          
      <reference anchor="EFFINGO" target="https://datatracker.ietf.org/meeting/121/materials/slides-121-hpwan-4-high-volume-data-mover-google-effingo-00.pdf">
        <front>
          <title>An exabyte a day: throughput-oriented, large scale, managed data transfers with Effingo</title>
          <author initials="M." surname="Zasadziński" fullname="Michał Zasadziński">
            <organization>Google</organization>
          </author>
          <date year="2024" month="November"/>
        </front>
      </reference>

      <reference anchor="HIC-OTN" target="https://ieeexplore.ieee.org/abstract/document/10926085">
        <front>
          <title>Decentralized Training over 100km Based on Optical Transport Network for Artificial Intelligence</title>

          <author fullname="Jiang Sun" surname="Sun">
            <organization>China Mobile</organization>
          </author>

          <date year="2024"/>
        </front>
      </reference>       

          <reference anchor="UEC" target="https://ultraethernet.org/uec-1-0-spec">
        <front>
          <title>Ultra Ethernet Specification v1.0.1</title>

          <author fullname="Ultra" surname="Ethernet">
            <organization>Ultra Ethernet</organization>
          </author>

          <date year="2025"/>
        </front>
      </reference>       
    </references>

  </back>

</rfc>
