]> ZTE Corporation

fu.huakai@zte.com.cn

China Mobile

liubo@chinamobile.com

China Mobile

lizhenqiang@chinamobile.com

ZTE Corporation

xiong.quan@zte.com.cn

CATS This document describes the OAM framework and requirements for Computing-Aware Traffic Steering (CATS). The framework defines the CATS OAM layering model and OAM components. It also describes the requirements to enable the fault and the performance management of end-to-end connections from clients to networks and finally to services instances.

Introduction As described in , edge computing provides lower response time and higher transmission rate than cloud computing by moving computing instances to the network edge. To meet the requirements of users that are highly distributive, service providers deploy the same type of service instances at multiple edge sites, which involves steering traffic from clients to the most appropriate computing instance. Compute-aware traffic steering (CATS) [I-D.ietf-cats-framework] is a traffic engineering approach as per developed to address the aforementioned traffic steering problem. This approach takes into account the dynamic nature of both the computing resources and the network states to optimize the way that traffic is forwarded towards a given service instance. Various metrics can be taken into account to devise and enforce such service-specific and computing-aware traffic steering policies.To achieve better service assurance, it is necessary to not only rapidly detect whether the QoS provided by the computing networks meets the SLA requirements of clients, but also dynamically trigger the calculation and the adjustment of both the computing and the networking services. There are some OAM technologies developed for networks, but they are only deployed to facilitate the operations and the maintenance of network operators, and cannot provide measurements of an end-to-end connection from a client to a service instance. To this end, based on the CATS framework as per [I-D.ietf-cats-framework], this document describes the OAM framework and requirements for Computing-Aware Traffic Steering (CATS). The framework defines the CATS OAM layering model and OAM components. It also describes the requirements to enable the fault and the performance management of end-to-end connections from clients to networks and finally to services instances.The deployment considerations are also described as well.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology This document makes use of the terms defined in [I-D.ietf-cats- framework].

• FM: Fault Management.
• PM: Performance Monitoring.
• SI-OAM: Service Instance OAM.
• TC-OAM: Traffic Classifier OAM.
• AF-OAM: Application Flow OAM.
• IOAM: In-situ OAM.

Motivation The main objectives of OAM are to detect anomalies before they intensify, reduce the number of traffic flows impacted by these abnormalities, and ensure that network operators fulfill their QoS guarantee commitments to meet the Service Level Agreement(SLA) of clients. As a traffic engineering method, computing-aware traffic steering (CATS) takes into account the dynamic nature of both the computing resources and the network states to optimize the way that traffic is forwarded toward a given service instance. However, existing OAM technologies cannot be used to collect metrics associated with the computing resources. Therefore, it is necessary to extend the existing OAM technologies to build an end-to-end OAM for CATS. Key objectives include:

• Convergence latency is compressed from the order of tens of seconds to sub-second timescale: In CATS,the status information of the computing instances is collected by the CATS Service Metric Agent (C-SMA) component and processed at the control plane for performance monitoring and failure detection. However, to limit control-plane load, such sensing mechanisms are typically engineered to operate on the order of tens of seconds.. Accordingly, rapid detection of data-plane degradation affecting both service instances and network states is mandatory, so that CATS Path Selector (C-PS) convergence is triggered and its latency compressed from tens of seconds to sub-second scale.
• Closed-loop network path evaluation : In CATS, the CATS Path Selector (C-PS) calculates and selects the paths towards appropriate egress PEs and computing service instances. In this process, it is necessary to verify whether the calculation and the selection results meet the SLA requirements of clients taking into account both the network states and the computing instance status.
• Closed-loop service SLAs guarantee for flows : In CATS, subsequent packets of service flows in an established session are forwarded through the CATS Traffic Classifier (C-TC) to the same service instance. However, during such a process, the computing/network performance may degrade. To ensure consistent experience for end users, it is necessary to measure the flow-level performance of service instances and make appropriate adjustments, e.g., change segments of routing paths or enable backup paths, according to the SLA requirements.
• Fault delimiting and troubleshooting: When user experience deteriorates, it is necessary to rapidly locate the fault on the end-to-end path from the user terminal through the network to the service instance to implement fast end-to-end fault location and troubleshooting.

CATS OAM Framework

CATS OAM Layering Model The CATS OAM layering model is shown in Fig. 1. In this architecture,both the CATS router and the underlay node are deployed with the existing OAM technologies.These OAM technologies are used to detect anomalies and monitor service performance in the network domain, and can be divided into three categories: link OAM, tunnel OAM, and service OAM.

• In link OAM, anomaly detection and performance monitoring are conducted for a single ethernet link. The link layer is an optional sublayer implemented in the data link layer between the Logical Link Control (LLC) and the MAC sublayer in the Open Systems Interconnection (OSI) model. Common detection tools of link OAM include IEEE-802 .3ah.
• A tunnel bears multiple services so the tunnel OAM must ensure that the performance of a given service is not degraded when the network fails or the number of services in the tunnel increases. As a result, failure detection and performance monitoring are conducted on the LSP layer to implement service protection.Common detection tools of tunnel OAM include ITU-T Y.1711, MPLS-LM-DM, BFD, etc.
• Service OAM is generally conducted for the L2VPN/L3VPN service layer that is provided by the network to evaluate the service quality and protect services. Common detection tools of service OAM include ITU-T Y.1731, TWAMP, STAMP, etc.

CATS simultaneously steers traffic along network paths and toward compute instances. Within the network domain the three conventional OAM mechanisms remain applicable, yet link-level OAM can at best cover the direct link between compute instances; no effective OAM exists from the ingress/egress gateways to the compute instances themselves. Moreover, the introduction of flow-affinity policies mandates that end-to-end quality assessment of service flows span both network and compute domains.

CATS OAM Components The CATS OAM layering model should flexibly support existing OAM detection tools and it consists of the following three components, SI-OAM, TC-OAM and AF-OAM as Figure 2 shown. |     |    +--+-----+                         +--------+    |     |    | TC_OAM |<------------------------------------->|     |    +--+-----+                                       |     |       |                                             |     |    +--+-----+                                       |     +----+ AF_OAM |<------------------------------------->|          +--+-----+                                        Figure 2: CATS OAM Functional Components ]]>

SI-OAM Component The functions of this component include (but are not limited to) detecting the failures that happen between the CATS-Forwarder 2 and the service instance, and measuring the associated metrics such as latency, packet loss, and bandwidth.The SI-OAM component generally would not dive into the internal structure of the network between the CATS-Forwarder 2 and the service instance and only makes the measurements of the end-to-end connection. These measurements are generally fed back to the C-SMA component to achieve faster failure detection and performance monitoring than the CATS control plane.

TC-OAM Component The functions of this component include but are not limited to detecting the failures that happen between the CATS-Forwarder 1 and the service instance of a certain specific ID, and measuring the associated metrics such as delay and packet loss. The testing packets are delivered through the CATS Path Selector (C-PS) to the associated service instance according to the corresponding forwarding table entry of the CATS Traffic Classifier (C-TC) to verify whether the measurements of the connection meet the service level agreement (SLA) requirements. And if it does not, recalculation is triggered.

AF-OAM Component The functions of this component include but are not limited to measuring the metrics such as delay, packet loss, and bandwidth, of the service flow in CATS. In general, the user experience of an active connection may be affected by a number of factors, such as the processing latency of the service instances may increase or the network performance may degrade due to the increase of the incoming traffic to the service instance. For CATS-Forwarder 1, it is necessary to evaluate whether the SLA requirements of service flows are achieved, and if the SLA requirements are not achieved, conduct appropriate path adjustments to compensate for the deviation as much as possible to ensure the clients have consistent experience. For client terminals, if the experience is degraded, it is necessary to accurately locate where the problem occurs and quickly conduct troubleshooting. It should be noted that related OAM tools can also be developed, so that the entire network stack (L2-L7) can be observed for applications and the entire network stack,instead of merely traditional application-level visibility or network-level visibility, providing a comprehensive solution for operators' efficiency.

CATS OAM Requirements

Operation

• Sub-second/second-granularity telemetry SHALL be collected for CPU, GPU, memory, accelerator utilization and energy consumption to produce unified compute metrics (e.g., TOPS/W, TFLOPS).
• These metrics SHALL be fused with network telemetry to generate an integrated “compute-network” telemetry stream encompassing packet loss, latency, throughput and compute load, providing real-time decision inputs to the C-PS.

Administration

• Compute-resource provisioning: A node SHALL present a compute-capability template (type, capacity, affinity) at boot; OAM SHALL authenticate the template and synchronize it to the network-wide routing database.
• Service contract and billing: OAM SHALL generate a billing model from multi-dimensional factors—compute class, usage duration, network distance—and push the model to edge controllers.
• Unified orchestration: OAM SHALL abstract compute workloads into routable Compute-SIDs and, together with network SIDs, inject them into the SRv6/BGP SR Policy orchestration plane to enable resource scheduling across domains, clouds, and edges.

Maintenance

• End-to-end quality assessment:1）Network segment: Employ BFD, TWAMP and IOAM to detect link/node faults; convergence latency SHALL be ≤ 50 ms.2）Compute segment: Utilize keep-alive plus health probes to monitor container/VM/accelerator liveness; crashes or overload SHALL be detected within seconds.
• Fault correlation and localization: OAM SHALL correlate “compute unavailable” events with “network-path degradation” events to determine whether the root cause is resource exhaustion or packet loss, eliminating needless path shifts.
• Intelligent self-healing: 1）Compute-node failure SHALL trigger the CATS Path Selector to re-select a path and move traffic in real time to an alternate node in the same or a remote pool. 2）Network-link failure SHALL invoke TI-LFA/SR-TE protection switching within < 50 ms while simultaneously updating the compute topology to prevent black-holing.

Deployment Considerations To demonstrate the complete CATS OAM procedure, a proper OAM detection tool needs to be selected and deployed on the network and service instance hosts of the CATS OAM architecture. The selection of OAM detection tools is out of the scope of this document. |S-ID 2  | +---+----+  |  |+---+--+|           OWAMP        |   +------+|       |SI-ID 1 |     |       |  ||TC-OAM|+------------------------+-----------+------>|        |     |       |  |+------+|                        |           |       |S-ID 2  |     |       | ++-------+|           IOAM         |           |       |SI-ID 2 |     |       | | AF-OAM |+------------------------+-----------+------>|        |     |       | +--------+|           IOAM         |           |       |        |     +-------+-----------+------------------------+-----------+------>|        |             +-----------+                        +-----------+       +--------+ Figure 3: An Example Of CATS OAM Deployment ]]> As illustrated in Fig. 3, the OWAMP and the IOAM tools are selected as examples to describe how the CATS OAM component works with these detection tools to fulfill the four objectives :

• Convergence latency is compressed from the order of tens of seconds to sub-second timescale: The SI-OAM component is deployed on the CATS-Forwarder 2 and the OWAMP tool is used to measure the delay and packet loss from the CATS-Forwarder 2 to the associated service instance. The source and the destination IP of the detection packets are the CATS-Forwarder 2 interface IP and the service instance IP, respectively.According to the returned packets, the status and the metrics of both the service instance and the network that connects the service instance with the clients are obtained. The SI-OAM component feeds back the measurement results to the C-SMA component, which further spreads the computing resource information in the CATS network to accelerate CATS Path Selector(C-PS) convergence to avoid black holes.
• Closed-loop network SLA guarantee: The TC-OAM component is deployed on the CATS-Forwarder 1 and the OWAMP tool is used to measure the delay and packet loss from the CATS-Forwarder 1 to the associated service instance. To ensure OWAMP packets are delivered according to the table item of TC, the source and the destination IP addresses of the detection packets are set to the IP address of the interface of CATS-Forwarder 1 and the IP address corresponding to the service ID, respectively. OWAMP packets usually pass through the tunnel to the egress network and are forwarded to the service instance. According to the returned OWAMP packets, the TC-OAM obtains the measurement results and feeds back the results to the C-PS component. If the measurement results deviate from the expected SLAs, recalculation is triggered to fulfill the closed-loop network SLA guarantee for the service ID.
• Closed-loop SLA guarantee for service flow: for service flows that have been initiated, the flow affinity function is executed to guarantee that subsequent packets reach the same service instance as the first packet. To conduct measuring and performance monitoring for the entire end-to-end flows, the flow-based detection tool such as IOAM is selected and the AF-OAM component is deployed on the CATS-Forwarder 1. Note that the PostCard or the PassPort modes are generally used in the flow-based detection and a centralized collector is required to obtain the measurement results and feed the results back to the C-PS. The network path can be adjusted according to the difference between the OAM measurement results and the SLA requirements to ensure a consistent user experience.
• Service fault delimiting and troubleshooting: For fast delimitation and troubleshooting under user experience degradation, the AF-OAM component can be deployed on a user terminal when a flow detection tool such as IOAM is performed.The IOAM can use the postcard mode and can directly report the location where packet loss or longer delay occurs according to the measurement results obtained by a centralized collector. This is a typical scenario of IOAM, and details are not described herein.

For different detection targets, flexible choices of detection protocols and mechanisms can be made, which will not be elaborated upon here.

Security Considerations To be discussed in future versions of this document.

Acknowledgements To be added upon contributions, comments and suggestions.

IANA Considerations TBD.

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Segment Routing (SR) leverages the source routing paradigm. A node steers a packet through an ordered list of instructions, called "segments". A segment can represent any instruction, topological or service based. A segment can have a semantic local to an SR node or global within an SR domain. SR provides a mechanism that allows a flow to be restricted to a specific topological path, while maintaining per-flow state only at the ingress node(s) to the SR domain. SR can be directly applied to the MPLS architecture with no change to the forwarding plane. A segment is encoded as an MPLS label. An ordered list of segments is encoded as a stack of labels. The segment to process is on the top of the stack. Upon completion of a segment, the related label is popped from the stack. SR can be applied to the IPv6 architecture, with a new type of routing header. A segment is encoded as an IPv6 address. An ordered list of segments is encoded as an ordered list of IPv6 addresses in the routing header. The active segment is indicated by the Destination Address (DA) of the packet. The next active segment is indicated by a pointer in the new routing header. Segment Routing can be applied to the IPv6 data plane using a new type of Routing Extension Header called the Segment Routing Header (SRH). This document describes the SRH and how it is used by nodes that are Segment Routing (SR) capable. Operations, Administration, and Maintenance (OAM) is a general term that refers to a toolset for fault detection and isolation, and for performance measurement. Over the years, various OAM tools have been defined for various layers in the protocol stack. This document summarizes some of the OAM tools defined in the IETF in the context of IP unicast, MPLS, MPLS Transport Profile (MPLS-TP), pseudowires, and Transparent Interconnection of Lots of Links (TRILL). This document focuses on tools for detecting and isolating failures in networks and for performance monitoring. Control and management aspects of OAM are outside the scope of this document. Network repair functions such as Fast Reroute (FRR) and protection switching, which are often triggered by OAM protocols, are also out of the scope of this document. The target audience of this document includes network equipment vendors, network operators, and standards development organizations. This document can be used as an index to some of the main OAM tools defined in the IETF. At the end of the document, a list of the OAM toolsets and a list of the OAM functions are presented as a summary. In situ Operations, Administration, and Maintenance (IOAM) collects operational and telemetry information in the packet while the packet traverses a path between two points in the network. This document provides a framework for IOAM deployment and provides IOAM deployment considerations and guidance. The One-Way Active Measurement Protocol (OWAMP) measures unidirectional characteristics such as one-way delay and one-way loss. High-precision measurement of these one-way IP performance metrics became possible with wider availability of good time sources (such as GPS and CDMA). OWAMP enables the interoperability of these measurements. [STANDARDS-TRACK] Huawei Technologies China Mobile Orange Telefonica Juniper Networks, Inc. This document describes a framework for Computing-Aware Traffic Steering (CATS). Particularly, the document identifies a set of CATS components, describes their interactions, and exemplifies the workflow of the control and data planes. Informative References China Mobile Telefonica Huawei Technologies China Unicom Alibaba Group Distributed computing is a computing pattern that service providers can follow and use to achieve better service response time and optimized energy consumption. In such a distributed computing environment, compute intensive and delay sensitive services can be improved by utilizing computing resources hosted in various computing facilities. Ideally, compute services are balanced across servers and network resources to enable higher throughput and lower response time. To achieve this, the choice of server and network resources should consider metrics that are oriented towards compute capabilities and resources instead of simply dispatching the service requests in a static way or optimizing solely on connectivity metrics. The process of selecting servers or service instance locations, and of directing traffic to them on chosen network resources is called "Computing-Aware Traffic Steering" (CATS). This document provides the problem statement and the typical scenarios for CATS, which shows the necessity of considering more factors when steering traffic to the appropriate computing resource to better meet the customer's expectations. Old Dog Consulting This document describes the principles of traffic engineering (TE) in the Internet. The document is intended to promote better understanding of the issues surrounding traffic engineering in IP networks and the networks that support IP networking, and to provide a common basis for the development of traffic engineering capabilities for the Internet. The principles, architectures, and methodologies for performance evaluation and performance optimization of operational networks are also discussed. This work was first published as RFC 3272 in May 2002. This document obsoletes RFC 3272 by making a complete update to bring the text in line with best current practices for Internet traffic engineering and to include references to the latest relevant work in the IETF.

Contributors ZTE Corporation

huang.guangping@zte.com.cn

ZTE Corporation

huang.cheng13@zte.com.cn

ZTE Corporation

duan.wei1@zte.com.cn