]> Cisco Systems, Inc.

Canada rgandhi@cisco.com

Cisco Systems, Inc.

cfilsfil@cisco.com

Bell Canada

daniel.voyer@bell.ca

Huawei

mach.chen@huawei.com

Colt

Bart.Janssens@colt.net

SPRING Working Group Segment Routing (SR) leverages the source routing paradigm. SR is applicable to both Multiprotocol Label Switching (SR-MPLS) and IPv6 (SRv6) data planes. This document specifies procedure for sending and processing probe query and response messages for Performance Measurement (PM) in Segment Routing networks. The procedure uses the messages defined in RFC 5357 (Two-Way Active Measurement Protocol (TWAMP) Light) for Delay Measurement, and uses the messages defined in this document for Loss Measurement. The procedure specified is applicable to SR-MPLS and SRv6 data planes and is used for both Links and end-to-end SR Policies.

Segment Routing (SR) leverages the source routing paradigm and greatly simplifies network operations for Software Defined Networks (SDNs). SR is applicable to both Multiprotocol Label Switching (SR-MPLS) and IPv6 (SRv6) data planes. SR takes advantage of the Equal-Cost Multipaths (ECMPs) between source and transit nodes, between transit nodes and between transit and destination nodes. SR Policies as defined in are used to steer traffic through a specific, user-defined paths using a stack of Segments. Built-in SR Performance Measurement (PM) is one of the essential requirements to provide Service Level Agreements (SLAs). The One-Way Active Measurement Protocol (OWAMP) defined in and Two-Way Active Measurement Protocol (TWAMP) defined in provide capabilities for the measurement of various performance metrics in IP networks using probe messages. These protocols rely on control-channel signaling to establish a test-channel over an UDP path. The TWAMP Light [Appendix I in RFC5357] provides simplified mechanisms for active performance measurement in Customer IP networks by provisioning UDP paths and eliminates the control-channel signaling. As described in Appendix A of , TWAMP Light mechanism is informative only. These protocols lack support for direct-mode Loss Measurement (LM) to detect actual Customer data traffic loss which is required in SR networks. This document specifies procedures for sending and processing probe query and response messages for Performance Measurement in SR networks. The procedure uses the messages defined in (TWAMP Light) for Delay Measurement (DM), and uses the messages defined in this document for Loss Measurement. The procedure specified is applicable to SR-MPLS and SRv6 data planes and are used for both Links and end-to-end SR Policies. This document also defines mechanisms for handling ECMPs of SR Policies for performance delay measurement. Unless otherwise specified, the messages defined in are not modified by this document.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in when, and only when, they appear in all capitals, as shown here.

BSID: Binding Segment ID. DM: Delay Measurement. ECMP: Equal Cost Multi-Path. HMAC: Hashed Message Authentication Code. LM: Loss Measurement. MPLS: Multiprotocol Label Switching. NTP: Network Time Protocol. OWAMP: One-Way Active Measurement Protocol. PM: Performance Measurement. PSID: Path Segment Identifier. PTP: Precision Time Protocol. SID: Segment ID. SL: Segment List. SR: Segment Routing. SRH: Segment Routing Header. SR-MPLS: Segment Routing with MPLS data plane. SRv6: Segment Routing with IPv6 data plane. TC: Traffic Class. TWAMP: Two-Way Active Measurement Protocol.

In the reference topology shown below, the sender node R1 initiates a probe query for performance measurement and the reflector node R5 sends a probe response for the query message received. The probe response is sent to the sender node R1. The nodes R1 and R5 may be directly connected via a Link or there exists a Point-to-Point (P2P) SR Policy on node R1 with destination to node R5. In case of Point-to-Multipoint (P2MP), SR Policy originating from source node R1 may terminate on multiple destination leaf nodes .

| | | R1 |---------------------| R5 | | |<- - - - - - - - - - | | +-------+ t4 Response t3 +-------+ Sender Reflector Reference Topology ]]>

For one-way, two-way and round-trip delay measurements in Segment Routing networks, the probe messages defined in are used. For direct-mode and inferred-mode loss measurements in Segment Routing networks, the messages defined in this document are used. Separate UDP destination port numbers are user-configured for delay and loss measurements. As specified in , the reflector supports the destination UDP port 862 for delay measurement probe messages by default. This UDP port however, is not used for loss measurement probe messages defined in this document. The sender uses the UDP port number following the guidelines specified in in . For both Links and end-to-end SR Policies, no PM session for delay or loss measurement is created on the reflector node R5 . For Performance Measurement, probe query and response messages are sent as following: For Delay Measurement, the probe messages are sent on the congruent path of the data traffic by the sender node, and are used to measure the delay experienced by the actual data traffic flowing on the Links and SR Policies. For Loss Measurement, the probe messages are sent on the congruent path of the data traffic by the sender node, and are used to collect the receive traffic counters for the incoming link or incoming SID where the probe query messages are received at the reflector node (incoming link or incoming SID needed since the reflector node does not have PM session state present). The In-Situ Operations, Administration, and Maintenance (IOAM) mechanisms for SR-MPLS defined in and for SRv6 defined in are used to carry PM information such as timestamp in-band as part of the data packets, and are outside the scope of this document.

An example of a provisioning model and typical measurement parameters for each user-configured destination UDP port for performance delay and loss measurements is shown in the following Figure 1:

Example of Measurement Protocol is TWAMP Light, the Timestamp Format is PTPv2 or NTP and the Loss Measurement mode is inferred-mode or direct-mode. The mechanisms to provision the sender and reflector nodes are outside the scope of this document. The reflector node R5 uses the parameters for the timestamp format and delay measurement mode (i.e. one-way, two-way or loopback mode) from the received probe query message.

The probe messages defined in are used for Delay Measurement for Links and end-to-end SR Policies. For Loss Measurement, the probe messages defined in this document are used. The Sender IPv4 or IPv6 address is used as the source address. When known, the reflector IPv4 or IPv6 address is used as the destination address. If not known, the address in the range of 127/8 for IPv4 or 0:0:0:0:0:FFFF:7F00/104 for IPv6 is used as destination address. This is the case for example, when using SR Policy with IPv4 endpoint of 0.0.0.0 or IPv6 endpoint of ::0 .

The message content for Delay Measurement probe query message using UDP header is shown in Figure 2. The DM probe query message is sent with user-configured Destination UDP port number for DM. The Destination UDP port cannot be used as Source port, since the message does not have any indication to distinguish between the query and response message. The payload of the DM probe query message contains the delay measurement message defined in Section 4.1.2 of . For symmetrical size query and response messages as defined in , the DM probe query message contains the payload format defined in Section 4.2.1 of .

Timestamp field is eight bytes and use the format defined in Section 4.2.1 of . It is recommended to use the IEEE 1588v2 Precision Time Protocol (PTP) truncated 64-bit timestamp format as specified in , with hardware support in Segment Routing networks.

When using the authenticated mode for delay measurement, the matching authentication type (e.g. HMAC-SHA-256) and key are user-configured on both the sender and reflector nodes. A separate user-configured destination UDP port is used for the delay measurement in authentication mode due to the different probe message format.

The message content for Loss Measurement probe query message using UDP header is shown in Figure 3. The LM probe query message is sent with user-configured Destination UDP port number for LM, which is a different Destination UDP port number than DM. Separate Destination UDP ports are used for direct-mode and inferred-mode loss measurements. The Destination UDP port cannot be used as Source port, since the message does not have any indication to distinguish between the query and response message. The LM probe query message contains the payload for loss measurement as defined in Figure 7 and Figure 8.

When using the authenticated mode for loss measurement, the matching authentication type (e.g. HMAC-SHA-256) and key are user-configured on both the sender and reflector nodes. A separate user-configured destination UDP port is used for the loss measurement in authentication mode due to the different message format.

The probe query message as defined in Figure 2 for delay measurement and Figure 3 for loss measurement is sent on the congruent path of the data traffic. The probe messages are routed over the Link for both delay and loss measurement.

The performance delay and loss measurement for segment routing is applicable to both SR-MPLS and SRv6 Policies.

The probe query messages for end-to-end performance measurement of an SR-MPLS Policy is sent using its SR-MPLS header containing the MPLS segment list as shown in Figure 4.

The Segment List (SL) can be empty to indicate Implicit NULL label case for a single-hop SR Policy. The Path Segment Identifier (PSID) of the SR-MPLS Policy is used for accounting received traffic on the egress node for loss measurement.

An SRv6 Policy setup using the SRv6 Segment Routing Header (SRH) and a Segment List as defined in . For SRv6, network programming is defined in . The probe query messages for end-to-end performance measurement of an SRv6 Policy is sent using its SRH with Segment List as shown in Figure 5.

For delay measurement of SRv6 Policy using SRH, END function END.OTP is used with the target SRv6 SID to punt probe messages on the target node, as shown in Figure 5. Similarly, for loss measurement of SRv6 Policy, END function END.OP is used with target SRv6 SID to punt probe messages on the target node.

The Control Code field is defined for delay and loss measurement probe query and response messages for TWAMP Light in unauthenticated and authenticated modes. The modified delay measurement probe query and response message format is shown in Figure 6. This message format is backwards compatible with the message format defined in as its reflectors ignore the received field (previously identified as MBZ). The usage of the Control Code is not limited to the SR networks and can be used for various bidirectional paths in a network.

Sender Control Code: Set as follows in TWAMP Light probe query message. For a Query: 0x0: Out-of-band Response Requested. Indicates that the probe response is not required over the same path in the reverse direction. This is also the default behavior. 0x1: In-band Response Requested. Indicates that this query has been sent over a bidirectional path and the probe response is required over the same path in the reverse direction. The bidirectional path does not have to be an SR path. Reflector Control Code: Set as follows in TWAMP Light probe response message. 0x1: Error - Invalid Message. Indicates that the operation failed because the received query message could not be processed. Additional Error Codes to be defined in future.

In this document, TWAMP Light probe query message formats are defined for loss measurement as shown in Figure 7 and Figure 8. The message formats are hardware efficient due to the well-known locations of the counters. They are similar to the delay measurement message formats (e.g. location of the Counter and Timestamp) and do not require any backwards compatibility or support for the existing DM message formats from as different user-configured destination UDP port is used for loss measurement.

Sequence Number (32-bit): As defined in . Transmit Counter (64-bit): The number of packets or octets sent by the sender node in the query message and by the reflector node in the response message. The counter is always written at the well-known location in the probe query and response messages. Receive Counter (64-bit): The number of packets or octets received at the reflector node. It is written by the reflector node in the probe response message. Sender Counter (64-bit): This is the exact copy of the transmit counter from the received query message. It is written by the reflector node in the probe response message. Sender Sequence Number (32-bit): As defined in . Sender TTL: As defined in Section 7.1. X: Extended counter format indicator. Indicates the use of extended (64-bit) counter values. Initialized to 1 upon creation (and prior to transmission) of an LM Query and copied from an LM Query to an LM response. Set to 0 when the LM message is transmitted or received over an interface that writes 32-bit counter values. B: Octet (byte) count. When set to 1, indicates that the Counter 1-4 fields represent octet counts. The octet count applies to all packets within the LM scope, and the octet count of a packet sent or received includes the total length of that packet (but excludes headers, labels, or framing of the channel itself). When set to 0, indicates that the Counter fields represent packet counts. Block Number (8-bit): The Loss Measurement using Alternate-Marking method defined in requires to color the data traffic. To be able to compare the transmit and receive traffic counters of the matching color, the Block Number (or color) of the traffic counters is carried by the probe query and response messages for loss measurement. HMAC: The PM probe message in authenticated mode includes a key Hashed Message Authentication Code (HMAC) () hash. Each probe query and response messages are authenticated by adding Sequence Number with Hashed Message Authentication Code (HMAC) TLV. It can use HMAC-SHA-256 truncated to 128 bits (similarly to the use of it in IPSec defined in ); hence the length of the HMAC field is 16 octets. HMAC uses its own key and the mechanism to distribute the HMAC key is outside the scope of this document. In authenticated mode, only the sequence number is encrypted, and the other payload fields are sent in clear text. The probe message may include Comp.MBZ (Must Be Zero) variable length field to align the packet on 16 octets boundary.

The probe response message is sent using the IP/UDP information from the received probe query message. The content of the probe response message is shown in Figure 9.

In one-way performance measurement mode, the probe response message as defined in Figure 9 is sent back out-of-band to the sender node, for both Links and SR Policies. The Sender Control Code is set to "Out-of-band Response Requested". In this delay measurement mode, as per Reference Topology, all timestamps t1, t2, t3, and t4 are collected by the probes. However, only timestamps t1 and t2 are needed to measure one-way delay.

In two-way performance measurement mode, when using a bidirectional path, the probe response message as defined in Figure 9 is sent back to the sender node on the congruent path of the data traffic on the same reverse direction Link or associated reverse SR Policy . The Sender Control Code is set to "In-band Response Requested". In this delay measurement mode, as per Reference Topology, all timestamps t1, t2, t3, and t4 are collected by the probes. All four timestamps are needed to measure two-way delay. Specifically, the probe response message is sent back on the incoming physical interface where the probe query message is received. This is useful for example, in case of two-way measurement mode for Link delay.

The message content for sending probe response message for two-way end-to-end performance measurement of an SR-MPLS Policy is shown in Figure 10.

The Path Segment Identifier (PSID) of the forward SR Policy in the probe query can be used to find the associated reverse SR Policy to send the probe response message for two-way measurement of SR Policy.

The message content for sending probe response message on the congruent path of the data traffic for two-way end-to-end performance measurement of an SRv6 Policy with SRH is shown in Figure 11.

The Loopback measurement mode can be used to measure round-trip delay for a bidirectional SR Path. The IP header of the probe query message contains the destination address equals to the sender address and the source address equals to the reflector address. Optionally, the probe query message can carry the reverse path information (e.g. reverse path label stack for SR-MPLS) as part of the SR header. The probe messages are not punted at the reflector node and it does not process them and generate response messages. The Sender Control Code is set to the default value of 0. In this mode, as the probe packet is not punted on the reflector node for processing, the querier copies the 'Sequence Number' in 'Session-Sender Sequence Number' directly. In this delay measurement mode, as per Reference Topology, the timestamps t1 and t4 are collected by the probes. Both these timestamps are needed to measure round-trip delay.

In this document, TWAMP Light probe response message formats are defined for loss measurement as shown in Figure 12 and Figure 13.

The procedures for delay and loss measurement described in this document for Point-to-Point (P2P) SR Policies are also equally applicable to the Point-to-Multipoint (P2MP) SR Policies as following: The sender root node sends probe query messages using the Replication Segment defined in for the P2MP SR Policy as shown in Figure 14. Each reflector leaf node sends its IP address in the Source Address of the probe response messages as shown in Figure 9. This allows the sender root node to identify the reflector leaf nodes of the P2MP SR Policy. The P2MP root node measures the end-to-end delay and loss performance for each P2MP leaf node of the P2MP SR Policy.

An SR Policy can have ECMPs between the source and transit nodes, between transit nodes and between transit and destination nodes. Usage of Anycast SID by an SR Policy can result in ECMP paths via transit nodes part of that Anycast group. The PM probe messages need to be sent to traverse different ECMP paths to measure performance delay of an SR Policy. Forwarding plane has various hashing functions available to forward packets on specific ECMP paths. The mechanisms described in and for handling ECMPs are also applicable to the performance measurement. In the IP header of the PM probe messages, sweeping of Destination Addresses in 127/8 range for IPv4 or 0:0:0:0:0:FFFF:7F00/104 range for IPv6 can be used to exercise particular ECMP paths. As specified in , Flow Label field in the outer IPv6 header can also be used for sweeping. The considerations for performance loss measurement for different ECMP paths of an SR Policy are outside the scope of this document.

The processing rules defined in this section are applicable to TWAMP Light messages for delay and loss measurement for Links and end-to-end SR Policies.

The TTL field in the IPv4 and MPLS headers of the probe query messages is set to 255 . Similarly, the Hop Limit field in the IPv6 and SRH headers of the probe query messages is set to 255 . When using the Destination IPv4 Address from the 127/8 range, the TTL in the IPv4 header is set to 1 . Similarly, when using the Destination IPv6 Address from the 0:0:0:0:0:FFFF:7F00/104 range, the Hop Limit field in the inner IPv6 header is set to 1 whereas in the outer IPv6 header is set to 255. For Link performance delay and loss measurements, the TTL and Hop Limit field in the probe message is set to 1 in both one-way and two-way measurement modes.

The Router Alert IP option is not set when using the routable Destination IP Address in the probe messages. When using the Destination IPv4 Address from the 127/8 range, to be able to punt probe packets on the reflector node, the Router Alert IP Option of value 0x0 for IPv4 may be added . Similarly, when using the Destination IPv6 Address from the 0:0:0:0:0:FFFF:7F00/104 range, the Router Alert IP Option of value 69 for IPv6 may be added in the destination option header, Section 4.6 of . For SRv6 Policy using SRH, it is added in the inner IPv6 header.

The UDP Checksum Complement for delay and loss measurement messages follows the procedure defined in and can be optionally used with the procedures defined in this document. For IPv4 and IPv6 probe messages, where the hardware is not capable of re-computing the UDP checksum or adding checksum complement , the sender node sets the UDP checksum to 0 . The receiving node bypasses the checksum validation and accepts the packets with UDP checksum value 0 for the UDP port being used for PM delay and loss measurements.

The performance measurement is intended for deployment in well-managed private and service provider networks. As such, it assumes that a node involved in a measurement operation has previously verified the integrity of the path and the identity of the far-end reflector node. If desired, attacks can be mitigated by performing basic validation and sanity checks, at the sender, of the counter or timestamp fields in received measurement response messages. The minimal state associated with these protocols also limits the extent of measurement disruption that can be caused by a corrupt or invalid message to a single query/response cycle. Use of HMAC-SHA-256 in the authenticated mode protects the data integrity of the probe messages. SRv6 has HMAC protection authentication defined for SRH . Hence, PM probe messages for SRv6 may not need authentication mode. Cryptographic measures may be enhanced by the correct configuration of access-control lists and firewalls.

This document does not require any IANA action.

&RFC0768; &RFC2119; &RFC4656; &RFC5357; &RFC8174; &I-D.ietf-6man-spring-srv6-oam; IEEE &RFC2104; &RFC2113; &RFC4868; &RFC5884; &RFC6038; &RFC6335; &RFC6437; &RFC6936; &RFC7506; &RFC7820; &RFC8029; &RFC8085; &RFC8186; &RFC8200; &RFC8321; &RFC8402; &RFC8545; &RFC8754; &I-D.ietf-spring-segment-routing-policy; &I-D.voyer-spring-sr-replication-segment; &I-D.ietf-spring-mpls-path-segment; &I-D.ietf-spring-srv6-network-programming; &I-D.gandhi-mpls-ioam-sr; &I-D.ali-spring-ioam-srv6; &I-D.ietf-pce-sr-bidir-path;

The authors would like to thank Thierry Couture for the discussions on the use-cases for Performance Measurement in Segment Routing. The authors would also like to thank Greg Mirsky for reviewing this document and providing useful comments and suggestions. Patrick Khordoc and Radu Valceanu, both from Cisco Systems have helped significantly improve the mechanisms defined in this document. The authors would like to acknowledge the earlier work on the loss measurement using TWAMP described in draft-xiao-ippm-twamp-ext-direct-loss.