]> Huawei

China zhangli344@huawei.com

Huawei

China zhoutianran@huawei.com

Verizon Inc.

United States of America gyan.s.mishra@verizon.com

OPS Area IPPM Working Group STAMP Multi-Path STAMP is typically used to perform the measurement of one-way and round-trip performance metrics. However, when using active measurement mechanisms in a multi-path topology, the default forwarding behavior is to go through one path. So, it cannot collect the information of all the paths at one time. This document extends STAM with a Multi-path TLV to implement the multi-path performance measurement, which can help the operators to know the performance of network comprehensively and efficiently.

Introduction Simple Two-way Active Measurement Protocol (STAMP) is an active performance measurement test protocol, which enables measurement of both one-way and round-trip performance metrics, like delay, delay variation, and packet loss. Based on that, specifies the use of optional extensions that use Type-Length-Value (TLV) encoding，these extensions enhance the STAMP base functions. extends STAMP to reflect any IPv6 options and MPLS Network Action Sub-Stacks for hop-by-hop and edge-to-edge active measurements. In section 4 of , it provides an example of reflecting IOAM data fields, in which the IPv6 options with IOAM option-types is carried in the STAMP Session-Sender and Session-Reflector test packets. However, currently the STAMP is typically used to collect the information of a specific path, when using it in a multi-path topology (there are multiple paths form the source node to the destination node and ECMP, UCMP or other multi-path routing strategy is used.), the default forwarding behavior is to go through one path. So, it can’t collect all the path’s information form source node to destination node. An example of active measurement in a multi-path topology is shown as follow:

A multi-path topology | N2 | | N7 |---->| N8 | +------+ +------+\ /+------+ +------+ SRC \ +------+ +------+ / DST \| N4 |-------> | N6 |/ +------+ +------+ ]]>

In , node N1 is the source node, node N8 is the destination node, N2-7 are transit node. Equal-Cost Multiple Path (ECMP) is applied in this topology. So, there are two paths form N1 to N8, one is N1-N2-N3-N5-N7-N8, and the other is N1-N2-N4-N6-N7-N8. When N1 use STAMP to measure the path performance, the test packet is forwarded along one of the paths (for example N1-N2-N4-N6-N7-N8), then the source node just can get one path’s information, however the traffic packets are forwarded in all paths. Although the IPv6 flow label and MPLS entropy label can be constructed variously to try to make packets go through all paths, but the hash algorithm cannot ensure that packets can go through all available paths. This document extends STAM with a Multi-path TLV to implement the multi-path performance measurement, which can help the operators to know the performance of network comprehensively and efficiently.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology The abbreviations used in this document are: ECMP: Equal-Cost Multiple Path UCMP: Unequal-Cost Multiple Path STAMP: Simple Two-Way Active Measurement Protocol IOAM: In-band Operation, Administration, and Maintenance

Multi-path TLV This document defines a new TLV in the STAMP test packets:

Multi-path TLV

The fields are defined as follows: STAMP TLV Flags: An eight-bit field. Its format is specified in . Type: A one-octet field. Need to be allocated by IANA. Length: A two-octet field, set equal to the value 4. M: A one-bit field, A Session-Sender MUST set the value of M filed to 1 When need to perform measurement on all paths.

Multi-path Measurement Procedures This section describes the procedures of session sender, transit node, and reflector.

Session-Sender procedures The Session-Sender procedures includes two steps, one is the test packet generation and another is the test packet validation.

Test Packet Generation The test packet generation procedures could be referred from section 4.2 of . The session-sender should generate a Multi-path TLV with the M bit set and encapsulate it into the test packet. An example of the format of STAMP Session-Sender test packet with Multi-path TLV in unauthenticated mode is shown in .

Format of a STAMP Session-Sender Test Packet with Multi-path TLV in Unauthenticated Mode

In order to identify different paths, the Test packet is required to collect the paths’ information. Therefore, the IOAM should be used in combination with STAMP. The Session-sender should insert a an IOAM IPv6 HBH option to the test packet to collect the HBH node information. According to , the Session-sender also need to add “Reflected IPv6 Option Data” TLV in the test packet with the size of the IOAM data field’s length and the value field in the TLV initialized to zeros. In order to perform a round trip for a specific path, the Co-routed Bidirectional Path flag in the Return Path Control Code Sub-TLV of Return Path TLV should be set to 1.

Test Packet Analysis The test packet analysis node could be a Session-Sender or a Controller. According to , the Session-Sender will receive a Session-Reflector test packet with an Reflected IPv6 Option Data TLV. The content of this TLV is copied from the IPv6 option of Session-Sender test packet. defines two kind of Session-Reflector behavior, one is stateless and another is stateful. When using stateless mode, the Session-Sender will receive a test packet and the values in the Sequence Number and Session-Sender Sequence Number fields are the same. The test packet analysis node will analysis the test packet in the following procedures:

• The analysis node determines which test packet the reflected packet belongs to base on the sequence number. The analysis node will receive many reflected test packets with the same Sequence Number.
• For the reflected test packets with same Sequence Number, the analysis node needs to distinguish the different paths by the node information recorded in the IOAM trace Option in Reflected IPv6 Option Data TLV.
• The analysis node needs to generate a table for all the paths and record the sequence number for each path.
• The analysis node gets all the available paths and their packet loss rate by comparing the sequence number and reflected test packets number for each path.
• The analysis node gets the forward and backward transit delay of each path by comparing the Session-Sender Timestamp and Receive Timestamp or Timestamp of each Session-Reflect test packet.

When using stateful mode, the Session-Sender will receive a test packet and the values of the Sequence Number and Session-Sender Sequence Number fields are independent. The test packet analysis node will analysis the test packet in the following procedures:

• The analysis node determines which test packet the reflected packet belongs to base on the Session-Sender sequence number. The analysis node will receive many reflected test packets with the same Session-Sender sequence number.
• For the reflected test packets with same Session-Sender sequence number, the analysis node needs to distinguish the different paths by the node information recorded in the IOAM trace Option in Reflected IPv6 Option Data TLV.
• The analysis node needs to generate a table for all the paths and record the sequence number and Session-Sender sequence number for each path.
• The analysis node gets all the available paths and their round-trip packet loss rate by counting the sequence number for each path.
• The analysis node gets the round-trip packet loss rate by comparing the reflected test packets number and Session-Sender sequence number for each path (the forward and backward packet loss can’t be measured because it requires the Session-Reflector maintains different counter for each path).
• The analysis node gets the forward and backward transit delay of each path by comparing the Session-Sender Timestamp and Receive Timestamp and Timestamp of each Session-Reflect test packet.

Transit node procedures

Packet Operation When the transit node receives a test packet with the IOAM trace option, it needs to add its node ID, ingress interface ID, and egress interface ID to the IOAM trace option of the test packet. For details about the content format, see section 4.4.2 of .

Packet Forwarding When a transit node with multiple paths to the destination node receives a packet with Multi-path bit = 1, it must copy the packet to each egress interface that can reach the destination node. When the transit node has only one path to the destination node, it just needs to forward the packet it received to the egress interface.

Session-Reflector procedures When a Session-Reflector receives a test packet with Multi-path TLV, it MUST copy the entire IPv6 option including the header into the STAMP "Reflected IPv6 Option Data" TLV in Session-Reflector payload, and reset the M bit of Multi-path TLV to prevent the replication in the backward path. The Session-Reflector also need to check whether there is a Return Path Control Code Sub-TLV with Co-routed Bidirectional Path flag set. If the Co-routed Bidirectional Path flag is set, the Session-Reflector Must send the reflect test packet along the path indicated in the IOAM IPv6 HBH option as described in . If the there is no Return Path Control Code Sub-TLV or the Co-routed Bidirectional Path flag is not set, it MUST drop this Multi-path test packet.

Example of Multi-path Measurement An example of a Multi-path measurement scenario is illustrated in . The figure depicts two endpoints: A STAMP Session-Sender and A Session-Reflector. The “STAMP session” is the bidirectional packet flow between the Session-Sender and Session-Reflector. There are four transit nodes and two routing paths between the Session-Sender and Session-Reflector.

Multi-path measurement topology

In , the Session-Sender generate a set of test packet with the IOAM Trace Option, Multi-path TLV (M bit = 1) and Co-routed Bidirectional Path flag set, indicating that these packets are used for multi-path measurement. When the packets arrive at N2, N2 find that there are two paths to the destination node, then it will copy the packet to each outgoing interface of the two paths and add its information (including the node id, ingress interface id and egress interface id) to the IOAM Trace Option. It should be noted that Node Identification, ingoing and outgoing interface Identification of N2 are mandatory for path recovering, other node data defined in section 4.1.1 of are optional. The following transit nodes just add its node data to the IOAM Trace Option as described in section 4 of . When the test packets arrive at Session-Reflector, it will copy the entire IPv6 option including the header into the STAMP "Reflected IPv6 Option Data" TLV in Session-Reflector payload. The Session-Reflector will also reset the Multi-path flag of Multi-path TLV. In addition, the Session-Reflector SHOULD exact all the route path information from the IOAM IPv6 option and encapsulate it in the SRH of reflected test packet, to perform a strict path forwarding. The transit node will forward the reflected test packets along the path indicated in SRH. When the reflected test packets arrive at the Session-Sender, it will analysis these reflected test packets as the procedures illustrated in section 3.1.2. Therefore, it can get the topology with all paths, the round-trip packet loss and the unidirectional path delay.

IANA Considerations This document defines a new bit in the registry "IOAM Trace-Flags" registry as follows:

Value	Description	Reference
Bit X	Multipath Flag	This

Security Considerations TBD

References Normative References This document describes the Simple Two-way Active Measurement Protocol (STAMP), which enables the measurement of both one-way and round-trip performance metrics, like delay, delay variation, and packet loss. This document describes optional extensions to Simple Two-way Active Measurement Protocol (STAMP) that enable measurement of performance metrics. The document also defines a STAMP Test Session Identifier and thus updates RFC 8762. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. In situ Operations, Administration, and Maintenance (IOAM) collects operational and telemetry information in the packet while the packet traverses a path between two points in the network. This document discusses the data fields and associated data types for IOAM. IOAM-Data-Fields can be encapsulated into a variety of protocols, such as Network Service Header (NSH), Segment Routing, Generic Network Virtualization Encapsulation (Geneve), or IPv6. IOAM can be used to complement OAM mechanisms based on, e.g., ICMP or other types of probe packets. Informative References Cisco Systems, Inc. Huawei Simple Two-Way Active Measurement Protocol (STAMP) defined in RFC 8762 and its optional extensions defined in RFC 8972 can be used for Edge-To-Edge (E2E) active measurement. In Situ Operations, Administration, and Maintenance (IOAM) data fields defined in RFC 9197 can be used for recording and collecting Hop-By-Hop (HBH) and E2E operational and telemetry information. This document extends STAMP to reflect any IPv6 options and MPLS Network Action Sub-Stacks for hop-by-hop and edge-to-edge active measurements, including IOAM data fields. Huawei Huawei This document extends STAM Return Path TLV with a Co-routed Bidirectional Path flag to implement the round-trip performance measurement for specific path.

Acknowledgements The authors would like to thank Ernesto Ruffini, Thomas Graf, Greg Mirsky for the valuable comments to this work.