MPLS Network Action (MNA) Sub-Stack Solution

MPLS Network Action (MNA) Sub-Stack Solution Cisco Systems, Inc.

Canada jrajaman@cisco.com

Cisco Systems, Inc.

Canada rgandhi@cisco.com

Broadcom

royi.zigler@broadcom.com

Futurewei Technologies

haoyu.song@futurewei.com

Juniper Networks

United States kireeti.ietf@gmail.com

MPLS Working Group This document defines the MPLS Network Action (MNA) sub-stack solution for carrying Network Actions and Ancillary Data in the label stack. MPLS Network Actions can be used to influence packet forwarding decisions, carry additional OAM information in the MPLS packet, or perform user-defined operations. This document addresses the MNA requirements specified in draft-ietf-mpls-mna-requirements. This document follows the MNA framework specified in draft-ietf-mpls-mna-fwk.

Introduction defines the encoding of the MPLS label stack, the basic structure used to define a forwarding path. Forthcoming applications require MPLS packets to perform special network actions and carry optional Ancillary Data (AD) that can affect the packet forwarding decision or trigger OAM logging. Ancillary Data can be used to carry additional information, such as a network slice identifier or an entropy value for load balancing. Several MNA applications are described in . User-defined network actions allow new, local actions to be defined. This document defines the syntax and semantics of network actions encoded within an MPLS Label Stack. Network actions can be encoded with or without Ancillary Data (AD), either in or after the label stack. In stack actions and ancillary data are contained in a Network Action Sub-Stack (NAS), which is recognized by a new base Special Purpose Label (bSPL) (value TBA). This document addresses the requirements specified in . This document follows the framework specified in .

Conventions Used in This Document

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in when, and only when, they appear in all capitals, as shown here.

Abbreviations The terminology defined in and are used in this document. Abbreviations

Abbreviation	Meaning	Reference
AD	Ancillary Data
bSPL	Base Special Purpose Label
BOS	Bottom Of Stack
HBH	Hop-By-Hop Scope
I2E	Ingress-To-Egress Scope
IHS	I2E, HBH, or Select Scope	This document
ISD	In-Stack Data
LSE	Label Stack Entry
MNA	MPLS Network Actions
NAI	Network Action Indicator
NAL	Network Action Length	This document
NAS	Network Action Sub-Stack
NASI	Network Action Sub-Stack Indicator	This document
NASL	Network Action Sub-Stack Length	This document
OAM	Operations And Management
P	Post-Stack Network Action Presence Indicator	This document
PSD	Post-stack data	and
TC	Traffic Class
TTL	Time To Live

Overview The MPLS Network Action Sub-Stack (NAS) is a set of Label Stack Entries (LSEs) that appear as part of an MPLS Label Stack and serve to encode information about the network actions that should be invoked for the encapsulated packet. Multiple NASs may appear in a label stack and the packet may contain Post-Stack Data, including additional network actions, as specified in . Network actions and their optional Ancillary Data (AD) may be encoded as part of the NAS as a series of LSEs.

Label Stack Entry Formats The NAS uses a variety of different formats of LSEs for different purposes. This section describes the syntax of the various formats while the overall structure of the NAS and the semantics of the various LSEs are described in the sections below.

LSE Format A: The MNA Sub-Stack Indicator LSE Format A is a traditional LSE, as described in and .

LSE Format B: The initial opcode LSE Format B is used to encode the first opcode in the NAS, plus a number of other fields about the NAS.

Opcode (7 bits) : The operation code for this LSE. See .
Data (13 bits) : Opcode specific data
P (1 bit) : If set, it indicates the presence of post-stack network actions.
IHS (2 bits) : The scope of the sub-stack. See .
S (1 bit) : The Bottom of Stack .
Res (2 bits) : Reserved bits. These must be transmitted as zero and ignored upon receipt.
U (1 bits): Unknown Action Handling. See .
O (1 bit) : If set, then the ordering of network actions is semantically significant. See .
NASL (4 bits) : The Network Action Sub-Stack Length (NASL). The number of additional LSEs in the sub-stack, not including the leading Format A LSE and the Format B LSE.

LSE Format C: Subsequent opcodes LSE Format C is used to encode the subsequent opcodes in the NAS.

Opcode (7 bits) : The operation code for this LSE. See .
Data (16 bits + 4 bits) : Opcode specific data
S (1 bit) : The Bottom of Stack .
NAL (4 bits): Network Action Length. The number of LSEs of additional data, encoded in LSE Format D () following this LSE.

LSE Format D: Additional Ancillary Data LSE Format D is used to encode additional ancillary data that did not fit in the LSE with the preceding opcode.

1 (1 bit) : The most significant bit MUST be set. This prevents legacy implementations from misinterpreting this LSE as containing a special label.
S (1 bit) : The Bottom of Stack .
Data (22 bits + 8 bits) : Opcode specific data

The MNA Sub-Stack The MNA Sub-Stack MUST begin with a Format A LSE (). The label field of the LSE contains the MNA bSPL (value TBA) to indicate the presence of the MNA Sub-Stack. The TC and TTL fields of the first LSE retain their traditional semantics, as the penultimate node on the path may copy the TTL and TC fields from the preceding LSE to the next LSE on the label stack, overwriting the TTL and TC fields of the next LSE, as specified in Section 3.5 of . If the node performing this copy is not aware of MNA, this could overwrite the values in the first LSE of the MNA sub-stack. The second LSE in a NAS MUST be a Format B LSE (). This LSE contains an initial opcode plus additional fields that describe the NAS. A NAS MAY contain more Format C () and Format D () LSEs, up to the length encoded in the NASL field. All Format D LSEs MUST follow a Format C LSE and be included in that LSE's NAL field.

Opcodes The opcode is a 7-bit field that indicates the semantics of its LSE. Several opcodes are assigned special semantics (), others act as Network Action Indicators and are allocated through IANA ( and ).

Data The data field carries opcode specific data. This may be ancillary data for a network action. To preserve backward compatibility, if a network action encodes data that will change during packet forwarding, then that data MUST be in the least significant 4 bits in the data field of a Format C LSE () or the least significant 8 bits of a Format D LSE (). Some legacy implementations may use the label field in all LSEs when computing ECMP decisions and modifying the label field might disrupt that packet's flow.

Scope The IHS field in the Format B LSE indicates the scope of the In-Stack and Post-Stack NAIs encoded in the NAS. Scope defines which nodes along the MPLS path should perform the network actions found within the NAS. The specific values of the IHS field are as follows: IHS Scope Values

Bits	Scope
00	I2E
01	HBH
10	Select
11	Reserved

Hop-By-Hop (HBH) - All nodes along the path MUST process the NAS.
Select - Only specific nodes along the path will perform the action.
Ingress To Egress (I2E) - The NAS MUST be processed only by the egress node.

A single NAS carries only one of the three scopes (HBH/Select/I2E). To support multiple scopes for a single packet, multiple NASs may be included in a single label stack. The egress node is included in the HBH scope. This implies that the penultimate node MUST NOT remove the last copy of a HBH NAS. The egress node MAY receive a NAS at the top of the label stack. An I2E scope NAS MUST be encoded after any HBH or Select scope NASs. This makes it easier for the transit nodes to process a NAS with HBH or Select scope. Forwarding and egress nodes should process at most a single NAS per scope.

Unknown Action Handling The Unknown Action Handling (U) field in a Format B LSE () is a 1-bit value that defines the action to be taken by a node that does not understand an action within the NAS. The different types of Unknown Action Handling actions are defined below. Unknown Action Handling

Bit	Action
0	Skip to the next NA
1	Drop the packet

Ordering If the 'O' bit is set in a Format B LSE () then it indicates that the network actions encoded in the NAS MUST be processed in the order that they appear in the NAS, from the top of the NAS to the bottom. NAI encoded as flags MUST be processed from the most significant bit to the least significant bit.

Examples A minimal NAS would have the following format, where the Label field would contain the MNA bSPL and the NASL value would be 0:

A more complex NAS might have multiple opcodes and additional Ancillary Data. This example has two opcodes and two additional LSEs of AD.

In this example, the NASL field would have value 3 and the NAL field would have value 2.

Special Opcodes

bSPL Protection Opcode: 0 Purpose: Legacy implementations may scan the label stack looking for bSPL values. As long as the opcode field is non-zero, an LSE cannot be misinterpreted as containing a bSPL. Opcode 0 is therefore reserved and is not used.

PSD Offset Opcode: 1 Purpose: This opcode carries the start offset of the Post-Stack Action Header (PAH) ( ) within the PSD. LSE Format: B Data: The data value of the LSE contains the offset from the MPLS BOS in units of 4 octets. This allows the Generic Control Word (0000b) and G-ACh (0001b) fields to be placed immediately after the BOS. In the absence of this opcode, the PSD is encoded immediately after the MPLS BOS. A data value of 1 indicates that the PAH starts 4 octets after the BOS. Scope: This opcode can be used with any scope.

Flag-Based NAIs without AD Opcode: 2 Purpose: Network actions that do not require Ancillary Data do not require an entire LSE. A single flag can be used to indicate each of these network actions. LSE Formats: B, C, D Data: The data field carries Network Action Indicators, which should be evaluated from the most significant bit to the least significant bit. If there are sufficient NAI, then Format D LSEs may be used to encode more flags for more network actions. Flags are allocated from the "Network Action Flags Without Ancillary Data" registry (). If flags need to be evaluated in a different order, multiple LSEs using this opcode may be used to specify the requested order. If this opcode is used with LSE Format B, then only 13 flags may be carried. Scope: This opcode can be used with any scope. This opcode MAY be used with no flags set in the data field to signify that no operation is to be performed. This can be used, for example, if the first action to be performed cannot be encoded in a Format B LSE.

Flag based NAIs with AD Opcode: 3 Purpose: This opcode supports flag-based network actions that have Ancillary Data. LSE Formats: C, D Data: The data field carries Network Action Indicators, which should be evaluated from the most significant bit to the least significant bit. Format D LSEs are used to encode the associated Ancillary Data, which appears in the same order as the flags. Flags are allocated from the "Network Action Flags With Ancillary Data" registry (). If flags need to be evaluated in a different order, multiple LSEs using this opcode may be used to specify the requested order. Scope: This opcode can be used with any scope. If a flag contained within this opcode is unknown and is skipped per , then the length of its associated ancillary data will also be unknown. Any subsequent flags within the opcode will not have the correct associated ancillary data, so all subsequent flags SHOULD be treated as unknown actions and also skipped.

PSD-ISD Ordering Opcode: 4 Purpose: In cases where the ordering of network action is significant and where some of the network actions reside in PSD, this opcode can be used to insert PSD network actions into the order of execution. The 'P' bit and 'O' bit MUST be set in the NAS's Format B LSE if this opcode is used. LSE Format: B, C, D Data: The data field contains one or more 8-bit Next Header (NH) indicators . When used with LSE Format B, only one NH indicator is carried. Two indicators MAY be carried in a Format C LSE, and if Format D LSEs are used, each may carry up to three indicators. The indicators are the stored concatenated in the most significant bits of the data field. If multiple indicators are carried, the most significant NH indicator is evaluated to the least significant. Indicators do not span LSEs. If some indicator positions are not to be used, then the indicator should be set to No Next Header (NONE). Scope: This opcode can be used with any scope.

Extension Opcode Opcode: 127 Purpose: This opcode is reserved to extend the current opcode range beyond 127. Future use of this opcode is out of scope.

NAS placement in the Label Stack Regardless of whether packets are being forwarded based on Segment Routing or on RSVP-TE , the node adding an NAS to the label stack will need to place a copy of the NAS where it can be read by the relevant nodes. Each node along the path will have a Maximum MPLS Stack Inspection depth, and if the NAS is to be processed by a particular node, then the entire NAS must be placed so that it is within this depth by the time the packet reaches the node. If the label stack is deep, several copies of the NAS may need to be encoded in the label stack. For a NAS with HBH scope, every node will processes the top copy of the NAS. The node that pops the forwarding label that exposes the NAS MUST NOT remove it. Instead, it forwards the packet with the NAS at the top of stack to the next node (e.g., the segment endpoint node). The node that receives the NAS at the top of the label stack has to remove it. For a NAS with Select scope, it is processed by the node that brings it to the top of stack and then the NAS is removed from the stack. For I2E scope, only one copy of the NAS needs to be added at the bottom of the stack.

Node Capability Signaling The head-end node which is adding a NAS MUST make sure that the egress node removes the NAS. The head-end node MUST make sure that the NAS can be processed by the appropriate transit and egress nodes.

Each participating node MUST signal the network actions that it supports.
Each participating node MUST signal its Maximum MPLS Stack Inspection Depth. This will allow the head-end node to place a copy of an NAS at the correct stack depth.

The above capability signaling will be added in appropriate protocols. Signaling details are outside the scope of this document.

Processing the Network Action Sub-Stack This section defines the specific responsibilities for nodes along a MPLS path.

Encapsulating Node Responsibilities The encapsulating node MAY add NASs to the label stack in accordance with its policies, the placement restrictions in , and the limitations learned from . The encapsulating node MUST NOT add a NAS to the label stack if the decapsulation node does not support MNA. If there is an existing label stack, the encapsulating node SHOULD NOT change the first 20 bits of each LSP in the label stack to avoid ECMP path change. If the encapsulating node is also a transit node, then it MUST also respect transit node responsibilities.

Transit Node Responsibilities Transit nodes SHOULD NOT change the first 20 bits in the LSEs in the label stack. A transit node MAY change the Ancillary Data found in the least significant 8 bits of an LSE. Transit nodes MUST process the NASs in the label stack, respecting if requested by the NAS. A transit node MUST respect the Unknown Action Handling value encoded in the NAS.

Penultimate Node Responsibilities In addition to the transit node responsibilities above, the penultimate node MUST NOT remove the last copy of a HBH or I2E NAS when it is exposed after removing the forwarding (transport) label. This allows the egress node to process the NAS.

Decapsulating Node Responsibilities The decapsulating node MUST remove any NAS it receives.

Network Action Indicator Allocation Procedures This section discusses the procedures and requirements for a allocating a new opcode or flag as a network action indicator (NAI) for a network action. A request for an NAI may make requests from any combination of the "Network Action Opcodes", "Network Action Flags With Ancillary Data", or "Network Action Flags Without Ancillary Data" registries. A request for a new NAI should include the following information:

Scope: The request must specify at-least one scope (I2E, HBH, Select) for the Network Action. The request MAY specify more than one scope.
Ancillary Data: A request should specify the quantity, syntax, and semantics of any associated Ancillary Data. The Ancillary Data MAY be variable length, but the length MUST be computable based on the data present in the NAS.
Processing: The request should specify the detailed procedure for processing the network action.

A request for a new NAI may request any combination of flags or an opcode. This decision should optimize for eventual encoding efficiency. If the NAI does not require any ancillary data, then a flag is preferred as only one bit is used in the encoding. If ancillary data is required, then the optimal choice may depend on how the action is likely to be combined with other actions. If the action is unlikely to be used in combination with other actions and at most 20 bits of ancillary data is required, then an opcode may be preferred as the encoding will only consume a single LSE. If the action is likely to be combined with other actions, then a flag is more likely to be optimal.

Backward Compatibility This section discusses interactions between MNA capable and legacy, non-MNA capable nodes. An MNA encapsulating node MUST ensure that the MPLS Network Action Sub-Stack indicator is not at the top of the MPLS Label Stack when the packet arrives at a non-MNA capable node. If such a packet did arrive at a non-MNA capable node, it will most likely be dropped. Legacy nodes may scan the label stack, potentially looking for a label field containing a bSPL. To ensure that the LSE formats described herein do not appear to contain a bSPL value, the opcode value of 0 has been reserved. By ensuring that there is a non-zero value in the high order 7 bits, we are assured that the high order 20 bits cannot be misinterpreted as containing a bSPL value (0-15). The TC and TTL fields of the Format A LSE are not re-purposed for encoding, as the penultimate node on the MPLS packet path may propagate TTL from the transport (or forwarding) label to the next label on the label stack, overwriting the TTL on the next label. If the penultimate node is a legacy node, it might perform this action, potentially corrupting other values stored in the TC and TTL fields. To protect against this, we retain the TC and TTL fields in the Format A LSE.

Security Considerations The security considerations in also apply to this document. In addition, MNA creates a new dimension in security concerns:

The actions of an encapsulating node can affect any or all of the nodes along the path. In the most common and benign situations, such as a syntactically incorrect packet, this could result in packet loss or corruption.
The semantics of a network action are unbounded and may be insecure. A network action could be defined that made arbitrary changes to the memory of the forwarding router, which could then be used by the encapsulating node to compromise every MNA capable router in the network. The IETF needs to ensure that only secure network actions are defined.
The MNA architecture supports locally defined network actions. For such actions, there will be limited oversight to ensure that the semantics do not create security issues. Implementors and network operators will need to ensure that locally defined network actions do not compromise the security of the network.

IANA Considerations

MNA bSPL Label This document requests that IANA allocate a value (TBA) for the MNA bSPL label from the "Base Special-Purpose MPLS Label Values" registry to indicate the presence of an MNA Sub-Stack in the label stack. The description of the value should be "MPLS Network Actions". The reference should be this document.

MPLS Network Actions Parameters This document requests that IANA create a new registry group called "MPLS Network Actions Parameters" within the "Multiprotocol Label Switching Architecture (MPLS)" registry group. The registries described below should belong to this new registry group.

Network Action Flags With Ancillary Data This document requests that IANA create a new registry with the name "Network Action Flags With Ancillary Data". Registration requests should comply with . The registration procedure for this registry is "IETF Review". The fields in this registry are "Bit Position" (integer), "Description" (string), and "Reference" (string). Bit Position refers to the position relative to the most significant bit in LSE Format C Data fields. Bit Position 0 is the most significant bit a LSE Format C Data field. There are 20 bit positions currently available, 0-19. This registry may be extended in the future. Further opcodes would need to be defined to carry additional flag ranges. The initial assignments for this registry are: Network Action Flags Without Ancillary Data Registry

Bit Position	Description	Reference
0-15	Unassigned
16-19	Private Use	This document

Network Action Flags Without Ancillary Data This document requests that IANA create a new registry with the name "Network Action Flags Without Ancillary Data". Registration requests should comply with . The registration procedure for this registry is "IETF Review". The fields in this registry are "Bit Position" (integer), "Description" (string), and "Reference" (string). Bit Position refers to the position relative to the most significant bit in LSE Format B or C Data fields and any subsequent Format D LSEs. Bit Position 0 is the most significant bit a LSE Format B or C Data field. Bit Position 20 is the most significant bit in the first LSE Format D Data field. There are 20 bits available in LSE Format C and 30 available in LSE Format D. There are at most 15 Format D LSEs per opcode, so there are at most 20 + 15 * 30 = 470 bit positions. The Bit Position is an integer with value 0-469. The initial assignments for this registry are: Network Action Flags With Ancillary Data Registry

Bit Position	Description	Reference
0-15	Unassigned
16-19	Private Use	This document
20-469	Unassigned

Network Action Opcodes This document requests that IANA create a new registry with the name "Network Action Opcodes". Registration requests should comply with . The registration procedure for this registry is "IETF Review". The fields are "Opcode" (integer), "Description" (string), and "Reference" (string). Opcode is an integer 0-127. The initial assignments for this registry are: Network Action Opcodes Registry

Opcode	Description	Reference
0	Reserved	This document
1	Offset of start of Post-Stack Network Action Header	This document
2	Flag-Based Network Action Indicators without AD	This document
3	Flag-Based Network Action Indicators with AD	This document
4	PSD-ISD Ordering	This document
5-110	Unassigned
111-126	Private Use
127	Opcode Range Extension Beyond 127	This document

Appendix A: Examples

Network Action Encoding Examples

Network Action Flags without AD

NAS with Network Action Flags This is an example of an NAS with Flag-Based NAIs without Ancillary Data. Details:

Opcode=2: This opcode to indicates that the LSE carries Flag-Based NAIs without AD.
Data: The data field carries the Flag-Based NAIs.
S: This is the bottom of stack bit. Set if and only if this LSE is the bottom of the stack.
U: Action to be taken if one of the NAIs are not recognized by the processing node.
NASL: The NASL field is set to "0", as there are no additional LSEs.

Network Action Flags without AD using LSE Format D In this example, the NAS contains a Format B LSE with no flags set, indicating no operation. The next LSE uses Format C, but the Network Action Flag is not in a bit position contained within the Format C LSE, so a single Format D LSE has been added to the NAS to carry the flag. NAL is set to "1" to indicate that Flag-Based NAIs are also encoded in the next LSE. NASL is set to "2" to indicate that 2 additional LSEs are used.

Network Action Opcode with AD

Network action opcode with Ancillary Data In this example, the NAS is carrying only one Network Action that requires 13 bits of Ancillary Data. Details on the Second LSE

Opcode=8: A network action allocated outside of this document.
Data: The data field contains 13 bits of ancillary data.

Network Action Opcode with more AD A network action may require more Ancillary Data than can fit in a single LSE. In this example, a Format D LSE is added to carry additional Ancillary Data.

Network Action With Additional Ancillary Data In this example, opcode 9 requires more than one LSE's worth of Ancillary Data, so a Format D LSE is added. Details on the third LSE:

Opcode=9: An opcode allocated outside of this document
Ancillary Data: Most significant bits of Ancillary data
AD: 4 bits of additional Ancillary Data

Details on the fourth LSE:

Ancillary Data: 22 bits of additional Ancillary data.
Ancillary Data: 8 bits of additional Ancillary Data.

Post-Stack Network Action Encoding The details of Post-Stack Network Action Extension Header encodings are specified in .

NAS that only indicates Post-Stack NAs

NAS encoding only Post-Stack NAs In some cases the NAS may encode only the presence of Post-Stack NAs. In this case, the P-Bit is set. The IHS field indicates the scope of the Post-Stack NAs (I2E, HBH, Select).

NAS with both in-stack and post-stack NAs

NAS with in-stack and post-stack NAs In some cases the NAS may encode in-stack NAs and indicate the presence of post-stack NAs. In this case, P-Bit is set. The NASL is set to "1", indicating the presence of one additional LSE. The IHS field indicates the scope of both the in-stack and post-stack NAs.

NASs with multiple scopes

NASs with multiple scopes In some cases the label stack may need to carry in-stack NAs with Hop-By-Hop scope and post-stack NAs with I2E scope. In this case, there will be two NASs in the label stack. In this case, the first NAS will encode the in-stack NA with the Hop-By-Hop scope and the second NAS will encode the presence of I2E scoped Post-Stack NAs.

Network Action Processing Order The semantics of a network action can vary widely and the results of processing one network action may affect the processing of a subsequent network action. To ensure that MNA has deterministic results, it may be necessary to specify the order in which actions are evaluated. See .

Network Action Processing Order

In-stack NA processing order In this example, opcode 8 is processed first, then opcode 7, and then the network action flags are processed from most significant to least significant. In a different case, some Flag-Based NAIs may need to be processed before opcode 7 and some Flag-Based NAIs may need to be processed after Opcode 7. This can done by causing some NAIs to appear earlier in the NAS.

Interleaving network actions In the above example, opcode 8 is processed first, then Flag-Based NAI 0x1 is processed before opcode 7, and finally NAI 0x2 is processed.

Post-Stack NA Processing Order By default, post-stack NAs follow the ordering specified in . However, the PSD-ISD ordering opcode can be used to override the default ordering and interleave PSD actions with in-stack actions.

A mix of in-stack and post-stack NAs In some cases, post-stack NAs needs to be processed before in-stack NAs. This section shows how to prioritize the post-stack NAs over in-stack NAs.

Post-stack and in-stack NA processing order In the above example, opcode 8 is processed first, then the Flag-Based NAIs, followed by Post-Stack NH 6, and finally opcode 7.

References Normative References &I-D.ietf-mpls-mna-fwk; &I-D.ietf-mpls-mna-requirements; &I-D.ietf-mpls-mna-usecases; &I-D.song-mpls-extension-header; &RFC2119; &RFC3032; &RFC3209; &RFC3443; &RFC4377; &RFC4385; &RFC5462; &RFC5586; &RFC8174; &RFC8662; &RFC9017;

Acknowledgments The authors of this document would like to thank the MPLS Working Group Open Design Team for the discussions and comments on this document. The authors would also like to thank Amanda Baber for reviewing the IANA Considerations and providing many useful suggestions. The authors would like to thank Loa Andersson, Stewart Bryant and Greg Mirsky for reviewing our draft and providing many useful suggestions.

Contributors The following people have substantially contributed to this document: