]> Verizon

steven.gringeri@verizon.com

Verizon

jeremy.whittaker@verizon.com

Deutsche Telekom

N.Leymann@telekom.de

Cisco Systems, Inc.

cschmutz@cisco.com

Ciena Corporation

cbrown@ciena.com

This document describes a method for encapsulating high-speed bit-streams as virtual private wire services (VPWS) over packet switched networks (PSN) providing complete signal transport transparency.

Introduction and Motivation This document describes a method called Private Line Emulation (PLE) for encapsulating high-speed bit-streams as Virtual Private Wire Service (VPWS) over Packet Switched Networks (PSN). This emulation suits applications where signal transparency is required and data or framing structure interpretation of the PE would be counter productive. One example is two Ethernet connected CEs and the need for synchronous Ethernet operation between them without the intermediate PEs interfering or addressing concerns about Ethernet control protocol transparency for carrier Ethernet services, beyond the behavior definitions of MEF specifications. Another example would be a Storage Area Networking (SAN) extension between two data centers. Operating at a bit-stream level allows for a connection between Fibre Channel switches without interfering with any of the Fibre Channel protocol mechanisms. Also SONET/SDH add/drop multiplexers or cross-connects can be interconnected without interfering with the multiplexing structures and networks mechanisms. This is a key distinction to CEP defined in where demultiplexing and multiplexing is desired in order to operate per SONET Synchronous Payload Envelope (SPE) and Virtual Tributary (VT) or SDH Virtual Container (VC). Said in another way, PLE does provide an independent layer network underneath the SONET/SDH layer network, whereas CEP does operate at the same level and peer with the SONET/SDH layer network. The mechanisms described in this document follow principals similar to but expanding the applicability beyond the narrow set of PDH interfaces (T1, E1, T3 and E3) and allow the transport of signals from many different technologies such as Ethernet, Fibre Channel, SONET/SDH / and OTN at gigabit speeds by treating them as bit-stream payload defined in sections 3.3.3 and 3.3.4 of .

Requirements Notation The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP14 when, and only when, they appear in all capitals, as shown here.

Terminology and Reference Model

Terminology

• ACH - Associated Channel Header
• AIS - Alarm Indication Signal
• CBR - Constant Bit Rate
• CE - Customer Edge
• CSRC - Contributing SouRCe
• ES - Errored Second
• FEC - Forward Error Correction
• IWF - InterWorking Function
• LDP - Label Distribution Protocol
• LF - Local Fault
• MPLS - Multi Protocol Label Switching
• NSP - Native Service Processor
• ODUk - Optical Data Unit k
• OTN - Optical Transport Network
• OTUk - Optical Transport Unit k
• PCS - Physical Coding Sublayer
• PE - Provider Edge
• PLE - Private Line Emulation
• PLOS - Packet Loss Of Signal
• PSN - Packet Switched Network
• P2P - Point-to-Point
• QOS - Quality Of Service
• RSVP-TE - Resource Reservation Protocol Traffic Engineering
• RTCP - RTP Control Protocol
• RTP - Realtime Transport Protocol
• SAN - Storage Area Network
• SES - Severely Errored Seconds
• SDH - Synchronous Digital Hierarchy
• SID - Segment ID
• SPE - Synchronous Payload Envelope
• SRTP - Secure Realtime Transport Protocol
• SRv6 - Segment Routing over IPv6 Dataplane
• SSRC - Synchronization SouRCe
• SONET - Synchronous Optical Network
• TCP - Transmission Control Protocol
• UAS - Unavailable Seconds
• VPWS - Virtual Private Wire Service
• VC - Virtual Circuit
• VT - Virtual Tributary

Similar to and the term Interworking Function (IWF) is used to describe the functional block that encapsulates bit streams into PLE packets and in the reverse direction decapsulates PLE packets and reconstructs bit streams.

Reference Models The generic reference model defined in and does apply to PLE. Further the model defined in and in particular the concept of a Native Service Processing (NSP) function defined in does apply to PLE as well. The resulting reference model for PLE is illustrated in

PLE Reference Model | | | | |<-PSN tunnel->| | v v v v +---------+ +---------+ | PE1 |==============| PE2 | +---+-----+ +-----+---+ +-----+ | N | | | | N | +-----+ | CE1 |-----| S | IWF |.....VPWS.....| IWF | S |-----| CE2 | +-----+ ^ | P | | | | P | ^ +-----+ | +---+-----+ +-----+---+ | CE1 physical ^ ^ CE2 physical interface | | interface |<--- emulated service --->| | | attachment attachment circuit circuit ]]>

PLE embraces the minimum intervention principle outlined in whereas the data is flowing through the PLE encapsulation layer as received without modifications. For some service types the NSP function is responsible for performing operations on the native data received from the CE. Examples are terminating Forward Error Correction (FEC), terminating the OTUk layer for OTN or dealing with multi-lane processing. After the NSP the IWF is generating the payload of the VPWS which is carried via a PSN tunnel.

Relative Network Scenario Timing |.............................|--------->| | +-----+ |- - -|=================|- - -| ^ +-----+ ^ +-----+ +-----+ | | ^ C D ^ | A | | | +-----------+-----------+ E | +-+ |I| +-+ ]]>

To allow the clock of the transported signal to be carried across the PLE domain in a transparent way the network synchronization reference model and deployment scenario outlined in is applicable. The local oscillators C of PE1 and D of PE2 are locked to a common clock I. As illustrated in , the attachment circuit clock E is generated by PE2 via a differential clock recovery method in reference to the common clock I. For this to work the difference between clock A and clock C (locked to I) MUST be explicitly transferred from PE1 to PE2 using the timestamp inside the RTP header. For the reverse direction PE1 does generate the attachment circuit clock J and the clock difference between G and D (locked to I) transferred from PE2 to PE1. The method used to lock clocks C and D to the common clock I is out of scope of this document, but there are already several well established concepts for achieving frequency synchronization available. While using external timing inputs (aka BITS) or synchronous Ethernet as defined in the characteristics and limits defined in have to be considered. While relying on precision time protocol (PTP) as defined in , the network limits defined in have to be considered.

Emulated Services This specification does describe the emulation of services from a wide range of technologies such as TDM, Ethernet, Fibre Channel or OTN as bit stream or structured bit stream as defined in and .

Generic PLE Service The generic PLE service is an example of the bit stream defined in . Under the assumption that the CE-bound IWF is not responsible for any service specific operation, a bit stream of any rate can be carried using the generic PLE payload. There is no NSP function present for this service.

Ethernet services Ethernet services are special cases of the structured bit stream defined in . IEEE has defined several layers for Ethernet in . Emulation is operating at the physical (PHY) layer, more precisely at the Physical Subcoding Layer (PCS). Over time many different Ethernet interface types have been specified in with a varying set of characteristics such as optional vs mandatory FEC and single-lane vs multi-lane transmission. Ethernet interfaces types with backplane physical media dependent (PMD) variants and ethernet interface types mandating auto-negotiation (except 1000Base-X) are out of scope for this document. All Ethernet services are leveraging the basic PLE payload and interface specific mechanisms are confined to the respective service specific NSP functions.

1000BASE-X The PCS layer of 1000BASE-X defined in clause 36 of is based on 8B/10B code. The PSN-bound NSP function does not modify the received data and is transparent to auto-negotiation but is responsible to detect 1000BASE-X specific attachment circuit faults such as LOS and sync loss. When the CE-bound IWF is in PLOS state or when PLE packets are received with the L-bit being set, the CE-bound NSP function is responsible for

10GBASE-R and 25GBASE-R The PCS layers of 10GBASE-R defined in clause 49 and 25GBASE-R defined in clause 107 of are based on a 64B/66B code. clauses 74 and 108 do define an optional FEC layer, if present the PSN-bound NSP function MUST terminate the FEC and the CE-bound NSP function MUST generate the FEC. The PSN-bound NSP function is also responsible to detect 10GBASE-R and 25GBASE-R specific attachment circuit faults such as LOS and sync loss. The PSN-bound IWF is mapping the scrambled 64B/66B code stream into the basic PLE payload. The CE-bound NSP function MUST perform

• PCS code sync
• descrambling

in order to properly

• transform invalid 66B code blocks into proper error control characters /E/
• insert Local Fault (LF) ordered sets when the CE-bound IWF is in PLOS state or when PLE packets are received with the L-bit being set

Note: Invalid 66B code blocks typically are a consequence of the CE-bound IWF inserting replacement data in case of lost PLE packets, or if the far-end PSN-bound NSP function did set sync headers to 11 due to uncorrectable FEC errors. Before sending the bit stream to the CE, the CE-bound NSP function MUST also scramble the 64B/66B code stream.

40GBASE-R, 50GBASE-R and 100GBASE-R The PCS layers of 40GBASE-R and 100GBASE-R defined in clause 82 and of 50GBASE-R defined in clause 133 of are based on a 64B/66B code transmitted over multiple lanes. clauses 74 and 91 do define an optional FEC layer, if present the PSN-bound NSP function MUST terminate the FEC and the CE-bound NSP function MUST generate the FEC. To gain access to the scrambled 64B/66B code stream the PSN-bound NSP further MUST perform

• block synchronization
• PCS lane de-skew
• PCS lane reordering

The PSN-bound NSP function is also responsible to detect 40GBASE-R, 50GBASE-R and 100GBASE-R specific attachment circuit faults such as LOS and loss of alignment. The PSN-bound IWF is mapping the serialized, scrambled 64B/66B code stream including the alignment markers into the basic PLE payload. The CE-bound NSP function MUST perform

• PCS code sync
• alignment marker removal
• descrambling

in order to properly

• transform invalid 66B code blocks into proper error control characters /E/
• insert Local Fault (LF) ordered sets when the CE-bound IWF is in PLOS state or when PLE packets are received with the L-bit being set

Note: Invalid 66B code blocks typically are a consequence of the CE-bound IWF inserting replacement data in case of lost PLE packets, or if the far-end PSN-bound NSP function did set sync headers to 11 due to uncorrectable FEC errors. When sending the bit stream to the CE, the CE-bound NSP function MUST also perform

• scrambling of the 64B/66B code
• block distribution
• alignment marker insertion

200GBASE-R and 400GBASE-R The PCS layers of 200GBASE-R and 400GBASE-R defined in clause 119 of are based on a 64B/66B code transcoded to a 256B/257B code to reduce the overhead and make room for a mandatory FEC. To gain access to the 64B/66B code stream the PSN-bound NSP further MUST perform

• alignment lock and de-skew
• PCS Lane reordering and de-interleaving
• FEC decoding
• post-FEC interleaving
• alignment marker removal
• descrambling
• reverse transcoding from 256B/257B to 64B/66B

Further the PSN-bound NSP MUST perform rate compensation and scrambling before the PSN-bound IWF is mapping the same into the basic PLE payload. Rate compensation is applied so that the rate of the 66B encoded bit stream carried by PLE is 528/544 times the nominal bitrate of the 200GBASE-R or 400GBASE-R at the PMA service interface. X number of 66 byte long rate compensation blocks are inserted every X*20479 number of 66B client blocks. For 200GBASE-R the value of X is 16 and for 400GBASE-R the value of X is 32. Rate compensation blocks are special 66B control characters of type 0x00 that can easily be searched for by the CE-bound IWF in order to remove them. The PSN-bound NSP function is also responsible to detect 200GBASE-R and 400GBASE-R specific attachment circuit faults such as LOS and loss of alignment. The CE-bound NSP function MUST perform

• PCS code sync
• descrambling
• rate compensation block removal

in order to properly

• transform invalid 66B code blocks into proper error control characters /E/
• insert Local Fault (LF) ordered sets when the CE-bound IWF is in PLOS state or when PLE packets are received with the L-bit being set

Note: Invalid 66B code blocks typically are a consequence of the CE-bound IWF inserting replacement data in case of lost PLE packets, or if the far-end PSN-bound NSP function did set sync headers to 11 due to uncorrectable FEC errors. When sending the bit stream to the CE, the CE-bound NSP function MUST also perform

• transcoding from 64B/66B to 256B/257B
• scrambling
• alignment marker insertion
• pre-FEC distribution
• FEC encoding
• PCS Lane distribution

Energy Efficient Ethernet (EEE) Section 78 of does define the optional Low Power Idle (LPI) capability for Ethernet. Two modes are defined

• deep sleep
• fast wake

Deep sleep mode is not compatible with PLE due to the CE ceasing transmission. Hence there is no support for LPI for 10GBASE-R services across PLE. When in fast wake mode the CE transmits /LI/ control code blocks instead of /I/ control code blocks and therefore PLE is agnostic to it. For 25GBASE-R and higher services across PLE, LPI is supported as only fast wake mode is applicable.

SONET/SDH Services SONET/SDH services are special cases of the structured bit stream defined in . SDH interfaces are defined in and SONET interfaces are defined in . The PSN-bound NSP function does not modify the received data but is responsible to detect SONET/SDH interface specific attachment circuit faults such as LOS, LOF and OOF. Data received by the PSN-bound IWF is mapped into the basic PLE payload without any awareness of SONET/SDH frames. When the CE-bound IWF is in PLOS state or when PLE packets are received with the L-bit being set, the CE-bound NSP function is responsible for generating the

• MS-AIS maintenance signal defined in clause 6.2.4.1.1 of for SDH services
• AIS-L maintenance signal defined in clause 6.2.1.2 of for SONET services

at client frame boundaries.

Fibre Channel Services Fibre Channel services are special cases of the structured bit stream defined in . The T11 technical committee of INCITS has defined several layers for Fibre Channel. Emulation is operating at the FC-1 layer. Over time many different Fibre Channel interface types have been specified in FC-PI-x and FC-FS-x standards with a varying set of characteristics such as optional vs mandatory FEC and single-lane vs multi-lane transmission. Speed negotiation is out of scope for this document. All Fibre Channel services are leveraging the basic PLE payload and interface specific mechanisms are confined to the respective service specific NSP functions.

1GFC, 2GFC, 4GFC and 8GFC FC-PI-2 specifies 1GFC and 2GFC. FC-PI-5 (including amendment 1) does define 4GFC and 8GFC. The PSN-bound NSP function is responsible to detect Fibre Channel specific attachment circuit faults such as LOS and sync loss. The PSN-bound IWF is mapping the received 8B/10B code stream as is into the basic PLE payload. The CE-bound NSP function MUST perform transmission word sync in order to properly

• replace invalid transmission words with the special character K30.7
• insert Not Operational (NOS) ordered sets when the CE-bound IWF is in PLOS state or when PLE packets are received with the L-bit being set

Note: Invalid transmission words typically are a consequence of the CE-bound IWF inserting replacement data in case of lost PLE packets. FC-FS-2 amendment 1 does define the use of scrambling for 8GFC, in this case the CE-bound NSP MUST also perform descrambling before replacing invalid transmission words or inserting NOS ordered sets. And before sending the bit stream to the, the CE-bound NSP function MUST scramble the 8B/10B code stream.

16GFC and 32GFC FC-PI-5 (including amendment 1) specifies 16GFC and defines a optional FEC layer. FC-PI-6 specifies 32GFC with the FEC layer and transmitter training signal (TTS) support being mandatory. If FEC is present it must be indicated via TTS during attachment circuit bring up. Further the PSN-bound NSP function MUST terminate the FEC and the CE-bound NSP function must generate the FEC. The PSN-bound NSP function is responsible to detect Fibre Channel specific attachment circuit faults such as LOS and sync loss. The PSN-bound IWF is mapping the received 64B/66B code stream as is into the basic PLE payload. The CE-bound NSP function MUST perform

• transmission word sync
• descrambling

in order to properly

• replace invalid transmission words with the error transmission word 1Eh
• insert Not Operational (NOS) ordered sets when the CE-bound IWF is in PLOS state or when PLE packets are received with the L-bit being set

Note: Invalid transmission words typically are a consequence of the CE-bound IWF inserting replacement data in case of lost PLE packets, or if the far-end PSN-bound NSP function did set sync headers to 11 due to uncorrectable FEC errors. Before sending the bit stream to the CE, the CE-bound NSP function MUST also scramble the 64B/66B code stream.

64GFC and 4-lane 128GFC FC-PI-7 specifies 64GFC and FC-PI-6P specifies 4-lane 128GFC. Both specify a mandatory FEC layer. The PSN-bound NSP function MUST terminate the FEC and the CE-bound NSP function must generate the FEC. To gain access to the 64B/66B code stream the PSN-bound NSP further MUST perform

• alignment lock and de-skew
• Lane reordering and de-interleaving
• FEC decoding
• post-FEC interleaving
• alignment marker removal
• descrambling
• reverse transcoding from 256B/257B to 64B/66B

Further the PSN-bound NSP MUST perform scrambling before the PSN-bound IWF is mapping the same into the basic PLE payload. Note : the use of rate compensation is for further study. The PSN-bound NSP function is also responsible to detect Fibre Channel specific attachment circuit faults such as LOS and sync loss. The CE-bound NSP function MUST perform

• transmission word sync
• descrambling

in order to properly

• replace invalid transmission words with the error transmission word 1Eh
• insert Not Operational (NOS) ordered sets when the CE-bound IWF is in PLOS state or when PLE packets are received with the L-bit being set

Note: Invalid transmission words typically are a consequence of the CE-bound IWF inserting replacement data in case of lost PLE packets, or if the far-end PSN-bound NSP function did set sync headers to 11 due to uncorrectable FEC errors. When sending the bit stream to the CE, the CE-bound NSP function MUST also perform

• transcoding from 64B/66B to 256B/257B
• scrambling
• alignment marker insertion
• pre-FEC distribution
• FEC encoding
• Lane distribution

OTN Services OTN services are special cases of the structured bit stream defined in . OTN interfaces are defined in . The PSN-bound NSP function MUST terminate the FEC and replace the OTUk overhead in row 1 columns 8-14 with all-0s fixed stuff which results in a extended ODUk frame as illustrated in . The frame alignment overhead (FA OH) in row 1 columns 1-7 is kept as it is.

Extended ODUk Frame

The PSN-bound NSP function is also responsible to detect OTUk specific attachment circuit faults such as LOS, LOF, LOM and AIS. The PSN-bound IWF is mapping the extended ODUk frame into the byte aligned PLE payload. The CE-bound NSP function will recover the ODUk by searching for the frame alignment overhead in the extended ODUk received from the CE-bound IWF and generates the FEC. When the CE-bound IWF is in PLOS state or when PLE packets are received with the L-bit being set, the CE-bound NSP function is responsible for generating the ODUk-AIS maintenance signal defined in clause 16.5.1 of at client frame boundaries.

PLE Encapsulation Layer The basic packet format used by PLE is shown in the .

PLE Encapsulation Layer PSN and VPWS | | / Demux Headers +-------------------------------+ -+ | PLE Control Word | \ +-------------------------------+ > PLE Header | RTP Header | / +-------------------------------+ --+ | Bit Stream | \ | Payload | > Payload | | / +-------------------------------+ --+ ]]>

PSN and VPWS Demultiplexing Headers This document does not imply any specific technology to be used for implementing the VPWS demultiplexing and PSN layers. When a MPLS PSN layer is used, a VPWS label provides the demultiplexing mechanism as described in . The PSN tunnel can be a simple best path Label Switched Path (LSP) established using LDP or Segment Routing or a traffic engineered LSP established using RSVP-TE or SR-TE . When a SRv6 PSN layer is used, a SRv6 service SID does provide the demultiplexing mechanism and the mechanisms defined in and section 6 and a new endpoint behavior End.DX1 defined in this document do apply. The "Endpoint with decapsulation and bit-stream cross-connect" behavior ("End.DX1" for short) is a variant of End.DX2 which is defined in . The End.DX1 SID MUST be the last segment in an SR Policy, and it is associated with a CE-bound IWF I. When N receives a packet destined to S and S is a local End.DX1 SID, N does the following: S01. When an SRH is processed { S02. If (Segments Left != 0) { S03. Send an ICMP Parameter Problem to the Source Address with Code 0 (Erroneous header field encountered) and Pointer set to the Segments Left field, interrupt packet processing, and discard the packet. S04. } S05. Proceed to process the next header in the packet S06. } When processing the Upper-Layer header of a packet matching a FIB entry locally instantiated as an End.DX1 SID, N does the following: S01. If (Upper-Layer header type == TBA1 (bit-stream) ) { S02. Remove the outer IPv6 header with all its extension headers S03. Forward the remaining frame to the IWF I S04. } Else { S05. Process as per S06. }

PLE Header The PLE header MUST contain the PLE control word (4 bytes) and MUST include a fixed size RTP header . The RTP header MUST immediately follow the PLE control word.

PLE Control Word The format of the PLE control word is in line with the guidance in and is shown in .

PLE Control Word

The bits 0..3 of the first nibble are set to 0 to differentiate a control word or Associated Channel Header (ACH) from an IP packet or Ethernet frame. The first nibble MUST be set to 0000b to indicate that this header is a control word as defined in . The other fields in the control word are used as defined below:

• L

• Set by the PE to indicate that data carried in the payload is invalid due to an attachment circuit fault. The downstream PE MUST play out appropriate replacement data. The NSP MAY inject an appropriate native fault propagation signal.

• R

• Set by the downstream PE to indicate that the IWF experiences packet loss from the PSN or a server layer backward fault indication is present in the NSP. The R bit MUST be cleared by the PE once the packet loss state or fault indication has cleared.

• RSV

• These bits are reserved for future use. This field MUST be set to zero by the sender and ignored by the receiver.

• FRG

• These bits MUST be set to zero by the sender and ignored by the receiver.

• LEN

• In accordance to the length field MUST always be set to zero as there is no padding added to the PLE packet. To detect malformed packets the default, preconfigured or signaled payload size MUST be assumed.

• Sequence number

• The sequence number field is used to provide a common PW sequencing function as well as detection of lost packets. It MUST be generated in accordance with the rules defined in and MUST be incremented with every PLE packet being sent.

RTP Header The RTP header MUST be included and is used for explicit transfer of timing information. The RTP header is purely a formal reuse and RTP mechanisms, such as header extensions, contributing source (CSRC) list, padding, RTP Control Protocol (RTCP), RTP header compression, Secure Realtime Transport Protocol (SRTP), etc., are not applicable to PLE VPWS. The format of the RTP header is as shown in .

RTP Header

• V: Version

• The version field MUST be set to 2.

• P: Padding

• The padding flag MUST be set to zero by the sender and ignored by the receiver.

• X: Header extension

• The X bit MUST be set to zero by sender and ignored by receiver.

• CC: CSRC count

• The CC field MUST be set to zero by the sender and ignored by the receiver.

• M: Marker

• The M bit MUST be set to zero by the sender and ignored by the receiver.

• PT: Payload type

• A PT value MUST be allocated from the range of dynamic values defined by for each direction of the VPWS. The same PT value MAY be reused both for direction and between different PLE VPWS.

• Sequence number

• The Sequence number in the RTP header MUST be equal to the sequence number in the PLE control word. The sequence number of the RTP header MAY be used to extend the sequence number of the PLE control word from 16 to 32 bits. If so, the initial value of the RTP sequence number MUST be 0 and incremented whenever the PLE control word sequence number cycles through from 0xFFFF to 0x0000.

• Timestamp

• Timestamp values are used in accordance with the rules established in . For bit-streams up to 200 Gbps the frequency of the clock used for generating timestamps MUST be 125 MHz based on a the common clock I. For bit-streams above 200 Gbps the frequency MUST be 250 MHz.

• SSRC: Synchronization source

• The SSRC field MAY be used for detection of misconnections.

PLE Payload Layer A bit-stream is mapped into a PLE packet with a fixed payload size which MUST be defined during VPWS setup, MUST be the same in both directions of the VPWS and MUST remain unchanged for the lifetime of the VPWS. All PLE implementations MUST be capable of supporting the default payload size of 1024 bytes.

Basic Payload The PLE payload is filled with incoming bits of the bit-stream starting from the most significant to the least significant bit without considering any structure of the bit-stream.

Byte aligned Payload The PLE payload is filled in a byte aligned manner, where the order of the payload bytes corresponds to their order on the attachment circuit. Consecutive bits coming from the attachment circuit fill each payload byte starting from most significant bit to least significant. The PLE payload size MUST be a integral number of bytes.

PLE Operation

Common Considerations A PLE VPWS can be established using manual configuration or leveraging mechanisms of a signaling protocol. Furthermore emulation of bit-stream signals using PLE is only possible when the two attachment circuits of the VPWS are of the same type (OC192, 10GBASE-R, ODU2, etc) and are using the same PLE payload type and payload size. This can be ensured via manual configuration or via a signaling protocol PLE related control protocol extensions to PWE3 and EVPN-VPWS are out of scope of this document and are described in .

PLE IWF Operation

PSN-bound Encapsulation Behavior After the VPWS is set up, the PSN-bound IWF does perform the following steps:

• Packetize the data received from the CE is into a fixed size PLE payloads
• Add PLE control word and RTP header with sequence numbers, flags and timestamps properly set
• Add the VPWS demultiplexer and PSN headers
• Transmit the resulting packets over the PSN
• Set L bit in the PLE control word whenever attachment circuit detects a fault
• Set R bit in the PLE control word whenever the local CE-bound IWF is in packet loss state

CE-bound Decapsulation Behavior The CE-bound IWF is responsible for removing the PSN and VPWS demultiplexing headers, PLE control word and RTP header from the received packet stream and play-out of the bit-stream to the local attachment circuit. A de-jitter buffer MUST be implemented where the PLE packets are stored upon arrival. The size of this buffer SHOULD be locally configurable to allow accommodation of specific PSN packet delay variation expected. The CE-bound IWF SHOULD use the sequence number in the control word to detect lost and mis-ordered packets. It MAY use the sequence number in the RTP header for the same purposes. The payload of a lost packet MUST be replaced with equivalent amount of replacement data. The contents of the replacement data MAY be locally configurable. All PLE implementations MUST support generation of "0xAA" as replacement data. The alternating sequence of 0s and 1s of the "0xAA" pattern does ensure clock synchronization is maintained. While playing out the replacement data, the IWF will apply a holdover mechanism to maintain the clock. Whenever the VPWS is not operationally up, the CE-bound NSP function MUST inject the appropriate native downstream fault indication signal. Whenever a VPWS comes up, the CE-bound IWF enters the intermediate state, will start receiving PLE packets and will store them in the jitter buffer. The CE-bound NSP function will continue to inject the appropriate native downstream fault indication signal until a pre-configured amount of payloads is stored in the jitter buffer. After the pre-configured amount of payload is present in the jitter buffer the CE-bound IWF transitions to the normal operation state and the content of the jitter buffer is played out to the CE in accordance with the required clock. In this state the CE-bound IWF MUST perform egress clock recovery. The recovered clock MUST comply with the jitter and wander requirements applicable to the type of attachment circuit, specified in:

• and for SDH
• for SONET
• for synchronous Ethernet
• for OTN

Whenever the L bit is set in the PLE control word of a received PLE packet the CE-bound NSP function SHOULD inject the appropriate native downstream fault indication signal instead of playing out the payload. If the CE-bound IWF detects loss of consecutive packets for a pre-configured amount of time (default is 1 millisecond), it enters packet loss (PLOS) state and a corresponding defect is declared. If the CE-bound IWF detects a packet loss ratio (PLR) above a configurable signal-degrade (SD) threshold for a configurable amount of consecutive 1-second intervals, it enters the degradation (DEG) state and a corresponding defect is declared. The SD-PLR threshold can be defined as percentage with the default being 15% or absolute packet count for finer granularity for higher rate interfaces. Possible values for consecutive intervals are 2..10 with the default 7. While the PLOS defect is declared the CE-bound NSP function SHOULD inject the appropriate native downstream fault indication signal. Also the PSN-bound IWF SHOULD set the R bit in the PLE control word of every packet transmitted. The CE-bound IWF does change from the PLOS to normal state after the pre-configured amount of payload has been received similarly to the transition from intermediate to normal state. Whenever the R bit is set in the PLE control word of a received PLE packet the PLE performance monitoring statistics SHOULD get updated.

PLE Performance Monitoring PLE SHOULD provide the following functions to monitor the network performance to be inline with expectations of transport network operators. The near-end performance monitors defined for PLE are as follows:

• ES-PLE : PLE Errored Seconds
• SES-PLE : PLE Severely Errored Seconds
• UAS-PLE : PLE Unavailable Seconds

Each second with at least one packet lost or a PLOS/DEG defect SHALL be counted as ES-PLE. Each second with a PLR greater than 15% or a PLOS/DEG defect SHALL be counted as SES-PLE. UAS-PLE SHALL be counted after configurable number of consecutive SES-PLE have been observed, and no longer counted after a configurable number of consecutive seconds without SES-PLE have been observed. Default value for each is 10 seconds. Once unavailability is detected, ES and SES counts SHALL be inhibited up to the point where the unavailability was started. Once unavailability is removed, ES and SES that occurred along the clearing period SHALL be added to the ES and SES counts. A PLE far-end performance monitor is providing insight into the CE-bound IWF at the far end of the PSN. The statistics are based on the PLE-RDI indication carried in the PLE control word via the R bit. The PLE VPWS performance monitors are derived from the definitions in accordance with

QoS and Congestion Control The PSN carrying PLE VPWS may be subject to congestion, but PLE VPWS representing constant bit-rate (CBR) flows cannot respond to congestion in a TCP-friendly manner as described in . Hence the PSN providing connectivity for the PLE VPWS between PE devices MUST be Diffserv enabled and MUST provide a per domain behavior that guarantees low jitter and low loss. To achieve the desired per domain behavior PLE VPWS SHOULD be carried over traffic-engineering paths through the PSN with bandwidth reservation and admission control applied.

Security Considerations As PLE is leveraging VPWS as transport mechanism the security considerations described in and are applicable.

IANA Considerations

Bit-stream Next Header Type IANA is requested to assign "Bit-stream Emulation" (value TBA1) in the "Assigned Internet Protocol Numbers" registry (see https://www.iana.org/assignments/protocol-numbers/). Value TBA1 in the next header field of an IPv6 header or any extension header indicates that the payload is a emulated bit-stream as originally defined in and for a certain set of interface types in this document more specifically.

SRv6 Endpoint Behaviors This document introduces a new SRv6 Endpoint behavior "End.DX1". IANA is requested to assign identifier values in the "SRv6 Endpoint Behaviors" sub-registry under "Segment Routing Parameters" registry.

Value	Hex	Endpoint Behavior	Reference
158	0x009E	End.DX1	this document
159	0x009F	End.DX1 with NEXT-CSID	this document
160	0x00A0	End.DX1 with REPLACE-CSID	this document

Acknowledgements The authors would like to thank all contributors and the working group for reviewing this document and providing useful comments and suggestions.

References Normative References This document describes an architecture for Pseudo Wire Emulation Edge-to-Edge (PWE3). It discusses the emulation of services such as Frame Relay, ATM, Ethernet, TDM, and SONET/SDH over packet switched networks (PSNs) using IP or MPLS. It presents the architectural framework for pseudo wires (PWs), defines terminology, and specifies the various protocol elements and their functions. This memo provides information for the Internet community. This document provides a framework for Layer 2 Provider Provisioned Virtual Private Networks (L2VPNs). This framework is intended to aid in standardizing protocols and mechanisms to support interoperable L2VPNs. This memo provides information for the Internet community. This document defines the specific requirements for edge-to-edge emulation of circuits carrying Time Division Multiplexed (TDM) digital signals of the Plesiochronous Digital Hierarchy as well as the Synchronous Optical NETwork/Synchronous Digital Hierarchy over packet-switched networks. It is aligned to the common architecture for Pseudo Wire Emulation Edge-to-Edge (PWE3). It makes references to the generic requirements for PWE3 where applicable and complements them by defining requirements originating from specifics of TDM circuits. This memo provides information for the Internet community. This document describes the preferred design of a Pseudowire Emulation Edge-to-Edge (PWE3) Control Word to be used over an MPLS packet switched network, and the Pseudowire Associated Channel Header. The design of these fields is chosen so that an MPLS Label Switching Router performing MPLS payload inspection will not confuse a PWE3 payload with an IP payload. [STANDARDS-TRACK] International Telecommunication Union (ITU) International Telecommunication Union (ITU) International Telecommunication Union (ITU) International Telecommunication Union (ITU) Telcordia International Telecommunication Union (ITU) International Telecommunication Union (ITU) International Telecommunication Union (ITU) International Telecommunication Union (ITU) IEEE International Telecommunication Union (ITU) International Telecommunication Union (ITU) In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This memorandum describes RTP, the real-time transport protocol. RTP provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. RTP does not address resource reservation and does not guarantee quality-of- service for real-time services. The data transport is augmented by a control protocol (RTCP) to allow monitoring of the data delivery in a manner scalable to large multicast networks, and to provide minimal control and identification functionality. RTP and RTCP are designed to be independent of the underlying transport and network layers. The protocol supports the use of RTP-level translators and mixers. Most of the text in this memorandum is identical to RFC 1889 which it obsoletes. There are no changes in the packet formats on the wire, only changes to the rules and algorithms governing how the protocol is used. The biggest change is an enhancement to the scalable timer algorithm for calculating when to send RTCP packets in order to minimize transmission in excess of the intended rate when many participants join a session simultaneously. [STANDARDS-TRACK] This document describes a profile called "RTP/AVP" for the use of the real-time transport protocol (RTP), version 2, and the associated control protocol, RTCP, within audio and video multiparticipant conferences with minimal control. It provides interpretations of generic fields within the RTP specification suitable for audio and video conferences. In particular, this document defines a set of default mappings from payload type numbers to encodings. This document also describes how audio and video data may be carried within RTP. It defines a set of standard encodings and their names when used within RTP. The descriptions provide pointers to reference implementations and the detailed standards. This document is meant as an aid for implementors of audio, video and other real-time multimedia applications. This memorandum obsoletes RFC 1890. It is mostly backwards-compatible except for functions removed because two interoperable implementations were not found. The additions to RFC 1890 codify existing practice in the use of payload formats under this profile and include new payload formats defined since RFC 1890 was published. [STANDARDS-TRACK] This document defines an architecture for implementing scalable service differentiation in the Internet. This memo provides information for the Internet community. This document defines and discusses Per-Domain Behaviors in detail and lays out the format and required content for contributions to the Diffserv WG on PDBs and the procedure that will be applied for individual PDB specifications to advance as WG products. This format is specified to expedite working group review of PDB submissions. This memo provides information for the Internet community. This document describes procedures for BGP MPLS-based Ethernet VPNs (EVPN). The procedures described here meet the requirements specified in RFC 7209 -- "Requirements for Ethernet VPN (EVPN)". Informative References This document provides encapsulation formats and semantics for emulating Synchronous Optical Network/Synchronous Digital Hierarchy (SONET/SDH) circuits and services over MPLS. [STANDARDS-TRACK] This document describes a pseudowire encapsulation for Time Division Multiplexing (TDM) bit-streams (T1, E1, T3, E3) that disregards any structure that may be imposed on these streams, in particular the structure imposed by the standard TDM framing. [STANDARDS-TRACK] This document describes a method for encapsulating structured (NxDS0) Time Division Multiplexed (TDM) signals as pseudowires over packet-switching networks (PSNs). In this regard, it complements similar work for structure-agnostic emulation of TDM bit-streams (see RFC 4553). This memo provides information for the Internet community. The architecture for Multiprotocol Label Switching (MPLS) is described in RFC 3031. A fundamental concept in MPLS is that two Label Switching Routers (LSRs) must agree on the meaning of the labels used to forward traffic between and through them. This common understanding is achieved by using a set of procedures, called a label distribution protocol, by which one LSR informs another of label bindings it has made. This document defines a set of such procedures called LDP (for Label Distribution Protocol) by which LSRs distribute labels to support MPLS forwarding along normally routed paths. [STANDARDS-TRACK] Segment Routing (SR) leverages the source routing paradigm. A node steers a packet through an ordered list of instructions, called "segments". A segment can represent any instruction, topological or service based. A segment can have a semantic local to an SR node or global within an SR domain. SR provides a mechanism that allows a flow to be restricted to a specific topological path, while maintaining per-flow state only at the ingress node(s) to the SR domain. SR can be directly applied to the MPLS architecture with no change to the forwarding plane. A segment is encoded as an MPLS label. An ordered list of segments is encoded as a stack of labels. The segment to process is on the top of the stack. Upon completion of a segment, the related label is popped from the stack. SR can be applied to the IPv6 architecture, with a new type of routing header. A segment is encoded as an IPv6 address. An ordered list of segments is encoded as an ordered list of IPv6 addresses in the routing header. The active segment is indicated by the Destination Address (DA) of the packet. The next active segment is indicated by a pointer in the new routing header. This document describes the use of RSVP (Resource Reservation Protocol), including all the necessary extensions, to establish label-switched paths (LSPs) in MPLS (Multi-Protocol Label Switching). Since the flow along an LSP is completely identified by the label applied at the ingress node of the path, these paths may be treated as tunnels. A key application of LSP tunnels is traffic engineering with MPLS as specified in RFC 2702. [STANDARDS-TRACK] Segment Routing (SR) allows a node to steer a packet flow along any path. Intermediate per-path states are eliminated thanks to source routing. SR Policy is an ordered list of segments (i.e., instructions) that represent a source-routed policy. Packet flows are steered into an SR Policy on a node where it is instantiated called a headend node. The packets steered into an SR Policy carry an ordered list of segments associated with that SR Policy. This document updates RFC 8402 as it details the concepts of SR Policy and steering into an SR Policy. This document defines procedures and messages for SRv6-based BGP services, including Layer 3 Virtual Private Network (L3VPN), Ethernet VPN (EVPN), and Internet services. It builds on "BGP/MPLS IP Virtual Private Networks (VPNs)" (RFC 4364) and "BGP MPLS-Based Ethernet VPN" (RFC 7432). The Segment Routing over IPv6 (SRv6) Network Programming framework enables a network operator or an application to specify a packet processing program by encoding a sequence of instructions in the IPv6 packet header. Each instruction is implemented on one or several nodes in the network and identified by an SRv6 Segment Identifier in the packet. This document defines the SRv6 Network Programming concept and specifies the base set of SRv6 behaviors that enables the creation of interoperable overlays with underlay optimization. Layer 2 services (such as Frame Relay, Asynchronous Transfer Mode, and Ethernet) can be "emulated" over an MPLS backbone by encapsulating the Layer 2 Protocol Data Units (PDU) and transmitting them over "pseudowires". It is also possible to use pseudowires to provide low-rate Time Division Multiplexed and a Synchronous Optical NETworking circuit emulation over an MPLS-enabled network. This document specifies a protocol for establishing and maintaining the pseudowires, using extensions to Label Distribution Protocol (LDP). Procedures for encapsulating Layer 2 PDUs are specified in a set of companion documents. [STANDARDS-TRACK] This document describes how Ethernet VPN (EVPN) can be used to support the Virtual Private Wire Service (VPWS) in MPLS/IP networks. EVPN accomplishes the following for VPWS: provides Single-Active as well as All-Active multihoming with flow-based load-balancing, eliminates the need for Pseudowire (PW) signaling, and provides fast protection convergence upon node or link failure. Verizon Verizon Cisco Systems, Inc. Cisco Systems, Inc. This document specifies the mechanisms to allow for dynamic signalling of Virtual Private Wire Services (VPWS) carrying bit- stream signals over Packet Switched Networks (PSN). This memo defines a media feature tag whose value is a Multipurpose Internet Mail Extensions (MIME) content type. [STANDARDS-TRACK] This memo documents the fundamental truths of networking for the Internet community. This memo does not specify a standard, except in the sense that all standards must implicitly follow the fundamental truths. This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind.

Contributors 1&1 Versatel

andreas.burk@magenta.de

AMD

faisal.dada@amd.com

Ciena Corporation

gsmalleg@ciena.com

Aimvalley

erik.vanveelen@aimvalley.com

Cisco Systems, Inc.

ldellach@cisco.com

Cisco Systems, Inc.

naikumar@cisco.com

North Carolina State University

cmpignat@ncsu.edu