Realtime Synchronization between Redundant Stateful PCEs.

describes the Path Computation Element Protocol (PCEP) as the communication between a Path Computation Client (PCC) and a Path Computation Element (PCE), or between PCEs, enabling computation of Multiprotocol Label Switching (MPLS) for Traffic Engineering Label Switched Paths (TE LSPs). specifies a set of extensions to PCEP to enable stateful control of LSPs in compliance with . It includes mechanisms for LSP state synchronization between a PCC and a PCE, i.e., all stateful PCEs synchronize their LSP states from the network. It further describe the handling of redundant stateful PCEs, where all PCEs receive the state from the network (PCCs). When the primary PCE fails, another PCE can take over. Apart from the synchronization from the network, it is also useful if there is realtime synchronization mechanism between the stateful PCEs. As stateful PCE make changes to its delegated LSPs, these changes (pending LSPs and the sticky resources) can be synchronized immediately to the other PCEs. Further PCE may also synchronize any status change of its delegated LSPs to other PCEs. Note that some synchronization issues are identified in . It should be noted that in some deployments the PCE function is part of the central controller architecture with multiple instances of PCE for load balancing and backup which uses proprietary mechanics to maintain consistent state between these PCE instance. In such deployment PCEP MAY not used as a database synchronization mechanism. This document specifies the mechanisms of realtime LSP-DB synchronization between redundant stateful PCEs via PCEP.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

The terminology is as per and . A database of LSPs that are active in the network as maintained by a stateful PCE. The temporarily assigned resources that are allocated to a pending LSP and are provisionally blocked.

Distributed computation model () refers to a domain or network that may include multiple PCEs where computation of paths is shared among the PCEs, this is further clarified in . When multiple stateful PCEs are operating in the network, they could be either - A backup PCE exists to perform functions in the network, only in the event of a failure of the primary PCE. In this case, all LSPs to be delegated are under primary stateful PCE control while other PCEs in the domain act as backup. Load-Balanced PCEs share the computation load at all times, as well as act backup to each other. One PCE MAY serve a set of PCCs as the primary computation server, and only addresses requests from other PCCs in the event of the failure of some other PCE. Delegated LSPs are thus distributed among stateful PCEs. In either case it is beneficial for the PCE to synchronize changes of its delegated LSPs to the other PCEs in realtime. This should include - Any update made by the PCE to its delegated LSP. Any status change learned from the network. Note that the state synchronization as per and remains unchanged. This include initial state synchronization as well as LSP state reports. The mechanism described in this document are in addition to those already present in .

specifies new functions to support a stateful PCE. It also specifies that a function can be initiated either from a PCC towards a PCE (C-E) or from a PCE towards a PCC (E-C). Capability negotiation (E-C,C-E) LSP state synchronization (C-E) LSP update request (E-C) LSP state report (C-E) LSP control delegation (C-E,E-C) Stateful PCE discovery via This document extends some of these functions to support realtime LSP-DB synchronization. These are initiated from a PCE towards another PCE (E-E). both the PCEs must announce during PCEP session establishment that they support PCEP Stateful PCE extensions defined in . It should also declare whether it has realtime synchronization capability between PCEs. This is done via Open message. a PCE sends an LSP state report to a PCE whenever the state of an delegated LSP changes. This is usually triggered on receiving the state report from the PCC. This is done via PCRpt message. When a PCE requests modification of attributes of a delegated LSP, this information should also be sent to other PCEs. This is done via PCUpd message. This is needed to synchronize the pending LSPs and sticky resources. PCE can advertise its realtime synchronization capability between PCEs via IGP.

PCE (including redundant stateful PCEs) learn LSP state from the PCCs. Apart from that, for each LSP delegated to a stateful PCE - When it sends an LSP Update (PCUpd message) to the PCC for the delegated LSP, it also sends an LSP update to other stateful PCEs. When it receives an LSP report (LSRpt message) from the PCC for the delegated LSP, it also sends an LSP report to other stateful PCEs. Thus a PCE may learn LSP state from both the PCC as well as the PCE to which LSP is delegated. In , PCE1 is the primary stateful PCE and PCE2 is the backup stateful PCE (all LSPs are delegated to PCE1). PCC1 and PCC2 synchronize the LSP-DB with PCE1 and PCE2 after session initialization phase. PCC1 and PCC2 delegates LSP1 and LSP2 to the primary PCE1. Whenever there is an update in LSP, PCE1 sends a PCUpd message to corresponding PCC and also to backup PCE2. This is LSP update request as described in and uses PCUpd message. This makes sure that the pending LSP changes and sticky resources are backed up. The PCC sends a PCRpt message to the primary PCE, indicating the LSP's status, the primary PCE further synchronizes the state with backup PCEs via PCRpt message.

+----+ +----+ +----+ +----+ |PCC1| |PCC2| |PCE1| |PCE2| +-+--+ +-+--+ +-+--+ +-+--+ | | | | |---- LSP SYNC ---+----------------->| | | |---- LSP SYNC --->| | | | | | | |---- LSP SYNC ----+------------------>| |---- LSP SYNC ---+------------------+------------------>| | | | | |-- PCRpt,lsp1,D -+----------------->| | |<----------------+----PCUpd,lsp1 ---| | | | |--- PCUpd,lsp1 --->| |-- PCRpt,lsp1,up-+----------------->| | |-- PCRpt,lsp1,up-+------------------+------------------>| | | |----PCRpt,lsp1,up->| | | | | | |-- PCRpt,lsp2,D ->| | | |<---PCUpd,lsp2 ---| | | | |--- PCUpd,lsp2---->| | |-- PCRpt,lsp2,up->| | | |-- PCRpt,lsp2,up--+------------------>| | | |----PCRpt,lsp2,up->| | | | | The backup PCE is used only in case the primary PCE fails. At the time of failure of primary PCE (PCE1), the backup PCE (PCE2) act as a primary. In , PCE1 and PCE2 are load-balanced stateful PCEs and share the computation load as well as act as backup to each other. PCC1 and PCC2 synchronize their LSP-DB with both PCEs after session initialization phase as per . PCC1 delegates LSP1 to PCE1. Whenever there is an update in LSP1, PCE1 sends the PCUpd message to PCC1 and other stateful PCEs (PCE2). Similarly, PCC2 delegates LSP2 to PCE2. Whenever there is an update in LSP2, PCE2 sends the PCUpd message to PCC2 and other stateful PCEs (PCE1). This is LSP update request as described in and it makes sure that the pending LSP changes and sticky resources are synchronized. The PCC sends an PCRpt message to the all load-balanced PCEs as per , indicating the LSP's status. The PCE to which LSP is delegated, also sends report message to other PCEs.

+----+ +----+ +----+ +----+ |PCC1| |PCC2| |PCE1| |PCE2| +-+--+ +-+--+ +-+--+ +-+--+ | | | | |---- LSP SYNC ---+----------------->| | |---- LSP SYNC ---+------------------+------------------>| | |---- LSP SYNC --->| | | |---- LSP SYNC ----+------------------>| | | | | |-- PCRpt,lsp1,D -+----------------->| | | |-- PCRpt,lsp2,D --+------------------>| | | | | | | | | |<----------------+----PCUpd, lsp1---| | | | |--- PCUpd, lsp1--->| |-- PCRpt,lsp1,up-+----------------->| | |-- PCRpt,lsp1,up-+------------------+------------------>| | | |----PCRpt,lsp1,up->| | | | | | | | | | |<---PCUpd, lsp2---|-------------------| | | |<--- PCUpd, lsp2 --| | |-- PCRpt,lsp2,up--+------------------>| | | |<---PCRpt,lsp1,up--| | |-- PCRpt,lsp2,up->| | | | | | At the time of failure of one of the PCEs (say PCE1), the other PCE (PCE2) may take up the load.

The computation mechanism and how PCE chooses to handle the sticky resources during computation is out of scope of this document. This document does not tackle the issue about TED synchronization which is described in detail in .

The format of PCRpt message is defined in . It specifies the PCRpt message is sent from PCC to PCE in reporting the LSP state. This document extends the usage of PCRpt message between redundant stateful PCEs for realtime LSP synchronization as described in . A unique PLSP-ID needs to be generated at the PCE and should also carry the PCC generated PLSP-ID along in a REALTIME-SYNC TLV in the LSP object.

The format of PCUpd Message is defined in . It specifies the PCUpd message is sent from PCE to PCC to request changes in LSP attributes. This document extends the usage of PCUpd message between stateful PCEs for realtime LSP synchronization as described in . Whenever there is a PCUpd message sent from PCE to PCC, PCE should also send it to other PCEs along with the PCC generated PLSP-ID in a REALTIME-SYNC TLV in the LSP object.

As per , STATEFUL-PCE-CAPABILITY TLV can be used in the OPEN object for stateful PCE capability negotiation. A stateful PCE must announce during PCEP session establishment that they support PCEP stateful PCE extensions defined in . A new flag is added - if set to 1 by PCE, the PCE has the capability for realtime synchronization between PCEs. In case of PCC, this bit has no meaning and is simply ignored.

describes 'Speaker Entity Identifier TLV' to be included in OPEN object. This document uses the same TLV in the LSP object for realtime sync between redundant stateful PCEs. For a PCE that supports realtime sync (REALTIME-SYNC-PCE R flag in Stateful PCE Capability TLV), the PCC MUST include 'Speaker Entity Identifier TLV' in the OPEN message. Note a PCC may have to bring down the current session and include the TLV in the subsequent open message. Any realtime state synchronization (PCRpt or PCUpd message between PCEs) MUST include 'Speaker Entity Identifier TLV' in LSP object with the PCC's speaker identity. If the TLV is missing, the PCE will generate an error with error-type 6 (mandatory object missing) and error-value TBD1 (Speaker Entity Identifier TLV missing) and close the session. The format of Speaker Entity Identifier is defined in .

PCC uses the PLSP-ID in LSP object to uniquely identify an LSP. For PCE to PCE realtime sync, another unique PLSP-ID needs to be generated at the PCE and should also carry the PCC's generated PLSP-ID in the REALTIME-SYNC TLV. This way redundant PCE can correlate the LSP from the state received from the PCCs. This TLV MUST be encoded in the PCRpt and PCUpd message between redundant stateful PCEs. If the TLV is missing, the PCE will generate an error with error-type 6 (mandatory object missing) and error-value TBD2 (REALTIME-SYNC TLV missing) and close the session. The format of the REALTIME-SYNC TLV is shown in the following figure:

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type=[TBD3] | Length=4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PCC's PLSP-ID | reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The type of the TLV is to be assigned by IANA and it has a fixed length of 4 octets. The value contains the following fields: The PCC's original PLSP-ID as received in the PCRpt message from the PCC. This along with Speaker Entity Identifier TLV can be used to co-relate information received from the network (PCCs).

and describe the mechanism to advertise the PCE Discovery information via OSPF and IS-IS respectively along with processing rules for the sub-TLVs. further enhances the optional PCE-CAP-FLAGS sub-TLV used to advertise PCE stateful capabilities. Further a new bit is added -

If this bit is set to 1, the PCE has the capability for realtime synchronization between PCEs.

describes the setup and teardown of PCE-initiated LSPs under the active stateful PCE model. As the PCE sends PCInitiate message to PCC to create or delete LSP, the PCE should also send PCUpd message to other PCEs. For the initiation, the PCUpd message should have PCC's PLSP-ID as zero. The rest of the processing remains unchanged.

This document does not introduce any new security concerns besides those in .

A PCE may be deployed to act only as a backup (), an operator SHOULD be able to configure a PCE as backup.

describes the PCEP MIB, there are no new MIB Objects for this document.

Mechanisms defined in this document do not imply any new liveness detection and monitoring requirements in addition to those already listed in .

Mechanisms defined in this document do not imply any new operation verification requirements in addition to those already listed in .

Mechanisms defined in this document do not imply any new requirements on other protocols.

Mechanisms defined in this document do not have any impact on network operations in addition to those already listed in .

As discussed in , a new STATEFUL-PCE-CAPABILITY TLV Flag Field has been defined. IANA has made the following allocation from the PCEP "STATEFUL-PCE-CAPABILITY TLV Flag Field" sub-registry:

As discussed in , a new bit is added, IANA is requested to allocate a new bit in "PCE Capability Flags" registry for backup stateful PCE capability as follows:

This document defines the following new PCEP TLV:

IANA is requested to make the following allocation in the "PCEP-ERROR Object Error Types and Values" registry.

Thanks to Adrian Farrel and Daniel King for writing . We would like to thank Avantika Kumar for her useful comments and suggestions.