BGP UPDATE for SD-WAN Edge DiscoveryFutureweiDallas, TXUSAldunbar@futurewei.comHuaweiUSAshares@ndzh.comOracleCaliforniaUSAkmajumdar@microsoft.comArrcusUSArobert@raszuk.netAristaUSAvenkit@arista.comThe document describes the BGP mechanisms for SD-WAN (Software Defined Wide Area Network)
edge node attribute discovery. These mechanisms include a new
tunnel type and sub-TLVs for the BGP Tunnel-Encapsulation Attribute [RFC9012] and
set of NLRI (network layer reachability information) for Software-Defined Wide-Area
Network (SD-WAN) underlay information.
In the context of this document, BGP Route Reflector (RR) is
the component of the SD-WAN Controller that receives the BGP UPDATE from SD-WAN edges
and in turn propagates the information to the intended peers
that are authorized to communicate via the SD-WAN overlay network.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in
when, and only when, they appear in all
capitals, as shown here.BGP [RFC4271] can be used as a control plane for a SD-WAN to support Secure VPNs or simply to support
a set of Secure links in a network. This section provides an overview of (1) L3VPN services supported by SD-WAN overlays and (2) SD-WAN Secure Links as an alternative tunneling mechanism for simpler deployments. Section 2 describes the BGP framework in terms
of NLRIs supported, example topologies, and objectives for the BGP mechanisms.
Section 3 describes the BGP mechanisms, and section 4 describes error handling
for this mechanism.
The BGP mechanisms supporting SD-WAN require that the Route Reflector (RR) establish a secure transport connection with each SD-WAN edge operating under the same BGP control plane instance.
The establishment of a Secure Transport Connection between each BGP Peer and the RR is
outside the scope of this specification.
This document describes the BGP mechanisms for SD-WAN edge nodes to established
a BGP peering with other SD-WAN edge nodes, and pass information in order to establish
and update SD-WAN overlay tunnels, as described in [Net2Cloud]. These mechanisms include a new
tunnel type and sub-TLVs for the BGP Tunnel-Encapsulation Attribute [RFC9012] and
a set of NLRIs for SD-WAN underlay information.
A SD-WAN network defined in [MEF70.1] and [MEF70.2], refers to a policy-driven network
over multiple heterogeneous underlay networks to get better WAN bandwidth
management, visibility, and control. In many deployments, L3VPN services are offered over SD-WAN overlays to provide site-to-site connectivity with traffic segmentation, security, and performance guarantees. These L3VPN services leverage SD-WAN Secure Links, i.e. encrypted data plane tunnels established between SD-WAN edge nodes using mechanisms such as IPsec, to carry user traffic between endpoints.This document describes the BGP mechanisms used to support such L3VPN deployments by enabling SD-WAN edge nodes to advertise underlay attributes, tunnel characteristics, and security association metadata. These mechanisms enable dynamic tunnel selection, service-level steering, and secure endpoint discovery.The SD-WAN usage model, including its deployment scenarios and BGP requirements, is detailed in and not repeated here. This document focuses solely on the signaling extensions and encapsulation mechanisms required to support those scenarios in BGP.
defines a BGP mechanisms
that links routes (prefix and Next Hop) to a specific tunnels using a specific encapsulation.
The SD-WAN Secure Links Topology uses a single Hybrid logical link on
a SD-WAN Peer to represent multiple underlay topology links. The SD-WAN
peer distributes IPsec security association (IPsec-SA) [RFC4301] related information
regarding the hybrid link or individual underlay links.
The traffic is routed via normal IPv4/IPv6 forwarding without
any VPN addition. The SD-WAN Secure Links provides some link security for some simple
cases of the three scenarios from [SD-WAN-BGP-USAGE] that do not require L3VPN addresses
(RD, prefix).
The following acronyms and terms are used in this document: A BGP peer that a given BGP speaker is permitted to exchange SD-WAN-related information with, based on local configuration or policy. This term is used in the context of SD-WAN edge discovery to indicate that not all discovered peers are automatically eligible for overlay connectivity. Off-premises data centers that host applications and workloads, typically across multiple tenants or organizations, as defined in [Net2Cloud].Refers to the SD-WAN Controller as defined in [SD-WAN-BGP-USAGE]. It is responsible for managing SD-WAN control-plane operations, including overlay path setup, teardown, policy enforcement, and route advertisement via BGP.Customer Premises Equipment that participates in the SD-WAN overlay network. As defined in [SD-WAN-BGP-USAGE], the term is used in this document to emphasize the role of a Provider Edge (PE) device functioning as the SD-WAN edge node, responsible for forming secure overlays across diverse underlay networks within the provider's infrastructure. Virtual Private Secure network formed among C-PEs. This is to differentiate such VPNs from most commonly used PE-based VPNs discussed in [RFC4364].IPsec Security Association [RFC4301].A BGP Path Attribute used to advertise reachability information for multiple address families, as defined in . This document uses MP_REACH_NLRI for carrying SD-WAN-specific NLRIs. Software-Defined Wide Area Network, as defined in [MEF 70.1] and [MEF 70.2], refers to a policy-driven overlay connectivity service that operates over one or more underlay networks.A single logical tunnel that combines several links of different
encapsulation into a single tunnel. This logical tunnel MAY exist as part of a SD-WAN Secure L3VPN or
simply be a SD-WAN secure link for a flat network. A transport layer security mechanism (e.g., IPsec, TLS, or SSL) layered under the BGP session to provide confidentiality, integrity, and authenticity of routing updates over untrusted networks. Abbreviation for Tunnel Encapsulation Path Attribute [RFC9012].Used in this document for brevity.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC8174] when, and only when, they appear in all capitals, as shown here.
This section provides a framework to describe the overall solution
parts based on the reference diagram shown in Figure 1.
The framework covers the following: client routes supported, example
topologies, objectives of the SD-WAN Edge discovery, comparison of
SD-WAN Secure VPNs with Pure IPsec VPNS, and what SD-WAN segmentation means in
this SD-WAN context.
Tunnel Encapsulation Attribute [RFC9012] MAY be attached to the prefixes carried in BGP UPDATEs for IPv4 and IPv6 unicast routes (AFI/SAFI 1/1 and 2/1), as well as for IPv4 and IPv6 L3VPNs (AFI/SAFI 1/128 and 2/128), as specified in [RFC4364] and [RFC4659]. In the context of this specification, these NLRIs are used to signal SD-WAN tunnel information and endpoint properties associated with specific prefixes.
As described in Section 1.1, this document builds on the SD-WAN Secure VPN model defined in [SD-WAN-BGP-USAGE], [MEF-70.1], and [MEF70.2] to support L3VPN services. In this context, SD-WAN Secure L3VPNs leverage IPv4 and IPv6 L3VPNs ([RFC4364], [RFC4659]) extended with SD-WAN-specific mechanisms, including SD-WAN Hybrid Tunnel support and signaling of IPsec Security Association (IPsec-SA) metadata for underlay tunnel establishment.
As described in Section 1.2, the SD-WAN Secure Links model uses SD-WAN tunnels to create secure hybrid overlay connections between SD-WAN edge nodes. Unlike SD-WAN Secure L3VPNs, this application does not identify VPNs via the L3VPN NLRI. Instead, SD-WAN edge nodes use BGP to advertise unicast IPv4 and IPv6 prefixes (AFI/SAFI 1/1 and 2/1) along with the Tunnel Encapsulation Attribute (TEA) [RFC9012] containing an SD-WAN Hybrid Tunnel TLV (see Section 3.1.1). The SD-WAN Secure Links model also includes IPsec-SA metadata for underlay tunnels carried in BGP.
This document specifies the BGP mechanism for SD-WAN Secure L3VPN
and SD-WAN Secure Links.
This section describes how the topologies for SD-WAN Secure L3VPN
and SD-WAN Secure Links can be served by the BGP SD-WAN logical
Tunnel links. This section summarizes how a BGP-based control plane, using the BGP Tunnel Encapsulation Attribute [RFC9012], can support three deployment scenarios described in [SD-WAN-BGP-USAGE]. These scenarios focus on different ways encrypted tunnels are used in SD-WAN service delivery. While Section 1.1 lists five BGP-specific control-plane requirements, the following list highlights service-level SD-WAN deployment models that BGP can support through tunnel advertisement and associated metadata.Homogeneous Encrypted SD-WAN, Differential Encrypted SD-WAN, and Private VPN PE based SD-WAN. Based on , it is easy
to see how Figure 1 matches scenario 1 (Homogeneneous Encrypted SD-WAN)
and scenario 2 (Differential Encrypted SD-WAN). Recasting Figure 1
as a logical BGP peering topology results in a BGP topology
between PEs and CN (CE at customer site) results in Figure 2.
Scenario 3 (Private VPNs based on SD-WAN) from
aligns with the Figure 2.
Hybrid SD-WAN tunnel infrastructure requires that the
PEs (C-PE1, C-PE2, C-PE3, and C-PE4) have an existing topology
that the Hybrid SD-WAN tunnel overlays. These PEs use local policy
to sending the appropriate traffic across the appropriate network
based on local policy.
Sample Topology
Logical BGP Topology for SD-WAN
Note that the Figure 1 SD-WAN node BGP peers (C-PE) are connected in the underlay
by both trusted VPNs and untrusted public networks. For trusted VPNs, IPsec Security
associations MAY not be set-up. In some circumstances,
some SD-WAN node peers MAY be connected only by untrusted public networks.
For the traffic over untrusted networks, IPsec Security Associations (IPsec SA)
must be established and maintained. If an edge node has network ports behind a NAT,
the NAT properties need to be discovered by the authorized SD-WAN peers.
Suppose the SD-WAN network topology from Figure 1 removes
the L3VPN links. Instead the links are simply a combination
of L3 WAN Networks (unsecured) and physically secure
direct L2 and L3 links. The topology in Figure 1 becomes
the underlay physical topology in Figure 3.
The unsecured links need IPSec encryptions so the
IPsec links must be configured. An SD-WAN Hybrid
tunnel allows connections between the C-PEs
(C-PE1, C-PE2, C-PE3, and C-PE) to be a single logical link.
Therefore, the logical topology in Figure 3,
reduces to the SD-WAN logical topology shown in Figure 2.
Sample Topology
For both the SD-WAN Secure L3VPN network and the SD-WAN Secure Links,
the BGP Peers are assumed to be connected to a Route Reflector via
Secure Transport Connection. For an SD-WAN deployment with multiple RRs, it is assumed that
there are Secure Transport Connection among those RRs.
How Secure Transport Connections are established between the BGP Peer and the RR,
or between the multiple RRs is out of the scope of this document.
The existing BGP UPDATE propagation mechanisms control the edge properties propagation among the RRs.For some environments where the communication to RR is highly secured [RFC7018],
IKE-less can be deployed to simplify IPsec SA establishment among edge nodes.
The objectives of SD-WAN edge discovery for an SD-WAN edge node are to
discover its authorized BGP peers and each peer's associated properties in
order to establish secure overlay tunnels [Net2Cloud]. The attributes to
be propagated for the SD-WAN Secure L3VPN case are:
the SD-WAN (client) VPN information, the attached client routes per VPN, and the properties of the underlay networks over which the client routes can be carried.
Like any VPN network, client routes within specific SD-WAN VPNs can only be exchanged between authenticated SD-WAN peer nodes. Authentication mechanisms, such as those provided by IPsec or TLS, ensure that only valid peers participate in route exchange. The attributes to be propagated for the SD-WAN Secure L3 Links are:
the attached client routes and the properties of the underlay networks over which the client routes can be carried. The properities of the underlay network MAY include the following:
IPsec-SA information, information needed for NAT transversal with IPsec, and link speeds.
Some SD-WAN peers are connected by both trusted VPNs and untrusted public networks.
Some SD-WAN peers are connected only by untrusted public networks.
For the traffic over untrusted networks, IPsec Security Associations (IPsec SA)
must be established and maintained. For the trusted VPNs, IPsec Security
associations MAY not be set-up. If an edge node has network ports behind a NAT,
the NAT properties need to be discovered by the authorized SD-WAN peers.
Suppose the environment is Figure 1 with the logical
SD-WAN Hybrid link topology of Figure 2. These
topologies are set-up via configuration with IPsec-SA
pre-configured. Suppose that C-PE1 and C-PE2 have 10 pre-shared keys
to use on IPsec links. Currently P3 is using IPSec-SA ID-1.
C-PE1 wants to receive traffic flowing from C-PE2 over the Hybrid SD-WAN links.
Refering to the Figure 2,
the C-PE1 routers send customer routes (L3VPN v4 route) to the RR with
NextHop set to self (C-PE1), and attaching a Tunnel Encapsulation attribute specifying a
Hybrid SD-WAN tunnel to the RR over a Secure Transport Connection Based on RR policy,
the RR forwards routes to other BGP peers that support
SD-WAN discovery for that AFI/SAFI (L3VPN v4).
Using the Figure 2 example with the L3VPN for v4, RR would
forward client routes with TEA of SD-WAN tunnel TLV and Next-Hop of C-PE1. Prior to CE-P2
installing the L3VPN Unicast Routes (1/128),
the C-PE2 MUST verify at least one of the underlay connections exist
between C-PE1 and C-PE2. Local policy will define which
links (or links) the traffic for this route goes over
between C-PE1 and C-PE2.
Suppose for some reason the L2 link between C-PE1 and C-PE2
has encounter attacks, and the IPsec encryption must now
run on links P3 and P4. C-PE1 detects the problem and
to change the encryption on P3 and add encryption on P4.
C-PE1 and C-PE2 MUST agree upon the next encryption
on the list, and will send the IPsec-SA information in-band via BGP.
C-PE1 sends two SD-WAN NLRIs in an BGP Update
with a TEA with SD-WAN Hybrid Tunnel TLV with
an IPsec-SA-ID TLV with 4 IPsec SA Identifiers (4, 5, 6, 7).
The first SD-WAN NLRI contains Port P3's information
(Port-local-ID P3, SD-WAN Color Gold (1), and SD-WAN Node-ID CE-PE1).
The second SD-WAN NLRI contains Port P4's information
(Port-local-id P4, SD-WAN Color Gold (1), and SD-WAN Node-ID CE-PE1).
The RR
forwards the 2 underlay routes to C-PEs (C-PE1, C-PE2, C-PE3, C-PE4).
After receiving C-PE1 update, C-PE2 process the update, and
switches the IPsec-SA on Port 3 to IPsec-SA 4, and the IPsec-SA on
P4 to IPsec-SA 4.
This simple examples shows the value of rotating the pre-shared keys.
Future IPsec SA can also be set-up, negotiated, or rekeyed in the same manner.
The following question MAY occur to the observant reader:
Why is IPsec related information passed on a different AFI/SAFI? Why do you do to support IPsec combined with NATs? Is this the best way to support NATs? The BGP SD-WAN Nodes can attach the TEA with a Hybrid SD-WAN TLV
attached to the client routes (AFI/SAFI 1/1, 2/1, 1/128, 2/128) or
attached to the SD-WAN NLRI (AFI/SAFI 1/74 or 2/74). The SD-WAN NLRI
allows the passing of IPsec information on a unique AFI/SAFI. How
implementation prioritize the processing of client routes versus
underlay routes (SD-WAN NLRI) is an implementation matter, and out
of scope for this document.
For NATs, the Extended Port Sub-TLV (see section 3.4) represents the
information the first deployment thought it needed to identify the
NAT port for a IPsec Security Association. The deployments of
this SD-WAN technology found the amount of data gave enough
information, but suggested future work might be able to
send less information. However, those revisions are outside the scope of this document.
This section describes why this document chose to pass IPSec-SA information
via a new BGP AFI/SAFI with a TEA [RFC9012] instead of using traditional IPsec mechanisms. A pure IPsec VPN has IPsec tunnels connecting all edge nodes over public networks.
Therefore, it requires stringent authentication and authorization (i.e., IKE Phase 1 [RFC7296])
before other properties of IPsec SA can be exchanged. The IPsec Security Association (SA)
between two untrusted nodes typically requires the following configurations and message exchanges:
Messages are sent to authenticate with each other. Establishing the IPsec SA requires the following set-up
Local key configuration, Setting the Remote Peer address, Information from IKEv2 Proposal directly sent to peer (encryption method, integrity sha512, etc.), andTransform set. Running routing protocol within each IPsec SA. If multiple IPsec SAs between two peer nodes are established to achieve load sharing,
each IPsec tunnel needs to run its own routing protocol to exchange client routes attached to the edges. Access Lists permit specific Local-IP1,
Remote-IP2, and other IPsec parameters.In a BGP-controlled SD-WAN network over hybrid MPLS VPN and public internet underlay networks,
all edge nodes and RRs are already connected by private secure paths. The RRs have the policies
to manage the authentication of all peer nodes. More importantly, when an edge node needs to
establish multiple IPsec tunnels to many edge nodes, all the management information can
be multiplexed into the secure management tunnel between RR and the edge node operating as a BGP peer.
Therefore, the amount of authentication in a BGP-Controlled SD-WAN network can be significantly reduced.
Client VPNs are configured via VRFs, just like the configuration of the existing MPLS VPN.
The IPsec equivalent traffic selectors for local and remote routes are achieved by
importing/exporting VPN Route Targets. The binding of client routes to IPsec SA is dictated by policies.
As a result, the IPsec configuration for a BGP controlled SD-WAN (with mixed MPLS VPN) can be simplified
in the following manner:The SD-WAN controller has the authority to authenticate edges and peers
so the Remote Peer association is controlled by the SD-WAN Controller (RR). The IKEv2 proposals (including the IPsec Transform set)
can be sent directly to peers, or incorporated in a BGP UPDATE. The BGP UPDATE announces the client route reachability through
the SDWAN hybrid tunnels. A SDWAN hybrid tunnel combines several
other tunnels into a single logical tunnel. The SD-WAN Hybrid
tunnel implementations insure that all tunnels within are
either running over secure network links or secured by IPsec.
Importing/exporting Route Targets under each client VPN (VRF) achieves the traffic selection
(or permission) among clients' routes attached to multiple edge nodes. Note: The web of SDWAN hybrid tunnels in a network is denoted in this
document as an SD-WAN underlay. BGP passes information about the
SDWAN hybrid tunnels between BGP peers by passing an
SD-WAN Underlay NLRI paired with the tunnel encapsulation attribute (TEA)
with an SDWAN tunnel type SD-WAN-Hybrid TLV.
Also, note that with this method there is no need to run multiple routing protocols in each IPsec tunnel.
In SD-WAN Secure L3VPN deployments, SD-WAN Segmentation is a frequently
used term which refers to partitioning a network into multiple subnetworks,
just like MPLS VPNs. SD-WAN Segmentation is achieved by creating
SD-WAN virtual topologies and SD-WAN VPNs.
An SD-WAN virtual topology consists of a set of edge nodes
and the tunnels (a.k.a. underlay paths) interconnecting those edge nodes. These tunnels forming the
underlay paths can be IPsec tunnels, or MPLS VPN tunnels, or other tunnels.
Figure 4 (top) illustrates an SD-WAN L3VPN underlay topology and Figure 4 (bottom)
show the same topology as the virtual topology based on SD-WAN Links.
An SD-WAN VPN is configured in the same way as the VRFs of an MPLS VPN. One SD-WAN client VPN
can be mapped to multiple SD-WAN virtual topologies. A Route Target is used to differentiate between the
SD-WAN VPNs. For example, in figure 4 below, the Payment-Flow on C-PE2 is only mapped to the virtual
topology of C-PEs to/from Payment Gateway, whereas other flows can be mapped to a multipoint-to-multipoint
virtual topology. SD-WAN Controller for the SD-WAN Secure L3VPN governs the policies of mapping
a client VPN to SD-WAN virtual topologies.Each SD-WAN edge node MAY need to support multiple VPNs. Route Target is used to differentiate the
SD-WAN VPNs. For example, in the picture below, the Payment-Flow on C-PE2 is only mapped to the virtual
topology of C-PEs to/from Payment Gateway, whereas other flows can be mapped to a multipoint-to-multipoint
virtual topology. The BGP mechanisms have two functions:
A BGP peer supporting SD-WAN
re-announces the routes passed by client routers with a
next hop self and an BGP attribute indicating the SD-WAN Hybrid Tunnel. Clients routes
MAY be one of the following AFI/SAFIs: Unicast IPv4/IPv6(1/1, 2/1) and L3VPN IPv4/IPv6
(1/128, 2/128). The term "next hop self" means the routers sets the next Hop Address
to an address indicating the BGP Peer. The following two BGP Path attributes can pass the Tunnel indication:
the Encapsulation Extended Community (Encap-EC) and the Tunnel Encapsulation attribute (TEA).
A BGP peer sends the
SD-WAN NLRI v4/v6 (AFI/SAFI 1/74 or 2/74) with Next Hop set to Next-Hop-self
and an BGP attribute indicating the Hybrid Tunnel. The SD-WAN NLRI identifies
the port or (ports) within the Hybrid SD-WAN Tunnel that the BGP peer
is sending encapsulation or IPsec-SA related information via the Hybrid SD-WAN TEA.
The Hybrid SD-WAN TEA contains the IPsec-SA and optionally NAT information.
This section describes the Hybrid SD-WAN tunnel, the SD-WAN NLRIs,
the new sub-TLVs for SD-WAN Tunnel IPSec-SA, sub-TLVs for Port attributes,
the procedures for the client routes, the procedures for underlay routes,
error handling, and considerations for managing SD-WAN technologies.
SD-WAN Hybrid Tunnel 25 (IANA assigned) The SD-WAN-Hybrid tunnel identifies a hybrid
tunnel that overlays a virtual path over a set of links between two BGP Peers
comprised of various technologies (e.g. MPLS, L2 direct link, or L3 through Public
Internet, and others). These links between the two BGP peers are denoted
as underlay links. The underlay links MAY or MAY not need encryption.
If some of these underlay links need encryption, the SD-WAN Hybrid Tunnel
provides a mechanism to pass this information via BGP. Passage of the
IPsec-SA information is optional.Per , the following two BGP attributes that MAY
encode a Tunnel Encapsulation attribute information: the Tunnel Encapsulation Attribute,
and the Encapsulation Extended Community (Encap-EC) as a "barebones" tunnel identification.
The encoding for the SD-WAN Hybrid tunnel is described for both BGP attributes.
The NextHop Field in the BGP update is the tunnel egress Endpoint,
and this SHOULD be set to the BGP Peer Address for the SD-WAN Peer.
The Encap-EC format (barebones)
of the tunnel encapsulation attribute cannot attach any sub-TLVs. The valid sub-TLV for the TEA
form of the Hybrid SD-WAN Tunnel TLV depends whether the TEA is attached to a client route or an underlay route.
Table 1 provides a list of valid sub-TLVs per NLRI type (client or underlay).
The valid sub-TLVs specified [RFC9012] are Color (3) and Tunnel Egress End Point (6).
The valid sub-TLV specified in this document are: Extended Port Attributes, IPsec SA-ID, IPSec SA Rekey,
IPsec Public Key, IPsec SA Proposal sub-TLV, and
Simplified IPsec SA SubTLV.
The validation procedure for the SD-WAN
tunnel TLV has the following components:
1) validation of tunnel TLV encoding,2) Check that sub-TLVs are valid for NLRI (see Table 1), and 3) Egress tunnel End Point Check.
Prior to installing a route with a SD-WAN tunnel as an
active route, the BGP peer installing the route MUST
also validate that the SD-WAN tunnel and underlay links are active.
Notes
*1 - For client traffic, the Color sub-TLV defined in this document must be validated using the same procedures specified in [RFC9012] for the Color Extended Community. When both the Color Extended Community and the Color sub-TLV are present, the value in the Color Extended Community [RFC9012] takes precedence and must be used for forwarding and policy decisions.
*2 - In the future, the SD-WAN TEA could specify more details on Underlay links beyond protocol.
However, those future extensions are outside the scope of this document. *3 - Encodings and validation of syntax are defined in
section 3.3. Section 3.5 provides content processing and
validation procedures for client routes, and section 3.6
provides content processing and validation procedures for underlay routes *4 - Encoding and mechanisms are defined 3.3.6. Section 3.6
provides error handling procedures in case of sub-TLV being MALFORMED
or included with the wrong NLRI.
The Edge BGP Peer using BGP SD-WAN discovery sends the hybrid SD-WAN NLRI with the SD-WAN Hybrid
tunnel attribute to advertise the detailed properties associated with the
public facing WAN ports and IPsec tunnels. The Edge BGP Peer sends this information to
its designated RR via the Secure Transport Connection. Each BGP Update message with a
SD-WAN Underlay NLRI MUST contain a Tunnel Encapsulation Attribute (TEA) for a Hybrid Tunnel type. The TEA
can include sub-TLVs for Extended Port attribute (see section 7) or IP Sec information (see section 8).
The IPsec information sub-TLVs include: IPsec-SA-ID, IPsec SA Nonce, IPsec Public Key,
IPsec SA Proposal, and Simplified IPsec SA.
A new NLRI SAFI (SD-WAN SAFI=74) is introduced within the MP_REACH_NLRI Path
Attribute of [RFC4760] for advertising the detailed properties of the SD-WAN tunnels
terminated at the edge node:where:
2 octet value to define the encoding of the rest of the SD-WAN the NLRI.
The SD-WAN Secure Links use case MAY have many different formats, the route type was assigned two octets.
2 octets of length expressed in bits as defined in [RFC4760].This document defines the following SD-WAN Route type: For advertising the detailed properties of the SD-WAN
tunnels terminated at the edge, where the transport network port can be uniquely identified by a
tuple of three values (Port-Local-ID, SD-WAN-Color, SD-WAN-Node-ID).
The SD-WAN NLRI Route-Type =1 has the following encoding:
SD-WAN edge node Port identifier, which is locally significant.
If the SD-WAN NLRI applies to multiple WAN ports, this field is zero. represents a group of tunnels, which correlate
with the Color-Extended-community included in the client routes UPDATE.
When a client route can be reached by multiple SD-WAN edges co-located at one site,
the SD-WAN-Color can represent a group of tunnels terminated at those SD-WAN edges co-located at the site.
If an SD-WAN-Color represents all the tunnels at a site, then the SD-WAN-Color effectively represents the site. The node's IPv4 or IPv6 address. For IPv4 SD-WAN NLRI (1/74)
the length is 4. For IPv6 SD-WAN NLRI (1/76), the length is 16. Route Type values outside of 1 are out of scope for this document.
The SD-WAN Node-ID field carries the Tunnel Endpoint.
The Tunnel Egress Endpoint in the Tunnel Encapsulation Attribute (TEA)
is not used to validate the tunnel indicated by the SD-WAN NLRI.
The Next Hop within the UPDATE message MAY be tested by local policy
for validity as a BGP Peer source, but the validation of the tunnel
endpoint depends solely on the SD-WAN Node-ID.
The BGP Path Attributes for the SD-WAN NLRIs which are required to be supported are:
Origin, AS_PATH, Next_HOP, MULTI_EXIT_DISC, LOCAL_PREF (),
Originator_ID and Cluster-LIST (), Tunnel Encapsulation Attribute (), Extended Communities ().
Per , the Encapsulation Extended Community
and the Color Extended Community MUST be supported.
The following optional BGP attributes MAY be attached to an BGP UDPATE with
the Hybrid SD-WAN NLRI in the MP-REACH-NLRI or MP_UNREACH_NLRI:
Community (), Large Community (), IPv6 Address Specific Extended Community (), AS4_Path and AS4_Aggregator () - ensures support of 4-byte ASNs where needed.
These attributes are extensions of required AS_PATH and BGP
Community support that MAY be used to set in local policy evaluations for
the Hybrid SD-WAN Underlay. However, the actions of local Policy
regarding these BGP attributes is outside scope of this document.
All other optional BGP attributes MAY also be attached to the NLRI,
but the meaning of these attributes when attached to the
NLRI is outside the scope of this document.
This section describes the sub-TLVs that pass data regarding IPsec parameters for
the Hybrid SD-WAN tunnel. During set-up of the Hybrid SD-WAN tunnels, the IPsec
parameters need to be securely passed to set-up secure association. For hybrid SD-WAN tunnels,
the IPsec security association for IPsec links MAY change to different
security associations over time.
The sub-TLVs related to IPsec supported by the TEA TLV for the SD-WAN Hybrid Tunnel type are:
IPSec-SA-ID, IPsec SA Nonce, IPsec Public Key, IPsec Proposal, and Simplified IPSec SA.
The IPSec-SA-ID sub-TLV provides a way to indicate the IPsec SA Identifiers (section 3.3.1)
for pre-configured security association. The other four sub-TLVs provide different ways to pass details regarding IPsec security associations.
The IPsec SA Nonce passes Nonce and rekey counters for a Secure Association identified by IPsec SA Identifier
(see section 3.3.2). The IPSec Public Key sub-TLV passes IPsec Public Key data with a time duration
(see section 3.3.3). The IPsec Proposal sub-TLV provides Transform attributes and Transform IDs
(see section 3.3.4). The Simplified IP SEC SA passes the information that identifies configuration
for 2 keys (see section 3.3.5). The NAT-related portion infromation is carried in the Extended Port
sub-TLV (see section 3.3.6)
If any sub-TLV is MALFORMED, the implementation MUST follow the procedure in
in section 13.
For a quick rotation between security associations, the SDWAN NLRI (port-id, color, node)
can quickly distribute a switch to a set of new security association using the BGP Update message.
In this case, the BGP UPDATE message would like Figure 10 IPsec-SA-ID - Send IPsec SA-IDs 64 (IANA assigned) IPsec-SA-ID Sub-TLV within the Hybrid Underlay
Tunnel UPDATE indicates one or more pre-established IPsec SAs by using their identifiers,
instead of listing all the detailed attributes of the IPsec SAs.
Using an IPsec-SA-ID sub-TLV not only greatly reduces the size of BGP UPDATE messages,
but also allows the pairwise IPsec rekeying process to be performed independently.
where:
IPSec-SA-ID Sub-Type (8 bits): 64(IANA Assigned). length (8 bits): Specifies the total length in octets of
the value field (not including the Type and Length fields).
For the IPSec-SA-ID Sub-Type, the length SHOULD be the 2 + 4 *(number of IPsec SA IDs) .
Reserved: Reserved for future use. In this version
of the document, the Reserved field MUST be set to zero and MUST be
ignored upon receipt. Received values MUST be propagated without
change. Value field: The value field consists of a sequence of IPsec SA Identifiers, each 4 octets long. As shown in figure above, n IPsec SAs are attached in the IPsec-SA-ID sub-TLV:
IPSec SA Identifier #1: A 4 octet identifier for a pre-established IP security association.
IPSec SA Identifier #2: A 4 octet identifier for a pre-established IP security association.
IPSec SA Identifier #n: A 4 octet identifier for a pre-established IP security association.
A IPsec-SA-ID Sub-TLV is a MALFORMED if the fields do not fit the limits
specified above. Per a MALFORMED Sub-TLV is ignored.
The procedures for content check for the IPsec-SA-ID Sub-TLV is specified in section 3.4 for client routes,
and section 3.5 for underlay routes. The IPsec-SA-ID Sub-TLV does not
help validate the Tunnel Egress EndPoint. However, the IPSec-SA provides the security
information associated with set of routes sent from a peer which MUST be correctly
handled upon reception of the data. IPsec SA ReKey Counter - Rekey Counter for a IPsec SA 67 (IANA assigned) The IPsec SA Rekey Counter Sub-TLV provides the rekey counter
for a security association (identified by IPsec SA Identifier). The encoding is shown in the figure below:
where:
IPSec-SA-ID Sub-Type (8 bits): 67 (IANA assigned) length (8 bits): Specifies the total length in octets of the value field
(not including the Type and Length fields). The total length is variable
with the value equal to 16 plus Nonce length.
Reserved: Reserved for future use. In this version
of the document, the Reserved field MUST be set to zero and MUST be
ignored upon receipt. Received values MUST be propagated without
change. ID Length (8 bits): indicates the length in octets of SA-Identifer. This length SHOULD be 4 octets.
Nonce Length (16 bits): indicates the length in octets of the Nonce Data.
I Flag: when set to 1, the I-flag indicates that the communication being established is new. when set to 0, it signals that the communication is a continuation of an existing session.
Flags (7 bits): Reserved for future use. In this version
of the document, the Reserved field MUST be set to zero and MUST be
ignored upon receipt. Received values MUST be propagated without
change. Rekey Counter (64 bits): the number of key updates or rekeys that have occurred. Each time a key is rotated or replaced, the ReKey Counter is incremented.IPsec SA Identifier (IPSec-SA-ID): a specific IPsec SAs terminated at the edge. The length of this
field is specified in ID Length. For this specification, the length is 4. Other lengths are outside
of the scope of this document.Nonce Data: a random or pseudo-random number for preventing replay attacks.
Its length is a multiple of 32 bits[RFC7296].
A IPsec SA Rekey Counter Sub-TLV s a MALFORMED if the fields
do not fit the limits specified above. Per a MALFORMED SubTLV is ignored.
The procedures for content checks for this sub-TLV are specified in section 3.4 for client routes,
and section 3.5 for underlay routes. Please note that:
The IPsec-SA-ID MAY also refer to the values carried in the same TEA in
the same Tunnel TLV (type SD-WAN Hybrid) as the IPsec SA Rekey Counter sub-TLV
in either the IPsec Public Key sub-TLV or the IPsec SA Proposal sub-TLV.
The IPsec SA Rekey Counter, IPsec Public Key, and IPsec SA Proposal
Sub-TLVs work together to create security associations.
The IPsec-SA-ID MAY refer to an IPsec SA-ID in another SD-WAN Hybrid Tunnel TLV in the
same TEA as the IPsec SA Rekey Counter sub-TLV,
The IPsec SA-ID MAY refer to a preconfigured IPsec SA-ID preconfigured
associated with this SD-WAN Hybrid Tunnel Endpoint.
The IPsec SA Rekey SubTLV
does not help validate the Tunnel Egress EndPoint. IPsec Public Key - IPsec Public Key information 68 (IANA assigned) The IPsec Public Key Sub-TLV provides the
Public Key exchange information and the life span for the Diffie-Hellman Key. The encoding is shown in the figure below:
where:
IPSec-SA-ID Sub-Type (8 bits): 68 (IANA assigned) length (8 bits): Specifies the total length in octets of the value field
(not including the Type and Length fields). The total length is variable
with the length being 14 + the Key Exchange Data length.
Diffie-Hellman Group Num (16-bits): identifies the Diffie-Hellman group used to compute the Key Exchange Data. Details on Diffie-Hellman group numbers can be found in Appendix B of IKEv2 [RFC7296] and [RFC5114]. The Key Exchange data: This refers to a copy of the sender's Diffie-Hellman public value.
The length of the Diffie-Hellman public value is defined for MODP groups in [RFC 7296]
and for ECP groups in [RFC 5903].
Duration (32 bits): a 4-octet value specifying the life span of the Diffie-Hellman key in seconds.
A IPsec Public Key SubTLV is a MALFORMED SubTLV if the fields
do not fit the limits specified above. Per a MALFORMED SubTLV is ignored.
The procedures for content checks for the IPsec Public is specified in section 3.4 for client routes,
and section 3.5 for underlay routes.
The IPsec SA Rekey SubTLV
does not help validate the Tunnel Egress EndPoint. IPsec SA Proposal - Indicates number of IPsec Transforms 69 (IANA assigned) The IPsec SA Proposal Sub-TLV is to indicate the number of Transform Sub-TLVs.
The encoding is shown in the figure below:
where:
IPSec-SA-ID Sub-Type (8 bits): 69 (IANA assigned) length (8 bits): Specifies the total length in octets of the value field
(not including the Type and Length fields). The total length is variable
with the length being 10 + the Transform attribute length.
Reserved-Cnt (16 bits): reserved for future use. These bits are ignored upon receipt
and set to zero when transmitted. Transform Attr Length (16 bits): indicates the length of the Transform Attributes field.
Transform Type (8 bits): The Transform Type values are defined in Section 3.3.2 of [RFC7296] and [IKEV2IANA].
Only ENCR, INTEG, and ESN values are permitted.
Reserved (8 bits): reserved for future use. These bits are ignored upon receipt and set to zero when transmitted.
Transform ID (16 bits): An identifer for the transform defined by the associated transform attributes.
Transform Attributes: These Sub-SubTLV are defined in section 3.3.5 of [RFC7296]. A IPsec SA Proposal Sub-TLV is a MALFORMED Sub-TLV if the fields
do not fit the limits specified above. Per a MALFORMED Sub-TLV is ignored.
The procedure for content checks for the IPsec SA Proposal SubTLV is specified in
section 3.4 for client routes, and section 3.5 for underlay routes.
The IPsec SA Rekey SubTLV
not help validate the Tunnel Egress EndPoint. The Reserved-Cnt field was allocated to allow future compatiability with
field for SA-Proposals. Currently only 1 transform
is allowed per IPsec SA Proposal SubTLV. Multiple IPsec SA-Proposal SubTLVs MAY be included in a
Tunnel-encapsulation Attribute. Simplified IPsec SA - Provides simple key for 2 keys 69 (IANA assigned) For a simple SD-WAN network with edge nodes
supporting only a few pre-defined encryption algorithms, a simple IPsec sub-TLV
can be used to encode the pre-defined algorithms.
The encoding is shown in the figure below:
where:
IPSec-SA-ID Sub-Type (8 bits): 70 variable (based on key length) Reserved for future use.
These bits SHOULD be set to zero on transmission and MUST be ignored on receipt.Transform = 1 means AH,Transform = 2 means ESP, orTransform = 3 means AH+ESP.
All other transform values are outside the scope of this document.
Mode = 1 indicates that the Tunnel Mode is used.Mode = 2 indicates that the Transport mode is used.
All mode values besides 1 and 2 are outside the scope of this document.
AH authentication algorithms supported. The values are specified by [IANA-AH].
Each SD-WAN edge node can support multiple authentication algorithms; send to its peers to negotiate the strongest one. the ESP algorithms supported. Its values are specified by [IANA-ESP].
One SD-WAN edge node can support multiple ESP algorithms and send them to its peers to negotiate the strongest one.
The default algorithm is AES-256.
When a node supports multiple authentication algorithms, the initial UPDATE needs
to include the "Transform Sub-TLV" indicates the count for rekeying. indicates the IPsec public key 1 length IPsec public key 1 indicates the IPsec public key 2 length IPsec public key 2 indicates the Nonce key length IPsec Nonce specifying the security association (SA) life span in seconds.
A Simplified IPsec SA Sub-TLV is a MALFORMED Sub-TLV if the fields
do not fit the limits specified above. Per a MALFORMED Sub-TLV is ignored.
The procedures for content checks for the IPsec SA Proposal SubTLV is specified in
section 3.4 for client routes, and section 3.5 for underlay routes.
Simplified IPsec SA does not participate in
Tunnel End Point Validation. Extended Port Attribute - Provides information related to IPsec and NATs 65 (IANA assigned) The Extended Port Attribute Sub-TLV advertises the
properties associated with a public Internet-facing WAN port that might be behind NAT.
A SD-WAN edge node can query a STUN Server (Session Traversal of UDP
through Network address translation [RFC8489]) to get the NAT properties,
including the public IP address and the Public Port number, to pass to its peers.
The location of a NAT device can be:
Only the initiator is behind a NAT device. Multiple initiators can be behind separate NAT devices.
Initiators can also connect to the responder through multiple NAT devices.Only the responder is behind a NAT device.Both the initiator and the responder are behind a NAT device.The initiator's address and/or responder's address can be dynamically assigned
by an ISP or when their connection crosses a dynamic NAT device that allocates
addresses from a dynamic address pool.As one SD-WAN edge can connect to multiple peers, the pair-wise NAT exchange as IPsec's
IKE[RFC7296] is not efficient. In the BGP Controlled SD-WAN, NAT properties for a WAN port are
encoded in the Extended Port Attribute sub-TLV. The encoding is shown in the figure below:
where:
Extended Port Attribute Type (=65): indicating it is the Extended Port Attribute Sub-TLVExtPort Length: the length of the sub-TLV in octets (variable). If IPv4, the
length is 32 (8 header, 32 address, 8 for 1 subSubTLV). If IPv6, the length is 64
(8 header, 48 addresses, 8 for 1 subSubTLV). Flags:
I bit (C-PE port address or Inner address scheme):
If set to 0, indicate the inner (private) address is IPv4.If set to 1, indicates the inner address is IPv6. O bit (Outer address scheme):
If set to 0, indicate the inner (private) address is IPv4.If set to 1, indicates the inner address is IPv6. R bits: reserved for future use. MUST be set to 0, and ignored upon reception.NAT Type: the NAT type can be one of the following values:
1: without NAT ; 2: 1-to-1 static NAT; 3: Full Cone; 4: Restricted Cone; 5: Port Restricted Cone; 6: Symmetric; or 7: Unknown (i.e. no response from the STUN server).
NAT type values outside of 1-7 are invalid for this Sub-TLV.
Encap-Type: the supported encapsulation types for the port.
Encap-Type=1: GRE;Encap-Type=2: VxLAN;
Notes:
The Encap-Type inside the Extended Port Attribute Sub-TLV is different from the RFC9012's
BGP-Tunnel-Encapsulation type. The port can indicate the specific encapsulations, such as:
If the IPsec-SA-ID sub-TLV or the IPsec SA detailed sub-TLVs (Nonce/publicKey/Proposal)
are included in the SD-WAN-Hybrid tunnel, the Encap-Type indicates the
encapsulation type within the IPsec payload.If the IPsec SA sub-TLVs are not included in the SD-WAN-Hybrid Tunnel,
the Encap-Type indicates the encapsulation of the payload without IPsec encryption.Encapsulation types outside of GRE and VxLAN are outside of the scope of this specification.
The Extended Port attribute SubTLV cannot support 6to4NAT encoding.
Transport Network ID: Central Controller assigns a global unique ID to each transport network.
Any value in this octet is valid RD ID: Routing Domain ID, need to be globally unique. Any value in this octet is valid.
Some SD-WAN deployment might have multiple levels, zones, or regions that are represented as logical
domains. Policies can govern if tunnels can be established across domains. For example, a hub node
can establish tunnels with different logical domains but the spoke nodes cannot establish
tunnels with nodes in different domains.Local IP: The local (or private) IP address of the WAN port.Local Port: used by Remote SD-WAN edge node for establishing IPsec to this specific port.Public IP: The IP address after the NAT. If NAT is not used, this field is set to all-zerosPublic Port: The Port after the NAT. If NAT is not used, this field is set to all-zeros.Extended SubSub-TLV: for carrying additional information about the underlay networks. A IPsec SA Proposal Sub-TLV is a MALFORMED Sub-TLV if the fields
do not fit the limits specified above. Per a MALFORMED Sub-TLV is ignored.
The procedures for content checks for the IPsec SA Proposal SubTLV is specified in
section 3.4 for client routes, and section 3.5 for underlay routes.
Simplified IPsec SA does not participate in
Tunnel End Point Validation.One Extended SubSub-TLVs is specified in this document: Underlay Network Transport SubSub-TLV.
Underlay Network Transport - caries WAN port types and speed 66 (IANA Assigned) The Underlay Network Transport SubSub-TLV
is an optional SubSub-TLV to carry the WAN port connection types and bandwidth,
such as LTE, DSL, Ethernet, and other types. It is only valid for the
Tunnel Type Hybrid SD-WAN in the Extended Port Attributes Sub-TLV. The encoding is shown in the figure below:
Where:
sub Type=66 always 6 bytes 2 octet of reserved bits. It SHOULD be set to zero on
transmission and MUST be ignored on receipt. are listed below as:
1 = Wired2 = WIFI3 = LTE 4 = 5G Any value outside of 1-4 is outside the scope of this specification. Port type define as follows:
1 = Ethernet 2 = Fiber Cable 3 = Coax Cable 4 = Cellular Any value outside of values 1-4 are outside the scope of this specification.The port seed is defined as 2 octet value. The values are defined in megabit speed. For example, a value of 1000 would mean 1 gigabit (1000 Mbps). The port speed of "0" is not valid.
The connection types of equipment and port types will continue to grow with technology change.
Future specifications MAY specify additional connection types or port types.
Underlay Network Transport SubSub-TLV is a MALFORMED SubSub-TLV
if the fields do not fit the limits specified above. If a MALFORMED SubSubTLV is contained in the
Extended Port Attribute SubTLV, then the Extended Port Attribute SubTLV is MALFORMED.
Per a MALFORMED Sub-TLV is ignored.
The procedures for content checks for the IPsec SA Proposal SubTLV is specified in
section 3.4 for client routes, and section 3.5 for underlay routes.
Underlay Network Transport SubSub-TLV does NOT
contribute to Tunnel End Point Validation. The Tunnel Encapsulation Attribute for the SD-WAN Hybrid Tunnel Type
MAY be associated with BGP UPDATE messages with NLRI with AFI/SAFI
IPv4 Unicast (1/1), IPv6 (2/1), L3VPN v4 Unicast (1/128), and
IPv6 L3VPN (2/128).
The SD-WAN Secure Links topology is supported by the
Unicast IPv4/IPv6 prefixes. The L3VPN topologies support
forming the SD-WAN Secure L3VPN described in
and MEF ([MEF 70.1][MEF70.2]).
Based on [RFC9012], there are two forms a Tunnel Encapsulation
Attribute (TEA) can take: "Barebones" using the Encapsulation
Extended Community and a normal Tunnel Encapsulation form.
The SD-WAN Client routes are sent with a the Encapsulation Extended Community [RFC9012]
BGP attribute that identifies the the Hybrid SD-WAN tunnel type.
Per [RFC9012], the Encapsulation Extended Community uses the NextHop Field
in the BGP UPDATE as the Tunnel Egress EndPoint. The validation for
the Tunnel Egress Endpoint uses the validation
in sections 6, 8, and 13 applied to the NextHop.
A Color Extended Community [RFC9012] or local policy applied to the
client route directs the traffic for the client route
to across appropriate interface within the Hybrid SD-WAN Tunnel
to the Tunnel Egress Endpoint.
The procedures for validating a client route with a TEA does the following:
A well-formed Hybrid SD-WAN Tunnel TLV MUST have a Tunnel Engress Endpoint SubTLV.
The validation for the Tunnel Egress Endpoint uses the
validation in section 6, 8, and 13. An invalid Tunnel Egress Endpoint,
cause the Hybrid SD-WAN Tunnel TLV to be invalid, and the TLV is ignored.
It MAY also have any of the following SubTLVs:
the Color SubTLV defined in , IPsec-SA-ID, IPsec SA Nonce, IPsec Public Key, IPSec SA Proposal, or Simplified IPsec SA) A MALFORMED SubTLV is ignored in the Tunnel TLV is ignored.
SubTLV with an unknown type is ignored.
As specified in [RFC9012], only the first instance of a SubTLV is processed; subsequent ones are ignored.
A Hybrid SD-WAN tunnel TLV MAY have multiple instances of the IPsec-SA-ID if the IPsec SA Identifiers are unique.
If all the IPsec SA Identifies are not unique, the second SubTLV is ignored and not propagated.
The Tunnel Egress Endpoint MUST link to
the remote end of one of the underlay links to be used. This validation adheres
to the Tunnel Egress Endpoint validation. The tunnel link
MAY be active or inactive.
Local policy is run to validate routes. The next hop MUST be be reachable via the tunnel." If a tunnel encapsulation attribute is malformed, it MUST be ignored per [RFC9012]. However, in SD-WAN environments where secure tunnels are required, traffic forwarding MUST be contingent on tunnel liveness. If a required secure tunnel is unavailable, the associated route MUST NOT be installed in the forwarding table or used to forward traffic. The route MAY still exist in the BGP control plane but MUST be marked as unusable for forwarding until a valid secure tunnel is established.A single SD-WAN client route MAY be attached to multiple SD-WAN Hybrid tunnels.
An Update with an SD-WAN client route MAY express these tunnels as
an Encapsulation Extended Community[RFC9012] or a TEA. Each of these tunnel descriptions is treated as a
unique Hybrid SD-WAN tunnel with a unique Egress Endpoint.
Local Policy on the BGP Peer determines which tunnel the client data traffic will use.
An SD-WAN VPN ID is the same as a client VPN ID in a BGP controlled SD-WAN network.
The Route Target Extended Community SHOULD be included in a Client Route UPDATE message to
differentiate the client routes from routes belonging to other VPNs.
Route Target value is taken as the VPN ID (for 1/1 and 2/1).
For 1/128 and 2/128, the RD from the NLRI identifies the VPN ID.
For EVPN, picking up the VPN-ID from EVPN SAFI.
SD-WAN edge node can be reached by either
an MPLS path or an IPsec path within the hybrid SD-WAN tunnel.
If client packets are sent via a secure MPLS network
within the Hybrid SD-WAN tunnel, then the data packets
will have MPLS headers with the MPLS Labels based on the scheme specified
by [RFC8277]. It is assumed the secure MPLS network assures
the security outer MPLS Label header.
If the packets are sent via a link with IPsec outer encryption
across a public network, the payload is still encrypted with
GRE or VXLAN encryption. For GRE Encapsulation within an IPsec tunnel,
the GRE key field can be used to carry the SD-WAN VPN ID.
For network virtual overlay (VxLAN, GENEVE, etc.) encapsulation
within the IPsec tunnel, the Virtual Network Identifier (VNI) field is
used to carry the SD-WAN VPN ID.
The Hybrid SD-WAN NLRI MUST be accompanied with the TEA, and
MUST NOT be accompanied by an Encapsulation Extended Community.
An underlay routes contains Hybrid SD-WAN NLRIs with TEA attached.
The procedures for processing underlay routes follows the following steps:
An Hybrid SD-WAN Tunnel TLV is well-formed using only SubTLVs valid for association
with the underlay Route.
A well-formed SD-WAN Hybrid Tunnel TLV MUST contain a valid Tunnel Egress
Endpoint sub-TLV to be a valid SD-WAN Hybrid TLV.
provides validation guidelines for the Tunnel Egress Endpoint in sections
13.
The SD-WAN Hybrid Tunnel TLV MAY contain the following
sub-TLVs: Tunnel Egress, IPsec-SA-ID, Extended Port sub-TLV, IPsec Nonce,
IPsec Public Key, IPsec SA Proposal, and Simplified IPsec SA.
Following intent, a
MALFORMED Sub-TLV is ignored, and a sub-TLV with an
unknown type is ignored.
As specified in [RFC9012], only the first instance of a Sub-TLV is processed; subsequent ones are ignored. The IPsec-SA-ID
sub-TLVs MAY have multiple instances of the sub-TLV if the IPsec SA Identifiers are
unique, but if the IPsec SA Identifiers are not unique the second sub-TLV is ignored
and not propagated. If multiple Extended Port SubTLVs exist, the TLVs
must be validated in step 4. The Tunnel Egress Endpoint MUST link to remote end point of
one of the underlay links from the router receiving the link.
If a single
Extended Port SubTLV exist, then validate that the port information
provides needed information to establish a connection to the
remote port. A SD-WAN edge node MAY utilize local policy, local
configuration, and the information from the SubTLV. The exact
mechanism of local port validation is outside the scope of this document.
If the Extended Port SubTLV is invalid, the SD-WAN tunnel TLV
must be considered to be invalid. The typed NLRI in the SD-WAN
Underlay MUST be Well-Formed having the format specified in
section 3.2.1. A MALFORMED NLRI MUST cause the NLRI to be discarded.
An implementation MAY log an error for a MALFORMED NLRI.
The local policy configuration in the BGP peer receiving this NLRI
MUST determine the validity of the route based on policy.
The IP address specified in the
Next Hop field MUST be reachable by the Tunnels.
Section 3.2.1 specifies that Route Type 1 has a tuple of (Port-Local-ID, SD-WAN-Color, SD-WAN-Node-ID).
Port-Local-ID MAY be zero if the NLRI applies to multiple ports. The BGP Peer receiving the NLRI
MUST have pre-configured inbound filters to set the preference for the SD-WAN NLRI tuple.
Since a Port-Local-ID value of zero indicates the NLRI applies to multiple ports, it is possible
to have the following NLRI within a packet (or received in multiple packets):
Port-Local-ID (0), SD-WAN-Color (10), SD-WAN-Node-ID (2.2.2.2),
Port-Local-ID (0), SD-WAN-Color (20), SD-WAN-Node-ID (2.2.2.2), and
Port-Local-ID (0), SD-WAN-Color (30), SD-WAN-Node-ID (2.2.2.2).
These NLRI MAY simply indicate that there are three groups of tunnels
for SD-WAN-Node-ID (2.2.2.2) assigned three colors. For example,
these tunnels could represent three types of gold, silver and bronze network service.
An underlay route (SD-WAN NLRI) MAY only attach to one Hybrid SD-WAN Tunnel.
If there are more than one Hybrid SD-WAN Tunnel TLV within a single
TEA, the first is processed and the subsequent Hybrid SD-WAN Tunnel TLVs are ignored.
The Error handling for SD-WAN VPN support has two components:
error handling for Tunnel Encapsulation signaling
(Encapsulation Extended Community and TEA) and the SD-WAN NLRI. An SD-WAN NLRI,
a Tunnel Encapsulation attribute MUST always accompany the SD-WAN NLRI.
The previous sections (3.4 and 3.5) provide the normal procedures for
handling client routes and undelay routes.
The error handling for the tunnel encapsulation signaling (Encapsulation Extended Community and
TEA) in this document follows the procedures specified in [RFC9012], Sections 6, 8, and 13. Unless otherwise stated, malformed or unrecognized Sub-TLVs MUST be handled as specified in [RFC9012]. This document defines new Sub-TLVs for Tunnel Type 25 (SD-WAN-Hybrid), but does not alter the validation behavior established in RFC 9012.
The Tunnel encapsulation signaled with the client routes indicates
the Egress endpoint via Next Hop in the Encapsulation Extended Community or the TEA SubTLV
for Tunnel Egress Endpoints. As indicated in the procedure in
sections 3.4.2 and 3.5.2, the SD-WAN Hybrid tunnel follows
the validation section 6, 8, and 13 from [RFC9012].
The SD-WAN client routes associate some of the NLRIs that [RFC9012] associates
with the Encapsulation Extended Community and the TEA using the validation specified in [RFC9012] in
sections 6, 8, and 13. When the SD-WAN Hybrid Tunnel is associated with the SD-WAN NLRI, and
all RFC9012 validation rules in section 6, 8, and 13 are extended to
apply to the SD-WAN NLRI.
[RFC9012] contains the necessary detail to specify validation for
the new SubTLVs present for the SD-WAN Tunnel type. However, to aid
users of this document the following recap of validation
of [RFC9012] is provided below. The validation from section 13 of
[RFC9012] includes:
Invalid tunnel type MUST be treated if the TLV was not present.
A malformed sub-TLVs MUST be handled per [RFC9012] Section 13. If Tunnel Egress Endpoint is malformed, the
entire TLV MUST be ignored. For security-sensitive attributes, such as those related to IPsec SA setup, malformed or invalid values MUST be discarded and MUST NOT be used in security association processing. The BGP UPDATE containing such attributes SHOULD still be processed if other attributes remain valid. Implementations SHOULD log the error for operational awareness and MAY trigger a session reset or rekeying if required by local policy. Unlike general BGP attributes, failure to process security-related information correctly could lead to misconfigurations or weakened security.
Multiple incidents of Tunnel Egress Endpoint Sub-TLV cause the first incident of
these sub-TLVs to be utilized. Subsequent TLVs after the first one per type are ignored
(per RFC9012), but propagated.
If a sub-TLV is meaningless for a tunnel type, the
sub-TLV is ignored, but the sub-TLV is not considered malformed or removed
from the Tunnel Attribute propagated with the NLRI.
For SD-WAN client routes with a TEA with a SD-WAN Hybrid Tunnel type TLV,
the IPSec SubTLVs (IPsec SA-ID, IPSec nonce, IPSec Public Key, IPsec Proposal,
and Simplified IPsec SA) are meaningful, but MAY be rarely sent.
Incorrect fields within any of these 5 TLVs. Per [RFC9012], a malformed sub-TLV
is treated as an unrecognized sub-TLV.
For SD-WAN NLRI underlay routes, the Extended Port sub-TLV
and the IPSec sub-TLVs (IPsec SA-ID, IPSec nonce, IPSec Public Key, IPsec Proposal,
and Simplified IPsec SA) are valid and meaningful.
Incorrect fields within any of these 6 TLVs or subSubTLVs within the TLVs SHOULD cause the sub-TLV
to be treated as malformed sub-TLV. Per [RFC9012], a malformed sub-TLV
is treated as an unrecognized sub-TLV.
If multiple instances of the IPsec nonce, IPsec Public Key, IPsec Proposal, and
Simplified IPsec are received within a SD-WAN Tunnel TLV , only the first is processed.
The second instance is ignored and not propagated. The IPsec SA-ID MAY have multiple
copies, but the IPsec SA Identifiers sent in the second sub-TLV MUST be different
than any in the first IPsec-SA ID sub-TLV.
If multiple instances of the Extended Port sub-TLV are received, the local
policy MUST determine which is to be used.
The SD-WAN NLRI [AFI 1/SAFI = 74] utilizes a route type field to describe the format of the NLRI.
This specification only allows an NLRI with a type value of 1. An NLRI with a type of field of
another value is ignored and not processed. The implementation MAY log an error upon
the reception of a type value outside of Route Type 1. Error handling for the SD-WAN NLRI
also adheres to the BGP Update error handling specified in [RFC7606].
The local policy configuration in the BGP peer receiving this
NLRI MUST determine the validity of the route based on policy.
Local configuration and policy MUST carefully constrain
the SD-WAN-NLRI, tunnels, and IPsec security associations
to create a "walled garden".
In the future, other proposals for a SD-WAN NLRI MAY specify a different route type.
Those proposals MUST specify the following:
validation for new Route Type in the SD-WAN-NLRI, and
how the new Route Type interacts with the Route Type 1.
The SD-WAN NLRI (AFI=1/SAFI=74) MUST be paired with
Tunnel Encapsulation attribute with a tunnel TLV for
tunnel type SD-WAN-Hybrid. If the SD-WAN NLRI exist in an
BGP UPDATE without a Tunnel Encapsulation Attribute (TEA)
with a tunnel TLV for tunnel type SD-WAN-Hybrid, the
NLRI is considered malformed and Treat-as-withdraw approach (per RFC7606).
The SD-WAN NLRI SHOULD not be paired with an Encapsulation Extended Community.
If an SD-WAN NLRI is paried with an Encapsulation Extended Community rather than
a Tunnel Encapsulation Attribute, the SD-WAN NLRI is considered malformed
and the Treat-as-withdraw approach (per ) SHOULD be used.
Unlike MPLS VPN whose PE nodes are all controlled by the network operators,
SD-WAN edge nodes can be installed anywhere, in shopping malls, in 3rd party Cloud DCs, etc.It is very important to ensure that client routes advertisement from an SD-WAN edge
node are legitimate. The RR needs to ensure the SD-WAN Hybrid Tunnels and routes
run over the appropriate Security associations. It is critical that the Hybrid SD-WAN Tunnel correctly forward traffic
based on the local policy on the client routes, the tunnel egress and tunnel
ingress, and the security association. The RR reflector and the BGP peer
MUST check that the client routes, tunnel egress, tunnel ingress, and security
associations align with expected values for a tunnel.
Each BGP peer (e.g. a C-PE) advertises a SD-WAN SAFI Underlay NLRI to the other
BGP peers via a BGP Route Reflector to establish pairwise IPsec Security Associations (SA)
between itself and other remote BGP Peers. During the SD-WAN SAFI NLRI advertisement,
the BGP Peer originating MAY pass information about security association in one of
three forms:
an identifier for a pre-configured and established IPsec Security Association,
a simplified set of security parameters for setting up a IPsec Security association
(Transform, IPsec Mode, AH and ESP Algorithms, rekey counter,
2 public keys, nonce, and duration of security association), or
a flexible set of security parameters where Nonce, Public Key, and SA Proposal
are uniquely specified.
For existing IPsec Security associations, the receiving BGP peer can simply
utilize one of these existing security associations to pass data.
If multiple IPsec associations are pre-configured, the local policy
on the SD-WAN Edge Node MAY help select which security association is chosen
for the SD-WAN Hybrid Tunnel.
If the receiving and originating BGP peer engage in a set-up for the
IPsec security associations for the link within the SD-WAN Hybrid tunnel, IPsec
mechanisms require that there are matching IPsec transforms. Without common
IPsec transforms, the IPsec set-up process cannot operate.
The TEA passes in the Tunnel TLV for the SD-WAN Hybrid Tunnel these three
sets of information in the following sub-TLVs:
passes the previous configured (pre-configured or generated)
IPsec SA identifiers.
specifies a simplified set of information
upon which to set-up the IPsec security associations for the tunnel.
IPsec SA Rekey Counter SubTLV, IPsec Public Key SubTLV,
and a IPSec Proposal SubTLV. Configuration on the local node uses this information
and any information in the underlay to create security associations.
Each BGP Peer needs to send the IPSec SA attributes received on the SD-WAN NLRI in the
TEA between the local and remote WAN ports.
If there is a match on the SA Attributes between the two ports, the IPSec Tunnel is established.
If there is a mismtach on the SA Attributes, no IPsec Tunnel is established.
The C-PE devices do not try to negotiate the base IPSec-SA parameters between the local
and the remote ports in the case of simple IPSec SA exchange or the Transform sets between local
and remote ports. If there is a mismatch in the IPsec SA, then no IPsec Tunnel is created.
If there is a mismatch on the Transform sets in the case of full-set of IPSec SA Sub-TLVs,
no tunnel is created.
This section provides one example of how IPsec Security associations are created over
the SD-WAN Hybrid tunnel. Figure 1 in Section 3 shows an establish an IPsec Tunnel being
created between C-PE1 and C-PE2 WAN Ports A2 and B2 (A2: 192.168.1.10 - B2:192.168.2.1).To create this tunnel C-PE1 needs to advertise the following attributes for establishing the IPsec SA:NextHop: 192.168.1.10SD-WAN Node ID: 1.1.1.1 SD-WAN-Site-ID: 15.0.0.2Tunnel Encap Attr (Type=SD-WAN) -
Extended Port Attribute Sub-TLV containing
Transport SubSubTLV - with information on ISP3. IPSec information for detailed information about the ISP3IPsec SA Rekey Counter Sub-TLV,IPsec SA Public Key Sub-TLV,Proposal Sub-TLV (type = ENCR, transform ID = 1)
type: ENCR Transform ID: 1 Tranform attributes = trans 1 [from RFC7296]C-PE2 needs to advertise the following attributes for establishing IPsec SA:
192.168.2.1 2.2.2.2 1500 Extended Port Attribute SubTLV
Transport SubSubTLV - with information on ISP3. IPsec SA Rekey Counter Sub-TLV, IPsec SA Public Key Sub-TLV,IPSec Proposal Sub-TLV with
transform type: ENCR Transform ID = 1 Transform attributes = trans 2 As there is no matching transform between the WAN ports A2 and B2 in C-PE1 and C-PE2,
respectively, no IPsec Tunnel will be established.The BGP-based signaling mechanisms described in this document are primarily intended to enable SD-WAN edge nodes to advertise underlay transport and tunnel parameters to their Route Reflector (RR). These parameters, once received, can be monitored and validated using existing BGP monitoring tools such as BMP or route policy inspection frameworks. Operators SHOULD implement logging and alerting mechanisms for cases where inconsistent or malformed Sub-TLVs are received, as specified in Section 3.6. Misaligned parameters, such as mismatched IPsec-SA-IDs or invalid NAT indicators, should trigger operational alerts to aid troubleshooting. No new MIB modules or YANG models are introduced in this document, but implementations are expected to expose relevant state (e.g., tunnel type, advertised properties) via standard operational interfaces. The use of secure transport connections (e.g., BGP over IPsec/TLS) is RECOMMENDED to ensure manageability in untrusted environments.The document describes the encoding for SD-WAN edge nodes to advertise its properties
to their peers to its RR, which propagates to the intended peers via a Secure Transport Connection across an
untrusted network.SD-WAN edge nodes MUST use secure transport connections to the Route Reflector to protect control-plane messages, especially when operating over public or untrusted networks. Failure to secure these BGP sessions may expose the SD-WAN fabric to route injection, impersonation, or metadata leakage. The use of IPsec, TLS, or SSL is RECOMMENDED to mitigate these threats.It is assumed that the
SD-WAN edges communication will result in aligned IPsec Attributes. Please see section 4.2
for a discussion of what happens if the IPsec Attributes are mismatched.
SD-WAN edge nodes MUST establish a secure transport connection to the RR. This can be achieved by running BGP over TCP secured with IPsec, SSL, or TLS. While BGP itself does not typically run over TLS, TLS can secure the underlying transport layer in specific SD-WAN environments. This secure transport also protects nonces exchanged in BGP messages, minimizing the risk of tampering and replay attacks, particularly in multi-hop environments. In such cases, BGP sessions MAY be established over TLS-based tunnels as part of a broader transport security framework.
In a walled garden SD-WAN deployment where all SD-WAN edges and the central controller are under one
administrative control and the network operates within a closed environment, the threat model
is primarily on internal threats, misconfigurations, and localized physical risks.
Unauthorized physical access to SD-WAN edge devices in remote locations is a concern.
Such access might allow attackers to compromise the edge devices and potentially manipulate the
advertised Client prefixes with VPN IDs (or Route Targets) that do not belong to them.
This can lead to unauthorized data interception and traffic redirection.
Ensuring secure communication between SD-WAN edge nodes and the central controller within a
walled garden deployment is crucial. It is essential to utilize secure communication channels
such as TLS, SSL or TCP over IPsec for all communications between edge nodes and the controller.
Therefore, it is necessary to ensure physical security controls are in place at remote locations,
including locks, surveillance, and access controls. Additionally, the RR needs to verify the BGP
advertisements from each SD-WAN edge to ensure in the SD-WAN Secure L3VPN that their
advertised VPN IDs (and Route Targets) are truly theirs. In addition, local BGP policy SHOULD
careful set-up access lists for the routes received and propagated. These verifications help prevent
unauthorized advertisement of prefixes and ensures the integrity of the routing information
within the SD-WAN environment.
This document does not specify a SD-WAN deployment outside of the above walled garden
described above. Such a deployment is outside the scope of this document.
IANA has assigned SAFI = 74 as the Hybrid (SD-WAN) SAFI.IANA is requested to assign a type from the BGP Tunnel Encapsulation Attribute Tunnel Types as follows [RFC8126]:IANA is requested to assign the following sub-Types in the BGP Tunnel Encapsulation Attribute Sub-TLVs registry:IANA is requested to assign a type from the BGP Tunnel Encapsulation Attribute Tunnel Types as follows [RFC8126]: Dynamic Networks to Hybrid Cloud DCs: Problem Statement and Mitigation PracticeL. Dunbar, A Malis, C. Jacquenet, M. Toy and K. Majumdar BGP Usage for SD-WAN Overlay NetworksL. Dunbar, A Sajassi, J Drake, and B. NajemSecure EVPNA Sajassi, A. Banerjee, S. Thoria, D. Carrell, B. Weis, J. Drake SD-WAN Service Attributes and Service FrameworkMEF SD-WAN Service Attributes and Service FrameworkMEF IANA-AH IANA IANA-ESP IANAAcknowledgements to Wang Haibo, Shunwan Zhuang, Hao Weiguo, and ShengCheng for implementation contribution.
Many thanks to Yoav Nir, Graham Bartlett, Jim Guichard, John Scudder, and Donald Eastlake for their
review and suggestions.ContributorsBelow is a list of other contributing authors:VerizonUSAgyan.s.mishra@verizon.com, HuaweiBeijingChinazhuangshunwan@huawei.com,HuaweiBeijingChinashengcheng@huawei.com, and FutureweiUSAd3e3e3@gmail.com.