Maintain consistency of inter-domain routing and forwarding

The inconsistency of inter-domain routing and forwarding The path from the origin AS to the destination AS is determined by BGP protocol and non-BGP factors. As a result, the actual forwarding AS path of the packet is usually different with the expected AS path which is the same as the AS_PATH attribute in BGP UPDATE. In other words, inter-AS routing and forwarding are inconsistent. The inconsistency means that inter-domain routing is a black box. The origin AS, maybe any AS, does not see the actual forwarding AS path from the route, which cause these paths may not comply with some local defined rules, e.g., including an AS that it does not prefer, violating the valley-free principle.

The complete forwarding AS path As shown in the figure, the comlpete forwarding AS path from AS1 to AS2 consists fo BGP paths and non-BGP paths.

Reasons for the inconsistency Inter-domain routing and forwarding inconsistency may be caused by traffic redirection, traffic engineering protocols, and route aggregation, etc.

Inter-domain traffic Redirection Due to load-balancing, anti-DDoS, etc., policy-based routing (PBR) or BGP Flowspec may be configured to redirect traffic to a new next-hop AS which is different with the next-hop AS determined by AS_PATH in BGP UPDATE. The subsequent forwarding AS path is determined by the next-hop AS, which is unknown to the origin AS and the upstream AS. The complete forwarding AS path consists of two parts: the AS path before redirection and the AS path after redirection. This path is not verified by BGP, which may violate the valley-free principle or even contain duplicate ASes, causing traffic blackholes or loops.

Traffic engineering protocols Due to load-balancing, network optimization, etc., operators may utilize other protocols such as MPLS，SR to steer traffic to the specified AS path which is different from the AS_PATH in BGP UPDATE. The complete forwarding AS path is determined by BGP and TE protocols. It may include multiple segments, for example, the first segment is an AS path determined by BGP that starts with the origin AS, the second segment is a TE path and the last segment is still the AS path determined by BGP that ends with the destination AS. The complete forwarding path is actually transparent to the origin AS and is not verified by its filters, which may incur similar risks as redirection.

Route Aggregation To minimize routing tables, route aggregation is widely used in IP networks. BGP route aggregation causes the ordered AS_Sequence, to be converted to the unordered AS_SET, which are different types of AS_PATH. However, AS_SET does not represent the AS forwarding path of the data packet, which can also lead to the inconsistency between inter-domain routing and forwarding.

Other factors Other factors, such as route convergence and route redistribution, may also contribute to the inconsistency between routing and forwarding. During route convergence, non-simultaneous addition and deletion of routes by multiple ASes may also cause a short-term inconsistency between routing and forwarding. However, as routing completes convergence, eventually routing and forwarding will be consistent. This short-term inconsistency is caused by the flaws in the routing protocol's own design and is beyond the scope of this draft. Redistributing BGP routes into IGP can result in the loss of AS_PATH. Before advertised to the other AS, the route should be redistributed from IGP into BGP. The other AS that receive the route by BGP considers the destination AS of the route to be the redistribution AS. The complete AS path that the origin AS wisher to obtain is actually the path from it to the redistribution AS. The forwarding AS path before redistribution are outside the scope of this draft for now. This draft focuses on the factors that contribute to the chronic inconsistency of inter-domain routing and forwarding: inter-domain redirection, TE protocols and route aggregation. There factors are inevitable obstacles to the consistency.

The risk of the inconsistency The inconsistency between inter-domain routing and forwarding result in the origin AS, possibly all ASes, not seeing the complete actual AS forwarding path and not knowing whether the forwarding is secure. Specifically, the inconsistency between the expected AS path and the actual AS path leads to the following risks:

Traffic blackhole: there is no corresponding route for the next-hop AS, resulting in a traffic black hole. For example, an AS redirects traffic that should be forwarded to the provider AS to another unrelated customer AS, which also violates the valley-free principle.
Loop: the forwarding AS path that has not been checked by BGP may lead to loops, e.g., the AS path before redirection and the AS path after redirection may contain the same AS, which is a risk that cannot be circumvented by the current BGP protocol itself.
Detour: the complete forwarding AS path is composed of multiple AS paths from different protocol, which may lead to unnecessary lengthening of AS_PATH.
Malicious AS: The actual forwarding path is not visible to the origin AS, which may cause its packets to pass through some ASes it does not expect.
Non-optimal route: The AS may prefer some ASes that may not be included in the actual pata to select a non-optimal route.

Consistent inter-domain routing and forwarding During packet transmission from the origin AS to the destination AS, partial AS path may be controlled by redirection or non-BGP protocols, such MPLS, SR, etc. Therefore, to get the complete forwarding AS path by BGP, there are two steps:

Obtaining deviation AS paths that results from local non-BGP factors.
Advertising the deviation AS path and the steered flow by BGP

Obtaining deviation AS paths

Obtaining redirection deviation AS paths A router redirecting traffic to a new next-hop AS generates a deviation AS path. In order to get the direction deviation AS path, the router first acquires the next-hop AS and the destination prefix of the redirection rule. The router then looks up the AS_PATH in Adj_RIBs_In from the next-hop AS by the destination prefix. The AS_PATH is the forwarding AS path after redirection, i.e., the redirection deviation AS path. Redirection rules usually specify only the next-hop address or the outgoing interface, so it is also necessary to obtain the AS to which the interface is connected. In the prior art, one way is that there will be relevant configuration in the BGP peer that specifies the AS to which the remote peer belongs and the outgoing interface used for the connection. This interface is connected to the AS to which the peer belongs (except for the loopback interface). Additional configuration to specify the AS to which the interface is connected is also a viable approach. All BGP routes sent by remote peers are stored in Adj-RIBs-in. The feasible redirection AS path should be among these unpreferred BGP routes. The new next-hop AS that was redirected to has no routes to the destination prefix of the redirection rule, which may result in the traffic black hole. If the redirection rule does not have the destination prefix field, the traffic routed to any destination prefix may be redirected. Thus all AS_PATHs in Adj-RIBs-In are deviation AS paths resulting from the redirection rule. Typically such redirection rules that do not include the destination prefix field and whose next-hop AS lacks reachability are not recommended.

Obtaining deviation AS path from other protocols Some TE protocols that may direct the specific flow into pre-planned AS paths. Their destination prefix of the steered traffic is obtained in a similar way as in . In the egress router of the TE path, the Adj-RIBs-In is then looked up by the destination prefix to obtain the subsequent AS path. The complete forwarding path, therefore, is stitched together from the TE Path and other AS Paths controlled by BGP.

Advertising the deviation path

Advertising the deviation path As shown in the figure, AS4 advertises a BGP UPDATE to ASx along the AS path ([AS1,AS2,AS4]). AS2 is a redirection AS which redirect the packet routing to AS4 to AS3. As a result, the actual forwarding path of packets from AS1 to AS2 is [AS1, AS2, AS3, AS4]. So AS2 should add D_PATH to BGP UPDATE. D_PATH refers to the actual forwarding AS path through which the packet sent to AS4. The AS that generates the deviation AS path is obliged to advertise it to other ASes. For example, the AS that is configured with the redirection rule advertises the redirection deviation path to the origin AS. The origin AS then combines the AS path to the redirection AS and the redirection deviation AS path to get the complete forwarding AS path which is verified to be optimal or secure using the local AS_PATH filter. The path that passes through malicious ASes will not be selected by the origin AS. However, only the flow that matches the redirection rule will go through the deviation path, and the other missed flow is still forwarded along the original path planned by BGP. Therefor, the redirection AS should jointly advertise the deviation path and the specific flow to other ASes. In order to maintain the consistency of inter-domain routing and forwarding, the deviation AS path should be propagated to other ASes in the form of route advertisement. In other words, the deviation AS path and attributes of the specific flow should be included in BGP UPDATE, so that utilizing the BGP protocol, other ASes can see that some traffic will go through the AS path controlled by non-BGP factors. The other ASes can perform TE and QoS according to the real forwarding AS path of the flow, which is helpful to reduce the risk of unreachability and the potential damage incurred by malicious ASes.

Benefits Since the birth of the Internet, inter-AS TE has been a very complex and difficult task. One of the major reasons is that inter-domain routing and forwarding are inconsistent. The AS_PATH attrtibute in BGP UPDATE only represents the propagation path of the route, which may not correspond to the actual forwarding path. The community has designed many complex protocols to plan the forwarding path and guarantee SLA, while routing protocols are usually utilized to get the basic reachability. If inter-domain routing and forwarding keep consistent, TE and routing security will be significantly simplified. Moreover, the ability of the origin AS to plan the forwarding AS path of its own path opens the black box of the Internet. Problems that have plagued the Internet for decades, such as visualization and troubleshooting, may be solved.

Management Considerations TBD

IANA Considerations TBD.

Security Considerations This document does not introduce any new security considerations.

Acknowledgements TBD