BGP Optimal Route Reflection as an alternative to BGP Add Path

Full iBGP Mesh and Route Reflection

To avoid loops in a Full iBGP Mesh, BGP routers are only allowed to learn prefixes over iBGP from the router that advertises them or, alternatively, learn them via eBGP. In such cases, all BGP speakers within a single AS must be fully meshed so that any external routing information is re-distributed to all routers within that Autonomous System (AS). This becomes a serious scaling problem when there is a large number of IBGP speakers exchanging a large volume of routing information.

Route Reflection, described in RFC 4456, is a solution to the iBGP scaling problem described above. When the Route Reflector (RR) receives a route from an iBGP peer it selects the best-path for that route. If RR receives a route from a non-client iBGP peer, it reflects it to all of its clients. When RR receives a route from its client it reflects it to all client and non-client peers.

RR uses the same path selection process as normal iBGP speakers do. The path whose next hop is resolved through the iBGP route with the lowest metric is preferred. This ensures that traffic exits a network at the lowest cost via the shortest path – the so called “hot potato” routing, as traffic is directed to the best AS exit point.

Suboptimal Path Reflection

Suboptimal Path Reflection occurs when a RR is not topologically near to the clients. RR performs the best path selection based on its proximity in IGP but not the proximity of its clients. Let’s take a look at the following topology, with a router reflector (RR) in AS6500 and its clients R1, R2 R3 and R4 (Picture 1).

Picture 1: Network Topology

The RR learns the route 8.8.8.0/24 from ISP1 (AS6501) through R1 and from ISP2 through R2 (Picture 2).

RR# show ip bgp | begin Network

Picture 2: BGP Table of Router RR

The path to 8.8.8.0/24 through R1 (1.1.1.1) is preferred to the path via R2 (1.1.1.2) because the IGP metric 2 is lower than 4 (Picture 3). Therefore, the RR router installs the 8.8.8.0/24 route via R1 into its routing table. As the BGP advertises only the best path, the path via R2 is not being advertised to the clients.

RR# show ip bgp detail

Picture 3: Inspecting BGP Table of RR for Route 8.8.8.0/24

It is important to mention here that although a path to a certain prefix is optimal for the Route Reflector based on the lowest IGP metric criteria, this might not be true from the client’s perspective. For instance, the R3 router has installed a path to 8.8.8.0/24 via R1 received from RR (Picture 4). However, it is obvious that a path via R2 would be definitely shorter thus truly optimal if received.

RP/0/0/CPU0:R3# show bgp | begin Network

Picture 4: BGP Table of R3

The solution to this problem might be hierarchical RRs placed in close proximity to clients, so that a best path advertised by RR would represent the best path for clients as well. Another solution is to configure RR to advertise multiple paths (ADD_PATH) to clients for the same prefix so that they can select the best path based on its own perspective. However, pushing all external paths from RR to clients may may be a challenge as the client’s resources (CPU, RAM) can be depleted by a larger BGP table size.

BGP Optimal Route Reflection

BGP Optimal Route Reflection is defined in an I-E Draft. Instead of sending the best-path based on RR point of view, optimal BGP path selection is based on a client’s perspective. The RR sends a specific best path to a specific BGP border router that is calculated based on the position of the router in the topology. Therefore, the RR sends a different best path to different BGP border routers. For the BGP Optimal Route Reflection to work properly, the RR must have a complete view of the network topology from the IGP perspective. Hence, network must run a link-state routing protocol (OSPF or IS-IS). Also, the border routers must be RR clients. The objective is that each ingress BGP border router can have a different exit or egress BGP router for the same prefix. If the ingress border router can always forward the traffic to the closes AS-exit router, then a requirement of the hot-potato routing is fulfilled.

The RR runs a Shortest Path First (SPF) calculation with the client as the root of the tree and then calculates the cost to the BGP next-hop. The IGP cost for use of BGP best path selection is stored and the route with the lowest IGP metric is chosen and advertised to a particular client. The SPF calculation is independent of BGP and runs only if ORR is enabled on the RR.

As for Cisco family products, Optimal Route Reflection (ORR) has been supported only by IOS-XR, since version 5.3.1. The OSPF, IS-IS and BGP-LS can be used to acquire IGP topology information.

Now let’s discuss the configuration of the RR and R3 routers. RR is configured as a Route Reflector with enabled BGP-ORR. The R3 router is a client of the RR. Both routers are running IOS-XR, version 6.1.8.

Router RR Configuration

interface Loopback0 
 ipv4 address 1.1.1.5 255.255.255.255 

interface GigabitEthernet0/0/0/0 
 description Link to R1 
 ipv4 address 10.0.0.2 255.255.255.252 

interface GigabitEthernet0/0/0/1 
 description Link to R4 
 ipv4 address 10.0.0.5 255.255.255.252

Configure OSPF as an IGP used in AS6500. Redistribute OSPF information into BGP with distribute bgp-ls command.

router ospf 1 
 distribute bgp-ls 
 router-id 1.1.1.5 
 address-family ipv4 unicast 
 area 0 
  mpls traffic-eng 
  interface Loopback0 
   network point-to-point 

  interface GigabitEthernet0/0/0/0 
   network point-to-point 

  interface GigabitEthernet0/0/0/1 
   network point-to-point

Define a primary root device for each of ORR group that we want to use to compute the IGP cost. The secondary and tertiary root device can be configured as well. Now enable optimal-route-reflection for each neighbor.

router bgp 6500 
 bgp router-id 1.1.1.5 
 address-family ipv4 unicast 
  optimal-route-reflection r1 1.1.1.1 
  optimal-route-reflection r2 1.1.1.2 
  optimal-route-reflection r3 1.1.1.3 
  optimal-route-reflection r4 1.1.1.4 

 neighbor 1.1.1.1 
  remote-as 6500 
  update-source Loopback0 
  address-family ipv4 unicast 
   optimal-route-reflection r1 
   route-reflector-client 

 neighbor 1.1.1.2 
  remote-as 6500 
  update-source Loopback0 
  address-family ipv4 unicast 
   optimal-route-reflection r2 
   route-reflector-client 

 neighbor 1.1.1.3 
  remote-as 6500 
  update-source Loopback0 
  address-family ipv4 unicast 
   optimal-route-reflection r3 
   route-reflector-client 

 neighbor 1.1.1.4 
  remote-as 6500 
  update-source Loopback0 
  address-family ipv4 unicast 
   optimal-route-reflection r4 
   route-reflector-client

mpls traffic-eng

To check ORR SPF database for the group 3, issue the command below (Picture 5). The actual Root for SPF is 1.1.1.3. The OSPF cost to the iBGP next-hops is computed from the R3’s perspective. The cost 4 is computed for the neighbor 1.1.1.1 (R1) and the cost 2 for 1.1.1.2 (R2).

RP/0/0/CPU0:RR# show orrspf database r3

Picture 5: Router RR ORR SPF Database for Group 1

RP/0/0/CPU0:R3# show route ospf

Picture 6: R3 OSPF Routes

Router R3 Configuration

interface Loopback0 
 ipv4 address 1.1.1.3 255.255.255.255 

interface GigabitEthernet0/0/0/0 
 ipv4 address 10.0.0.10 255.255.255.252 

interface GigabitEthernet0/0/0/1 
 ipv4 address 10.0.0.13 255.255.255.252 

router ospf 1 
 distribute bgp-ls 
 router-id 1.1.1.3 
 address-family ipv4 unicast 
 area 0 
  mpls traffic-eng 
  interface Loopback0 
   network point-to-point 

  interface GigabitEthernet0/0/0/0 
   network point-to-point 

  interface GigabitEthernet0/0/0/1 
   network point-to-point 

router bgp 6500 
 bgp router-id 1.1.1.3 
 address-family ipv4 unicast 

 neighbor 1.1.1.5 
  remote-as 6500 
  update-source Loopback0 
  address-family ipv4 unicast 

mpls traffic-eng

Finally, we will inspect the BGP table of R3 to check whether a suboptimal path to the prefix 8.8.8.0/24 via the BGP next-hop 1.1.1.1 (R1) is changed to the the shorter path via 1.1.1.2 (R2).

RP/0/0/CPU0:R3# show bgp

Picture 7: R3 BGP Table

The router R3 has installed a path to the prefix 8.8.8.0/24 via router R2 (BGP next-hop 1.1.1.2).

Conclusion:

BGP Optimal Route Reflection is a new routing technology that still exists as an I-E draft. ORR should be tested in networks with 10k IGP nodes to check how it converges in such networks, as each next-hop change triggers SPF calculations. Instead of running ORR on hardware routers it should be run on a virtual Route Reflector with enough CPU and RAM. Nevertheless, the concept has been implemented by major network vendors such as Cisco, Juniper and Nokia already. It will be interesting to watch how it progresses.