To avoid loops in a Full iBGP Mesh, BGP routers are only allowed to learn prefixes over iBGP from the router that advertises them or, alternatively, learn them via eBGP. In such cases, all BGP speakers within a single AS must be fully meshed so that any external routing information is re-distributed to all routers within that Autonomous System (AS). This becomes a serious scaling problem when there is a large number of IBGP speakers exchanging a large volume of routing information.
Route Reflection, described in RFC 4456, is a solution to the iBGP scaling problem described above. When the Route Reflector (RR) receives a route from an iBGP peer it selects the best-path for that route. If RR receives a route from a non-client iBGP peer, it reflects it to all of its clients. When RR receives a route from its client it reflects it to all client and non-client peers.
RR uses the same path selection process as normal iBGP speakers do. The path whose next hop is resolved through the iBGP route with the lowest metric is preferred. This ensures that traffic exits a network at the lowest cost via the shortest path – the so called “hot potato” routing, as traffic is directed to the best AS exit point.
Suboptimal Path Reflection occurs when a RR is not topologically near to the clients. RR performs the best path selection based on its proximity in IGP but not the proximity of its clients. Let’s take a look at the following topology, with a router reflector (RR) in AS6500 and its clients R1, R2 R3 and R4 (Picture 1).
Picture 1: Network Topology
The RR learns the route 8.8.8.0/24 from ISP1 (AS6501) through R1 and from ISP2 through R2 (Picture 2).
RR# show ip bgp | begin Network
Picture 2: BGP Table of Router RR
The path to 8.8.8.0/24 through R1 (1.1.1.1) is preferred to the path via R2 (1.1.1.2) because the IGP metric 2 is lower than 4 (Picture 3). Therefore, the RR router installs the 8.8.8.0/24 route via R1 into its routing table. As the BGP advertises only the best path, the path via R2 is not being advertised to the clients.
RR# show ip bgp detail
Picture 3: Inspecting BGP Table of RR for Route 8.8.8.0/24
| Note: The RFC7911 describes a BGP extension that allows the advertisement of multiple paths (ADD-PATH) for the same prefix (NLRI). We have covered the BGP ADD-PATH topic in a separate article. |
It is important to mention here that although a path to a certain prefix is optimal for the Route Reflector based on the lowest IGP metric criteria, this might not be true from the client’s perspective. For instance, the R3 router has installed a path to 8.8.8.0/24 via R1 received from RR (Picture 4). However, it is obvious that a path via R2 would be definitely shorter thus truly optimal if received.
RP/0/0/CPU0:R3# show bgp | begin Network
Picture 4: BGP Table of R3
The solution to this problem might be hierarchical RRs placed in close proximity to clients, so that a best path advertised by RR would represent the best path for clients as well. Another solution is to configure RR to advertise multiple paths (ADD_PATH) to clients for the same prefix so that they can select the best path based on its own perspective. However, pushing all external paths from RR to clients may may be a challenge as the client’s resources (CPU, RAM) can be depleted by a larger BGP table size.
BGP Optimal Route Reflection is defined in an I-E Draft. Instead of sending the best-path based on RR point of view, optimal BGP path selection is based on a client’s perspective. The RR sends a specific best path to a specific BGP border router that is calculated based on the position of the router in the topology. Therefore, the RR sends a different best path to different BGP border routers. For the BGP Optimal Route Reflection to work properly, the RR must have a complete view of the network topology from the IGP perspective. Hence, network must run a link-state routing protocol (OSPF or IS-IS). Also, the border routers must be RR clients. The objective is that each ingress BGP border router can have a different exit or egress BGP router for the same prefix. If the ingress border router can always forward the traffic to the closes AS-exit router, then a requirement of the hot-potato routing is fulfilled.
The RR runs a Shortest Path First (SPF) calculation with the client as the root of the tree and then calculates the cost to the BGP next-hop. The IGP cost for use of BGP best path selection is stored and the route with the lowest IGP metric is chosen and advertised to a particular client. The SPF calculation is independent of BGP and runs only if ORR is enabled on the RR.
As for Cisco family products, Optimal Route Reflection (ORR) has been supported only by IOS-XR, since version 5.3.1. The OSPF, IS-IS and BGP-LS can be used to acquire IGP topology information.
Now let’s discuss the configuration of the RR and R3 routers. RR is configured as a Route Reflector with enabled BGP-ORR. The R3 router is a client of the RR. Both routers are running IOS-XR, version 6.1.8.
interface Loopback0 ipv4 address 1.1.1.5 255.255.255.255 interface GigabitEthernet0/0/0/0 description Link to R1 ipv4 address 10.0.0.2 255.255.255.252 interface GigabitEthernet0/0/0/1 description Link to R4 ipv4 address 10.0.0.5 255.255.255.252
Configure OSPF as an IGP used in AS6500. Redistribute OSPF information into BGP with distribute bgp-ls command.
router ospf 1 distribute bgp-ls router-id 1.1.1.5 address-family ipv4 unicast area 0 mpls traffic-eng interface Loopback0 network point-to-point interface GigabitEthernet0/0/0/0 network point-to-point interface GigabitEthernet0/0/0/1 network point-to-point
Define a primary root device for each of ORR group that we want to use to compute the IGP cost. The secondary and tertiary root device can be configured as well. Now enable optimal-route-reflection for each neighbor.
router bgp 6500 bgp router-id 1.1.1.5 address-family ipv4 unicast optimal-route-reflection r1 1.1.1.1 optimal-route-reflection r2 1.1.1.2 optimal-route-reflection r3 1.1.1.3 optimal-route-reflection r4 1.1.1.4 neighbor 1.1.1.1 remote-as 6500 update-source Loopback0 address-family ipv4 unicast optimal-route-reflection r1 route-reflector-client neighbor 1.1.1.2 remote-as 6500 update-source Loopback0 address-family ipv4 unicast optimal-route-reflection r2 route-reflector-client neighbor 1.1.1.3 remote-as 6500 update-source Loopback0 address-family ipv4 unicast optimal-route-reflection r3 route-reflector-client neighbor 1.1.1.4 remote-as 6500 update-source Loopback0 address-family ipv4 unicast optimal-route-reflection r4 route-reflector-client mpls traffic-eng
To check ORR SPF database for the group 3, issue the command below (Picture 5). The actual Root for SPF is 1.1.1.3. The OSPF cost to the iBGP next-hops is computed from the R3’s perspective. The cost 4 is computed for the neighbor 1.1.1.1 (R1) and the cost 2 for 1.1.1.2 (R2).
RP/0/0/CPU0:RR# show orrspf database r3
Picture 5: Router RR ORR SPF Database for Group 1
RP/0/0/CPU0:R3# show route ospf
Picture 6: R3 OSPF Routes
interface Loopback0 ipv4 address 1.1.1.3 255.255.255.255 interface GigabitEthernet0/0/0/0 ipv4 address 10.0.0.10 255.255.255.252 interface GigabitEthernet0/0/0/1 ipv4 address 10.0.0.13 255.255.255.252 router ospf 1 distribute bgp-ls router-id 1.1.1.3 address-family ipv4 unicast area 0 mpls traffic-eng interface Loopback0 network point-to-point interface GigabitEthernet0/0/0/0 network point-to-point interface GigabitEthernet0/0/0/1 network point-to-point router bgp 6500 bgp router-id 1.1.1.3 address-family ipv4 unicast neighbor 1.1.1.5 remote-as 6500 update-source Loopback0 address-family ipv4 unicast mpls traffic-eng
Finally, we will inspect the BGP table of R3 to check whether a suboptimal path to the prefix 8.8.8.0/24 via the BGP next-hop 1.1.1.1 (R1) is changed to the the shorter path via 1.1.1.2 (R2).
RP/0/0/CPU0:R3# show bgp
Picture 7: R3 BGP Table
The router R3 has installed a path to the prefix 8.8.8.0/24 via router R2 (BGP next-hop 1.1.1.2).
BGP Optimal Route Reflection is a new routing technology that still exists as an I-E draft. ORR should be tested in networks with 10k IGP nodes to check how it converges in such networks, as each next-hop change triggers SPF calculations. Instead of running ORR on hardware routers it should be run on a virtual Route Reflector with enough CPU and RAM. Nevertheless, the concept has been implemented by major network vendors such as Cisco, Juniper and Nokia already. It will be interesting to watch how it progresses.