Cisco Express Forwarding (CEF) is a Cisco proprietary technology that allows their platform to move the forwarding decision from CPU down to hardware Application-Specific Integrated Circuits (ASICs). The process of programming these ASICs requires two tables of information: the Forwarding Information Base and the Adjacency table.
The RIB starts construction in the control plane with the IOS software collecting all of the routes from the locally configured static, connected, and dynamic routing protocol (EIGRP, OSPF, BGP, ISIS, RIP) routes. All of this information is collected and stored within the Routing Information Base (RIB) for each routing process on the device. An example of a RIB would be the OSPF database. The control plane does not eliminate routes at this point. The RIB will contain all of the routes that it learns, regardless of whether they are the best route to the destination or not.
An LFIB is the forwarding information base for MPLS labels. This is not utilized often in environments, but it is still an information source utilized by CEF to create its final table for making its decisions.
The router now goes through the selection process from each of the associated route types and their administrative distances to select which of all of the routes should go into the Forwarding Information Base (FIB). Each time the RIBs are updated and a change needs to occur to the FIB, this update will occur.
At the same time each of the routing processes are compiling their information and are performing calculation on which routes are the best out of their RIB, the device is also building an adjacency table with layer 2 address to interface mapping. ARP would be the most common example of this, with a MAC address being correlated with a specific physical interface.
With the two tables formed with information, CEF will now go through each forwarding entry and tie it to an adjacency entry. This is then installed in the actual CEF table which is either placed in the lower level code for virtualized routers, or places it within actual ASICs in certain hardware platforms.
Some platforms are capable of performing Distributed CEF. This would be useful in the case of the Catalyst 6500 series multicard chassis design. Instead of sending the packets to the CEF ASICs in a centralized location on the supervisor cards, it can make these decisions locally on each line card, thus providing a linear scaling capability for each line card installed in the chassis.
CEF supports more than just Ethernet. It can support High-Level Data Link Control, Tunnels, and PPP just to name a few that are more common.
In the event that Equal Cost Multi Pathing (ECMP) is enabled and multiple routes to the same prefix are available, CEF has a few options for load balancing. CEF provides two options for load balancing: per-destination and per-packet. Each has its own specific strength and weakness which I’ll discuss here.
Per-destination load balancing is the default behavior for CEF. I know that it states as “per-destination,” but what it is really looking at is the source and destination pairing. The reason why this is the default behavior is to allow for all of the packets to arrive in sequence, as some applications have a hard time recovering from out of order packets. This is typically the most common configuration scenario. There are a few reasons you would want to change this. One scenario is where you have a high bandwidth application between two hosts. These hosts will tend to cause a situation we call CEF polarization. You will see symptoms of this when you look at the route options and seeing one path being more highly utilized than the other options. The reason why you see this is due to the fact that the CEF load balancing algorithm only looks at the source and destination IPs, which if that does not change much for your traffic flow, you will have unequal utilization of paths.
A way to correct this is to include layer 4 information in the routing decision. This might allow the polarization to be less noticeable if the traffic has multiple sessions to between the same two hosts which would utilize different source ports at the least, and sometimes it might even have different destination ports depending on the application.
If this does not alleviate your issue, the other option is to implement per-packet load balancing. And this is just as it sounds. Each packet will require CEF to select a different route by the use of a round-robin selection method. This ensures that you have a more equalized utilization of the routing paths, but it still will not be perfectly equal due to the variability in packet sizes. But as I stated earlier, this creates a scenario of packets arriving at the destination out of order and if the application does not like it, this can cause you problems.
References
-
RIB, FIB, LFIB, Adjacency table
-
Load balancing Hash
-
​Polarization concept and avoidance
