Setting Flow Table Sizes on Juniper Routers

Let’s illustrate this problem on a Juniper MX Series Virtual 3D Universal Edge Router running Junos 18.2R1.9, which is configured for IPFIX messages export as follows:

chassis {
    fpc 0 {
        sampling-instance noction-nfa;
    }
}                                       
interfaces {
    ge-0/0/0 {
        description "Monitored interface";
        unit 0 {
            family inet {
                sampling {
                    input;
                    output;
                }
                address 192.168.1.1/24;
            }
        }
    }
}
forwarding-options {
    sampling {
        instance {
            noction-nfa {
                input {
                    rate 1024;
                }
                family inet {
                    output {
                        flow-server 192.168.2.100 {
                            port 2055;  
                            version-ipfix {
                                template {
                                    ipfix-templatev4;
                                }
                            }
                        }
                        inline-jflow {
                            source-address 192.168.2.1;
                        }
                    }
                }
            }
        }
    }
}
services {                              
    flow-monitoring {                   
        version-ipfix {                 
            template ipfix-templatev4 {
                flow-active-timeout 60;
                flow-inactive-timeout 60;
                template-refresh-rate {
                    seconds 60;         
                }                       
                ipv4-template;          
            }                           
        }                               
    }                                   
}               

Legend:

192.168.2.100 – IP address of NFA server
192.168.2.1 – Source IP address of flow packets (router IP address)

We can verify the size of flow tables using the following command:

root> show services accounting status inline-jflow fpc-slot 0

The default flow size for IPv4 flows is 1024 and 512 for the IPv6 flows (Figure 1).

Figure 1 – Checking Size of Flow Tables on Juniper vMX Router

Obviously, the sizes of both flow tables are really small, so there is a very high risk that we can routinely overflow tables. In a scenario where a large-scale distributed DoS attack (DDoS) is being carried out, this becomes a 100% certainty. A flow is created for each packet passing through the monitoring point, so the number of flows would increase dramatically. However, there is no space in the table; the flow is dropped, and the error count rises.

To carry out a DDoS attack against a victim and overflowing the flow tables left in their default configuration on the Juniper vMX router is as simple as running the hping command:

$ sudo hping3 -S --flood -V -p 22 T.T.T.T --rand-source

Now, let’s check the number of active flows with the command:

root> show services accounting flow inline-jflow fpc-slot 0

The number of active flows in the flow table is 1024, reaching the “maximum number of IPv4 flows” (Figure 2). A new flow will not be created until an existing flow sitting in the flow table is exported to the flow collector, freeing up the space in the cache.

Figure 2 – Active Flows in IPv4 Flow table During DoS Attack

When DDoS is stopped, the number of active flows can be reduced to 0 if there is no traffic on the network (Figure 3).

Figure 3 – Active Flows Count When DoS Attack is Finished

However, we have reached the max IPv4 flow count. As a result, there are 15164 IPv4 flow creation errors reported by the command:

root> show services accounting errors inline-jflow fpc-slot 0

Figure 4 – IPv4 Flow Creation Failures

The flows are not exported to the flow collector, and the amount of data displayed by the NFA does not match the actual data.

1. Manual IPv4 and IPv6 Flow Tables Sizing

We can manually configure the size of both IPv4 and IPv6 flow tables in units of 256k entries. Starting with Junos OS Release 16.1R1 and 15.1F2, by default, the software allocates 1K entries for IPv4 flow tables. To allocate eleven 256K IPv4 and four 256k Ipv6 flow tables, enter the command:

root# set chassis fpc 0 inline-services flow-table-size ipv4-flow-table-size 11
root# set chassis fpc 0 inline-services flow-table-size ipv6-flow-table-size 4

This provides 11*256K flows for IPv4 and 4*256K flows for IPv6 (Figure 5).

Figure 5 – IPv4 and IPv6 Flow Tables Sized Manually

Figure 6 – IPv4 and IPv6 Flow Tables Sizes After Manual Sizing

Before Junos OS Release 16.1R1 and 15.1F2, any changes in the configured size of the flow table initiate an automatic reboot of the FPC. Therefore, it is highly recommended that you run this command in a maintenance window.

2. Flex Flow Tables Sizing

The MX series supports a flex-flow-sizing option that does not require manual sizing between IPv4 and IPv6 tables. To test this feature, we have deleted the previous configuration and reset the counters:

root> clear services accounting statistics inline-jflow fpc-slot 0

Now, we can configure flexible size allocation of flow tables for all proto families.

root# set chassis fpc 0 inline-services flex-flow-sizing

Figure 7 confirms that the Flex Flow Sizing parameter is enabled, and the new flow tables sizes are set automatically for all protocol families.

Figure 7 – Enabled Flex Flow Sizing for All Proto Families

Conclusion

This tutorial discussed the intricacies of setting the flow table size for IPv4 and IPv6 on Juniper routers. Checking the flow table size settings on Juniper devices is most likely the first step one should take when noticing any traffic discrepancies issues in NFA.