NetFlow Generators explained

What is a NetFlow Generator?

A NetFlow generator is a dedicated network appliance or software running on a PC and listening on a single or multiple network interfaces (NICs). NetFlow generators create NetFlow records aggregating packets into flows based on the criteria such as IP source and destination addresses, L4 source and destination ports, IP protocol, a source interface and a class of service. Generators export records to a centralized place in a network, called a NetFlow collector. Network operators accessing NetFlow records stored permanently on the collector use the analysis applications for the purpose of creating reports and presenting data.

The generator typically operates as a dedicated NetFlow exporter, sending flows to a collector. However, it may also play a role of a NetFlow collector and analyzer, all in a single appliance since it possesses all the necessary capabilities for this.

How are the NetFlow Generators Implemented?

A NetFlow generator can be positioned either in an in-line or a mirroring mode. In the in-line mode, the generator is inserted between points A and B of the monitored link, passing all traffic between the points. Flow records are sent via a management interface to a NetFlow collector and the interface is also used for the management of the generator. In the mirroring mode, a NetFlow generator is connected to the Switched Port Analyzer (SPAN) port of a network device (router). The SPAN port configuration on the router ensures that a copy of network traffic from all source SPAN ports is mirrored to the destination SPAN port, where the NetFlow generator is connected. The network interface of the NetFlow generator receiving mirrored packets from the router is set to promiscuous mode in order to avoid accidental discarding of the received packets. The management interface is also needed to access the generator and for export of the flow records.

How Are the NetFlow Records Exported?

NetFlow generator captures raw data and generates NetFlow records based on them. However, the flows may be generated also from the pcap file. The generators export records as NetFlow packets via UDP to a single or multiple NetFlow collectors.

Why Do We Need NetFlow Generators?

Typical network devices have limited computing power and resources. Sure, many of them support NetFlow but monitoring is not certainly their primary task. Network devices use application-specific integrated circuits (ASICs) chipsets to do what they are supposed to do – packet forwarding. Thanks to using ASCIs or merchant chips, network devices meet the specific performance requirements. However, when network devices are involved in aggregating packets to flows, their CPU and RAM are involved, as well. A certain amount of RAM is needed to hold the flow cache and its consumption increases with the higher traffic volumes. CPU is required to walk the flow cache and flush expired flows. It might have a negative impact on devices’ performance, specially on high-speed links when traffic volume is high. Certainly, there are methods such as packet sampling that reduce the CPU load, memory consumption and the amount of exported NetFlow packets. For instance, in a random sampling mode, instead of capturing every single packet, a NetFlow exporter randomly selects M packets out of every N packets for sampling, and only those packets can create flows. CPU is not involved in the process of packet sampling as it is done by the hardware (ASICs). However, sampling exchanges monitoring accuracy for performance. We have written a separate article that discusses this problem. NetFlow generators usually do not suffer from these limitations as they are equipped with sufficient RAM and powerful CPU so sampling is not a must. Moreover, their primary task is flow creating and export thus their resources are not consumed by the tasks such as routing, firewalling, QoS, VPNs etc.

Although, NetFlow has been with us for many years and it is widely implemented by vendors, not all network devices are NetFlow compatible. Some white box solutions such as openvswitch support NetFlow but others may completely miss the feature, even the older version such as the traditional NetFlow v5.

What Are Pros and Cons of Netflow Generators?

An embedded device used for creating flows benefits from a more powerful CPU and the larger RAM used for flows caching. The large memory represent a big advantage of a NetFlow generator when compared to traditional network devices with limited RAM size. It allows a Netflow generator to cache all flows even under a heavy network load so no flows are dropped. This increases the accuracy of the exported NetFlow records that is so needed for billing and network forensic.

The cost of a NetFlow generator may represent a downside of this solution. It is in high contrast with the built-in and usually cost-free NetFlow features implemented in routers or switches.

Conclusion:

If one decides to use a NetFlow generator, he/she should pay attention to the following: NetFlow generators must be able to capture network packets at wire-speed and to handle gigabit speeds. They must be capable to monitor multiple network interfaces at the same observation point. At least, they must support NetFlow v9. Ideally, they should be compatible with Internet Protocol Flow Information Export (IPFIX) as it is the standard for exporting the information about network flows from devices. When installed as software daemons, they must support the common operating systems such as Linux or Windows and different CPU architectures (x86, ARM). Regardless, whether they are hardware or software based, NetFlow generators should be capable to export records to multiple flow collectors.