Usage-based billing with NetFlow and sFlow

Usage-based billing enables customers to pay only for data that they have used.  In order to charge customers based on data usage, a company must have a good metering system in place. The system must be able to provide accurate information on the volume of traffic transmitted and received by each customer. The metering system should be able to convert data usage to readable billing information which is sent to a billing system for invoice creation.

There are several protocols available that the metering system can use to accurately measure data usage. Examples are SNMP, Netflow or sFlow. Each protocol has its advantages and disadvantages which the provider must carefully consider to be able to perform accurate billing. For example, traditional SNMP may be more convenient to measure data consumption, but it lacks information about the source and destination of the transmission. SNMP only reports the total amount of traffic. While this is a perfect choice for billing when the entire port is assigned to a customer, it cannot be used for accurate billing if customers share the same port.

Itemized Billing with sFlow/NetFlow

Unlike SNMP, NetFlow and sFlow collect information about the source and destination involved with the traffic, application, volume of traffic, etc. Because customers may require a more granular view of traffic usage, the metering system should support NetFlow/sFlow to provide itemized billing reports.  As a result, customers are confident that the reports reflect the exact use of data. In addition, they can better control costs in the future as reports contain detailed breakdown of data usage, e.g based on VLAN, priority, type of service, etc. In addition to sFlow, which provides full layer 2 – 7 traffic visibility, we can also use NetFlow to report the various traffic attributes required for itemized billing. More specifically, Flexible NetFlow v9 allows to match or collect the following attributes:

1. IP protocol
2. Type of service
3. IPv4/IPv6 source and destination addresses
4. TCP/UDP source and destination ports
5. BGP origin/destination and peer ASs
6. MPLS TOP
7. Layer 2 information such as  source/destination MAC addresses, Vlan IDs
8. Timestamps
9. Flow direction
10. Replication factor and number of replicated packets/bytes for multicast traffic

So, should one use NetFlow or sFlow for usage-based billing? Before answering this question, let’s summarize key facts about sFlow and NetFlow. 

The Accuracy of sFlow Measurement

Sampled flow (sFlow) is a packet sampling protocol that captures packet headers and partial packet payload (the first 64 or 128 Bytes) into sFlow datagrams. A device with an enabled sFlow has no cache to temporarily store datagrams. Immediately after sampling, samples are forwarded to sFlow collectors. sFlow datagrams can carry multiple samples. sFlow provides full layer 2 – 7 traffic visibility since the entire packet headers are captured. This allows reporting MAC, VLAN, and application information along with L3 – L4 attributes, typically reported by NetFlow. 

Unlike NetFlow, sFlow does not support the unsampled mode. Individual packets are randomly sampled based on the defined sampling rate; an average 1 out of N packets is randomly sampled.  This type of sampling does not provide a 100% accurate result, but it does provide a result with quantifiable accuracy [1]. The accuracy of sFlow measurement is therefore dependent on the number of samples. If the variable N is set too high, sFlow records generated from the samples may be less representative of the actual traffic. Moreover, some low volume flows may not be counted at all, especially if the traffic time window is short. Increasing a sampling rate thus lowering the variable N seems to be a solution to this problem. However, while increasing the sampling rate improves the accuracy of sFlow measurement, more sFlow transmissions are generated that are sent over the network to the sFlow collector. In the worst case, the collector may be completely overwhelmed by the traffic. The sampling rate is therefore always a compromise between scalability and accuracy and should reflect the interface speed and the traffic volume. For example, the proposed sampling rate that works for most networks is 1 in 1000 for the 1000Mb/s link and 1 in 2000 for the 10Gb/s link [2]. 

The graph in Figure 1 is useful for determining the number of samples that should be collected to obtain a given % error [3]. This can then be used to determine the sampling rate. The relative sampling error is less than 2.5% when the number of samples in class is 10000. The accuracy of sFlow results may be further improved with monitoring over a longer period of time.

Picture 1: Dependence Between Relative Sampling Error and Number of Samples
source: https://inmon.com/pdf/sFlowBilling.pdf

---

Note: If traffic volume is unusually high, increasing the N value helps reduce the number of sFlow datagrams sent to the collector.

The Accuracy of NetFlow Measurement

Instead of directly forwarding packet samples to the collector, NetFlow tracks packets statefully and aggregates them into flows. The first unique packet received by a NetFlow enabled device creates a flow in the device cache. Other packets sharing all of the same attributes (source/destination IP address, source/destination layer 4 port, class of Service, IP Protocol) are aggregated into the same flow. The packet and byte counters of flow are updated. If any of the traffic parameters do not match, a new flow record is created in the cache. The same applies to the other packets – either a new flow is created or the counters of the flow stored in the cache increase. When the timer for a particular flow expires, the device exports the cached flow as a NetFlow datagram to collectors. 

NetFlow allows tracking each packet received by a device. The accuracy of the NetFlow metering system is therefore 100%. Sampling can be employed to reduce CPU usage and the volume of flow records exported to collectors. However, if the device resources permit, feel free to avoid sampling to achieve 100% accuracy. Flow volume can be cut down by changing the flow aggregation method and we still get 100% accountability.

---

Conclusion:

NetFlow is a stateful flow tracking technology that provides 100% accuracy of measurement of IP traffic. Although, Flexible NetFlow v9 can report some layer 7 attributes such as BGP origin and destination AS, peer AS, full L7 traffic visibility is not supported. Traffic spikes or the unusually high volume traffic may lead to higher CPU utilization of a device with the enabled NetFlow. Sampling can be used to reduce CPU usage and to decrease the volume of exported NetFlow datagrams. In this case, however, the 100% measurement accuracy of data usage is not guaranteed. 

SFlow is a packet sampling technology that provides full layer 2 – 7 traffic visibility. It is not limited to IP only, instead, it supports all protocols of the OSI model.  sFlow may be tuned for better measurement accuracy by increasing the number of samples or monitoring over a longer time period. However, 100% measurement accuracy cannot be achieved because sampling is always employed.

Both sFlow and NetFlow are technologies used for collecting usage data. They are inexpensive and supported by multiple network devices. NetFlow, however, is preferable to sFlow, since it is able to provide 100% data measurement accuracy.