Network Flow Monitoring Explained: NetFlow vs sFlow vs IPFIX

Modern networking equipment is capable of processing billions of packets every second, but most of that work happens behind the scenes. Network Flow Monitoring, also known as packet sampling, aims to give network engineers some visibility into the vast amounts of traffic that cross our wired and wireless connections every day. In this blog post, we’ll explore what network flow monitoring is and how it’s used to keep traffic flowing smoothly and securely. We’ll also look at three of the most popular flow monitoring technologies.

What is Network Flow Monitoring?

Network Flow Monitoring is the collection, analysis, and monitoring of traffic traversing a given network or network segment. The objectives may vary from troubleshooting connectivity issues to planning future bandwidth allocation. Flow monitoring and packet sampling can even be useful in identifying and remediating security issues. 

Flow monitoring gives networking teams a good idea of how a network is operating, providing insights into overall utilization, application usage, potential bottlenecks, anomalies that may signal security threats, and more. There are several different standards and formats used in network flow monitoring, including NetFlow, sFlow, and Internet Protocol Flow Information Export (IPFIX). Each works in a slightly different way, but all are distinct from port mirroring and deep packet inspection in that they do not capture the contents of every packet passing over a port or through a switch. However, flow monitoring does provide more information than SNMP, which is generally limited to broad statistics like overall packet and bandwidth use.

How to Use Flow Monitoring and Packet Sampling

Flow Monitoring and Packet Sampling can provide a number of benefits but do require careful planning and setup to be effective. Here are a couple of general tips for getting the most out of a flow monitoring solution:

Clearly Define Your Use Case

Different flow monitoring technologies – and even different versions of the same technology – have differences in both their capabilities and operation. NetFlow version 5, for example, doesn’t support IPv4 or VLANs, making it of limited use in a highly segmented IPv6 network. You should have clear requirements and expectations documented before you even begin to set up flow monitoring on your network. Start with general questions like ‘Where do we need visibility?’, ‘What specific types of information are we interested in?’, and ‘Do we need complete flows or is sampled data sufficient?’. More technical questions like ‘Do we need visibility into both ingress and egress traffic?’ and ‘Which packet headers do we need to capture?’ will help you narrow down your requirements.

Cost is also a consideration, as is the potential performance of enabling flow monitoring on your network hardware. Nearly all modern networking equipment supports the capture and export of traffic flows, but collecting, analyzing, and monitoring those flows generally takes place in a third-party tool. These tools range widely in cost, with both open source and commercial options available. 

Know Your Hardware

Although most enterprise-grade networking equipment supports some form of flow monitoring, the exact technology and capabilities vary, as does the configuration procedure. With your use case in mind, you’ll want to evaluate what your hardware supports.  In cases where existing routers/switches/firewalls don’t support the features you need, standalone probes can also be used to capture flow monitoring data.

Choose What to Monitor

Before hopping onto the CLI of your closest router to turn the feature on, you’ll need to decide exactly what type of network traffic flows you’re interested in. This is usually defined in terms of a “tuple”. The most common version of NetFlow, for example, uses a “5-tuple” consisting of source and destination address, source and destination port, and the protocol field. Other flow monitoring technologies may use a “7-tuple” or even a “9-tuple”. Regardless, you’ll need to do a little homework beforehand to ensure you’re capturing the right traffic.

Configure Your Flow Exporter

Configuration of a flow monitoring technology is unfortunately not a uniform process, even between different products from the same vendor. You’ll want to consult the documentation for not only your specific product but also the software version in use in your network. 

Configure Your Flow Collector and Analyzer(s)

Exporting flow records is just the beginning. To get real value out of flow monitoring, you’ll want to store your flow records in a collector and use an analyzer to make real sense of the data. A Flow Analyzer can help spot both short and long-term trends, visualize important data, and reveal new insights that you might have missed.

Benefits of Flow Monitoring

Network Flow Monitoring can provide a number of benefits to organizations that implement it. Understanding how, when, and by whom a network is used can lead to greater resiliency, a lower cost of operation, and a faster time to resolution when things do go wrong. Here are just a few of the benefits that network flow monitoring can provide:

More Efficient Use of Resources

Imagine a highly congested network link or WAN interface. If relying on older monitoring techniques like SNMP, it may appear that additional bandwidth must be purchased. Flow Monitoring could give the organization deeper visibility into what and who is consuming all of the existing bandwidth. Perhaps an employee has set up an unauthorized file server, or a database is misconfigured. Remediating these types of issues would remove the requirement for more bandwidth, thus saving the organization money. 

When it does come time to upgrade network capacity, flow monitoring can provide a highly accurate basis from which to plan. Looking at historical data can yield a measure of how quickly traffic is increasing, giving business decision-makers an idea of how soon upgrades may be required.

Detecting Anomalous Traffic and other Network Security Threats

The ability to detect anomalous network traffic can be a major incentive to implementing a flow monitoring solution. Monitoring traffic within a network, as opposed to at the boundary or perimeter, can help identify threats that have slipped past other types of defenses. Large “east-west” traffic spikes between machines in the network could be indicative of a worm or virus propagating. A spike in outbound traffic could signal data exfiltration or command and control messages. A large amount of incoming traffic might be the first wave of a DDoS attack.

Flow Monitoring is a great complement to signature-based security technologies like Intrusion Detection Systems and Antivirus. New threats might not match any existing signatures, but with flow monitoring, a deviation from standard network behavior can still raise alarms. These types of solutions can also be used to narrow down the focus of a threat hunting campaign by providing security analysts a good starting point to do a deeper packet capture with a tool like Wireshark. 

Verifying Application Performance and Quality of Service (QoS)

Since flow monitoring can be enabled for a particular protocol, it can be immensely useful in verifying or troubleshooting the performance of business-critical applications like VoIP or videoconferencing. Network administrators can see the path(s) these types of important packets take through the network, and can also ensure the correct Class of Service is being applied to the traffic. 

Flow monitoring is also useful in measuring the impact of a new application, network configuration, or change in the number of users accessing the network. 

Top 3 Network Flow Monitoring Tools

There are a variety of flow monitoring solutions available on the market today, but the top three are Netflow, sFlow, and IPFIX. Here’s a brief look at each of these top options:

What is NetFlow?

NetFlow is the original flow monitoring solution, originally developed by Cisco in the late 1990s. Several different versions exist, but most deployments are based on either NetFlow v5 or NetFlow v9. While each version has different capabilities, the basic operation remains the same: 

First, a router, switch, firewall, or another type of device will capture information on the network “flows” – basically a set of packets that share a common set of characteristics like source and destination address, source, and destination port, and protocol type. After a flow has gone dormant or a predefined amount of time has passed, the device will export the flow records to an entity known as a “flow collector”. 

Finally, a “flow analyzer” makes sense of those records, providing insights in the form of visualizations, statistics, and detailed historical and real-time reporting. In practice, collectors and analyzers are often a single entity, often combined into a larger network performance monitoring solution.

NetFlow operates on a stateful basis. When a client machine reaches out to a server, NetFlow will begin capturing and aggregating metadata from the flow. After the session is terminated, NetFlow will export a single complete record to the collector. 

Though it’s still commonly used, NetFlow v5 has a number of limitations. The fields exported are fixed, monitoring is supported only in the ingress direction, and modern technologies like IPv6, MPLS, and VXLAN aren’t supported. NetFlow v9, also branded as Flexible NetFlow (FNF), addresses some of these limitations, allowing users to build custom templates and adding support for newer technologies. 

Many vendors also have their own proprietary implementations of NetFlow, such as jFlow from Juniper and NetStream from Huawei. Though the configuration may differ somewhat, these implementations often produce flow records that are compatible with NetFlow collectors and analyzers.

What is sFlow?

Short for sampled flow, sFlow takes a slightly different approach to network monitoring than other options. Where NetFlow statefully tracks flows, sFlow works by randomly sampling the full packet headers of a given flow at a predetermined interval. This can cut down on bandwidth and CPU utilization on the switch or router capturing flow information, but may also reduce the accuracy of the information collected. sFlow does, however, capture deeper levels of information than NetFlow, including full packet headers and even partial packet payloads.

sFlow is supported on a wide range of networking equipment, even on many products from Cisco. On the collector/analyzer side, sFlow exports records that are incompatible with NetFlow, but many network monitoring and analysis tools support both formats. 

sFlow exports sampled packets in near real-time, and unlike NetFlow, there is no flow cache on the network device. This can make sFlow a more scalable option in very high-speed networks. However, it’s critical to configure an appropriate sampling rate. Sampling a greater percentage of the total packets can yield greater accuracy, but too high a percentage negates the benefits of statistical sampling. There is, unfortunately, no “correct” sampling rate; the number of variables involved means that each network will be different.

What is IPFIX?

An IETF standard that emerged in the early 2000s, Internet Protocol Flow Information Export (IPFIX) is extremely similar to NetFlow. In fact, NetFlow v9 served as the basis for IPFIX. The primary difference between the two is that IPFIX is an open standard, and is supported by many networking vendors apart from Cisco. With the exception of a few additional fields added in IPFIX, the formats are otherwise nearly identical. In fact, IPFIX is sometimes even referred to as “NetFlow v10”.

Owing in part to its similarities to NetFlow, IPFIX enjoys wide support among network monitoring solutions as well as network equipment.

Network Flow Tools Compared

Considerations for Flow Monitoring

As discussed previously, there are some important considerations when deploying any flow monitoring solution, whether it be NetFlow, sFlow, or IPFIX. The choice of what platform or tool you use to analyze your flow records can greatly impact the value you get out of flow monitoring. 

You’ll also want to consider storage requirements, which can vary based on how much traffic you’re monitoring or sampling. 

Perhaps most importantly, you’ll want to consider whether flow monitoring is even the most appropriate type of Network Traffic Analysis for your needs. Do you need the deeper level of inspection possible with a full packet capture? Or will metadata and summarized information suffice? 

Is This Solution Right for You?

Flow Monitoring is an indispensable addition to any network engineer’s toolbox, but it’s not the be-all and end-all, especially when it comes to security. Packet sampling in particular can miss highly sophisticated threats that don’t make a lot of noise. Pairing the network-centric perspectives of flow monitoring with a data-centric solution like the Varonis Data Protection Platform can help you avoid blind spots, identify advanced threats, and maintain compliance. If you’re interested in learning more about how Varonis can complement your existing monitoring solutions, schedule a one-on-one demo today!