NetFlow 笔记

1. NetFlow 笔记

1.1. 模拟器

• https://github.com/nerdalert/nflow-generator

1.2. 什么是 NetFlow?

NetFlow 是最广泛使用的流量数据统计标准, 由 Cisco 开发, 用于监控和记录进出接口的所有流量。NetFlow 分析它收集的流量数据, 以提供关于流量和流量的可见性, 并跟踪流量从何处来、流向何处以及在任何时候生成的流量。记录的信息可用于使用情况监视、异常检测和其他各种网络管理任务。

1.3. Cisco NetFlow 版本

所有初始 NetFlow 版本都支持第一种 NetFlow 格式。版本 2、3 和 4 仅作为内部版本提供。Netflow v5 是最流行的版本, 许多路由器品牌仍然支持它。NetFlow v5 有一个固定的数据包格式, 这使得 NetFlow 流量监控和报告更加容易, 因为每个数据包的内容都可以快速识别。版本 5 引入了多种增强功能, 如 BGP 作为信息和流序列号。

尽管版本 7 和版本 8 有一些增强, 但它们已经不再使用。与此同时, NetFlow 版本 9 正在慢慢普及。Netflow v9 中的数据包格式是动态的, 这个版本具有 FNF 功能, 使其更加灵活。IPFIX 通常被称为 Netflow v10, 其大部分功能都是基于 Netflow v9 构建的, 但它只是 NetFlow 的一个行业标准化版本。

版本	说明
v1	Cisco 首次实现, 仅限于 IPv4。这个版本现在已经过时了。
v2	内部版本-从未公开发布。
v3	内部版本-从未公开发布。
v4	内部版本-从未公开发布。
v5	这是最常见的版本, 在大多数路由器品牌上都有。
v6	思科不再支持。
v7	与版本 5 类似, 具有源路由器并添加了对 Cisco catalyst 交换机的支持。这个版本现在已经过时了。
v8	支持各种基于路由器的聚合方案。这个版本现在已经过时了。
v9	此版本基于模板, 主要用于支持带有 BGP nexthop 流的 IPv6、MLPS 和 IPv4。
v10	用于识别 IPFIX。

1.4. NetFlow 是如何工作的?

Cisco NetFlow 包括以下组件:

1.4.1. IP 流量

IP 流是一组具有一组特定 IP 数据包属性的数据包, 交换机或路由器中转发的每个数据包都包含以下信息:

• IP 源
• IP 目的地
• 源端口
• 目的港
• 服务等级
• 第三层协议类型
• 接口

1.4.2. NetFlow 缓存

NetFlow 缓存是一个压缩信息的数据库, 这些信息来自于对 IP 包进行监视和分组后生成的数据。

1.4.3. NetFlow 收集器

流被分组为导出流数据报, 并使用流收集器接收和处理的用户数据报协议 (UDP) 导出。

访问 NetFlow 数据有两种方法: 使用 CLI 或 NetFlow 收集器。NetFlow collector 或 NetFlow traffic analyzer 是一种报告服务器, 用于收集和分析流量数据, 以便于故障排除。它不是硬件设备就是软件程序。

1.5. 为什么使用 NetFlow?

NetFlow 数据可用于多个网络管理任务, 例如:

• 监控: 监控您的网络, 跟踪进出流量, 并确定顶级用户。
• 产能规划: 跟踪网络使用情况, 以评估未来的带宽需求。
• 安全性分析: 检测网络行为的变化以识别网络异常。使用这些数据作为有价值的取证工具来理解和回放安全事件的历史, 以便您的安全团队可以从中学习。
• 故障排除: 诊断并排除网络减速、带宽占用和流量峰值。使用报告工具快速了解网络难点。
• QoS 参数验证: 确保为每一类服务 (CoS) 分配适当的带宽, 以避免关键 CoS 的订阅不足。

1.6. SNMP 与 NetFlow

SNMP 是带宽监控中最古老、最有效的协议之一。虽然 SNMP 是实时监控最相关的选项, 但只有 NetFlow 可以提供有关网络的用途和用户的信息。NetFlow 适用于高流量的复杂网络以及异常检测。

1.7. sFlow、NetFlow、SNMP 三者之间有什么不同?

高效的网络监控和流量管理是确保网络性能最佳的重要方法, 目前, sFlow、NetFlow、SNMP 可提供不同的方式来监控管理网络流量, 那么有人就会问了, sFlow、NetFlow、SNMP 之间有什么区别? 采用哪种方式最好? 本文将通过对比 sFlow 与 NetFlow、sFlow 与 SNMP 以及 NetFlow 与 SNMP 之间的区别, 来解答上述问题。

1.7.1. sFlow 与 NetFlow 有什么不同?

sFlow 是一种用于监控数据网络上交换机或路由器流量转发状况的技术, 采用内置在硬件中的专用芯片, 旨在消除路由器或交换机的 CPU 和内存负担。一个 sFlow 系统主要由若干内嵌于交换机或路由器等转发设备(即 sFlow Agent, sFlow 代理)和一个核心 sFlow Collector(即 sFlow 采集器)组成。如下图, 分布在网络不通位置的 sFlow 代理把 sFlow 数据报源源不断地传送给中央 sFlow 采集器, 然后采集器会对 sFlow 数据报进行分析并生成丰富、实时、全网范围的传输流视图或表报, 帮助网络管理员更加有效地管理整个站点的网络流量。由于 sFlow 是一种纯数据包采样技术(基于样本), 如果没有复杂的算法来尝试推测准确的会话字节量, 那么几乎不可能获得每台主机流量 100% 的准确值。

而 NetFlow 是一种基于软件的技术, 是思科网络设备互联网操作系统 (IOS) 中专有的技术, 主要用来将网络流量记录到设备的高速缓存中, 从而提供精准的流量测试。一个 NetFlow 系统主要包含了三个部分: 探测器(即 NetFlow Monitor)、采集器(即 NetFlow collector) 和报告系统(即 Flow Records)。如下图, NetFlow 利用标准的交换模式处理数据流的第一个 IP 包数据, 然后生成 NetFlow 缓存, 随后同样的数据基于缓存信息在同一个数据流中进行传输, 不再匹配相关的访问控制等策略, NetFlow 缓存同时包含了随后数据流的统计信息。由于 NetFlow 是用来跟踪每个开启 NetFlow 功能接口上的所有进入会话的技术, 因此它可以接近 100%的准确性来描述谁经过设备进行了通信。

虽然 sFlow 和 NetFlow 都是通过对交换机的端口或路由器的接口之间的流量进行采样来提供有关网络活动的数据的技术, 但两者之间有着明显的区别, 如, NetFlow 技术只针对 IP 流量; 而 sFlow 技术可采样所有内容并访问来自 OSI 第二层到第七层的流量。两者之间的对比见下方表格:

特点	NetFlow	sFlow
流量监控方式	收集并缓存经过设备的 IP 流量供管理员进行进一步检查分析。	随机采样数据包, 以计算出更广泛的网络流量趋势。
优势	1. NetFlow 可以对特定网络位置的每个数据包进行采样, 这使网络管理员可以深度监视和分析收集的数据。 2. NetFlow 可以捕获每个数据流, 并完整地包含体积信息, 并在信息收集到收集器之前对其进行汇总, 使得数据监控的精确度更高。	sFlow 的监视功能范围更广, 其采样分析方式有助于故障排除和识别异常/问题流量模式。用户无需通过尝试记录所有内容, 可减少对交换机造成的 CPU 压力, 但仍可以根据采样的数据迅速诊断异常行为。
不足	1. NetFlow 不能提供实时流量监控, 网络流量收集与查看之间存在时延, 可能导致网络中的问题监控不及时。 2. NetFlow 在传统的传统网络上具有出色的可见性, 但在具有云服务和软件定义的网络的更复杂的网络上, 数据可见性一般。 3. NetFlow 需要对监控的网络流量数据先进行缓存再做分析处理, 大量的数据缓存容易导致交换机/路由器的性能问题。	1. sFlowd 的采样分析不能像 Netflow 那样提供详细的数据包分析, 而且在数据采集数量有限时, 随机采样的方式仅适用于一般数据分析, 不利于深层分析。 2. 如果 sFlow 使用高采样率, 则精度不是问题, 但若采样率很低, 则用户将获得不可靠且不准确的采样数据。 3. 如果要清晰的监控网络流量状况, 网络中的每个交换机和路由器都必须兼容 sFlow。如果支持 sFlow 的组件数量有限, 采集数据的不准确性将会进一步增加。 4. sFlow 的随机采样只能针对网络中的部分数据进行分析, 容易忽略其他大量未被分析的网络流量中的恶意数据包, 致使网络受到外部攻击

1.7.2. sFlow 与 SNMP 有什么区别?

SNMP(即简单网络管理协议)是一种标准的 Internet 协议, 用于收集和组织有关 IP 网络上被管理设备的信息, 同时支持修改该信息以更改设备行为, 此外, SNMP 可监控路由器和交换机端口的带宽利用, 以及 CPU 负荷和内存利用等设备读数。与用于特定流量收集、分析和监控的 sFlow 相比, SNMP 通常用于收集基本流量详细信息, 例如字节/数据包。但 SNMP 可通过"利用网络管理网络"的方式, 实现对网络设备的批量管理, 从而提高设备管理效率。因此在多数情况下, SNMP 使用的更为广泛。欲知更多有关 SNMP 的信息可访问《【疑问解答】什么是 SNMP? SNMP 是如何工作的? 》。

1.7.3. NetFlow 与 SNMP 有什么区别?

相对于 SNMP 而言, NetFlow 是一种更紧凑的协议, 它可以更好地扩展性能采集和网络流量管理。NetFlow 与 SNMP 两者之间的区别有:

SNMP 可对网络实行实时(每秒)监控, 而 NetFlow 虽然可为每个流提供开始时间和结束时间, 但它不如 SNMP 实时。

NetFlow 会告诉您谁在消耗带宽以及消耗带宽的原因, 相比 SNMP 更加细致, 因此 NetFlow 会消耗更多的磁盘空间来存储历史信息。

NetFlow 会告诉您谁在消耗带宽以及消耗带宽的原因, 相比 SNMP 更加细致, 因此 NetFlow 会消耗更多的磁盘空间来存储历史信息。

SNMP 能用于收集 CPU 和内存使用信息, 但 NetFlow 不能。

1.7.4. sFlow VS NetFlow VS SNMP: 谁更合适?

由上可知, sFlow、NetFlow、SNMP 之间最大的区别在于: SNMP 用于标准的网络监控, 而 sFlow 和 NetFlow 用于网络流量收集、监控和分析。sFlow 与 NetFlow 相比, sFlow 更合适在多协议网络中使用, 而 NetFlow 更合适需要提高准确性和可扩展性且基于 IP 流量的情况下使用。目前市面上供应商正在推出同时支持 sFlow、NetFlow 和 SNMP 的以太网交换机, 为用户提供一种更为经济明智的选择。

1.8. Enabling NetFlow on Cisco ASA

Access the firewall CLI:

1. Telnet or SSH into the firewall.
2. Enter privileged mode by typing enable and entering your enable password.

Then:

1. Log into your firewall via CLI and enter enable mode, then enter configure terminal mode.

User Access Verification
Password:********
Type help or '?' for a list of available commands.
PetesASA> enable
Password: ********
PetesASA# conf t
PetesASA(config)#

2. We haven’t set one up yet, but we need to let the firewall know the IP address that the NetFlow “Collector” will be running on, in this case I’m going to use 10.254.254.253. (Note: the port number on the end is unimportant).

PetesASA(config)#
PetesASA(config)# flow-export destination inside 10.254.254.234 2055
PetesASA(config)#

3. The next command aggregates multiple events into separate NSELs on a 15 second interval.

PetesASA(config)#
PetesASA(config)# flow-export delay flow-create 15
PetesASA(config)#

4. Now we are going to set the refresh rate at which the templates are sent, if you do not do this it will default to 30 minutes.

PetesASA(config)#
PetesASA(config)# flow-export template timeout-rate 1
PetesASA(config)#

5. I’m going to apply this with the default global-policy, because most of you will have one, (Though I notice every 8.2(1) 5505 I’ve put in recently does NOT have one so check).

PetesASA(config)# policy-map global_policy
PetesASA(config-pmap)# class class-default
PetesASA(config-pmap-c)# flow-export event-type all destination 10.254.254.234
PetesASA(config-pmap-c)# exit

6. If you haven’t got a global policy, this will not apply until you have applied the global_policy globally, this is done with a service-policy command, check to see if you already have this command in your config, or simply execute the command and the firewall and will tell you, like so….

Note: If it does not error then it was NOT applied 🙂

PetesASA(config)#
PetesASA(config)# service-policy global_policy global
WARNING: Policy map global_policy is already configured as a service policy
PetesASA(config)#

7. Don’t forget the save the config with a write mem command.

1.8.1. Configure the NetFlow exporter (ASA v.7.x)

Run the following command. Replace AuvikCollectorIP with the IP of your Auvik collector and AuvikPort with one of the following ports: 2055, 2056, 4432, 4739, 6343, 9995, or 9996.

class global_class
flow-export event-type all destination <AuvikCollectorIP>
class-map flow_export_class
match access-list flow_export_acl
flow-export destination <AuvikCollectorIP> <AuvikPort>
flow-export template timeout-rate 15
flow-export delay flow-create 60

1.8.2. Configure the NetFlow exporter (ASA v.8.4)

Run the following command. Replace AuvikCollectorIP with the IP of your Auvik collector and AuvikPort with one of the following ports: 2055, 2056, 4432, 4739, 6343, 9995, or 9996.

flow-export destination inside <AuvikCollectorIP> <AuvikPort>
flow-export template timeout-rate 15
flow-export delay flow-create 60

class-map flow_export_class
match any

policy-map global_policy
class flow_export_class
flow-export event-type all destination <AuvikCollectorIP>

service-policy global_policy global

1.8.3. NetFlow Configuration Example - Cisco ASA

!
flow-export destination inside 1.1.1.1 2055 
flow-export template timeout-rate 1 
flow-export delay flow-create 60 
access-list netflow-export extended permit ip any any 
class-map netflow-export-class 
match access-list netflow-export 
policy-map global_policy 
class netflow-export-class 
flow-export event-type all destination 1.1.1.1 
service-policy global_policy global
! 

More:

• https://techmusa.com/netflow-config-asa-router/
• https://www.solarwinds.com/free-tools/netflow-configurator

1.9. What is NetFlow? An Overview of the NetFlow Protocol

1.9.1. What is NetFlow?

NetFlow is a protocol used to collect metadata on IP traffic flows traversing a network device.

Developed by Cisco Systems, NetFlow is used to record metadata about IP traffic flows traversing a network device such as a router, switch, or host. A NetFlow-enabled device generates metadata at the interface level and sends flow data information to a flow collector, where the flow records are stored to enable network traffic analytics and management. A network operator can use NetFlow data to determine network throughput, packet loss, and traffic congestion at a specific interface level. NetFlow data also supports other network-level monitoring use cases such as DDoS detection and BGP peering.

1.9.2. The History of NetFlow and Versions Over Time

Netflow was initially developed by Cisco in 1995 to monitor and record all network traffic coming through their network devices. Over time, they realized that the network flow data was incredibly useful and it led to an entire branch of network monitoring that several other platforms have copied and rebranded. Now, a version of NetFlow protocols have become industry standard for optimizing network performance. This ongoing evolution of NetFlow has led to several versions over the years with different features.

1.9.3. NetFlow Variants

While the term “NetFlow” is commonly used to refer to all types of flow records and datagrams, there are actually three important variants in regular use within live production networks:

• NetFlow is the technology and term used exclusively by Cisco Systems.

• IPFIX is an IETF standard flow record format that is very similar in approach and structure to NetFlow v9 (see more on NetFlow version numbering below). It is sometimes called “NetFlow v10” since IPFIX plays a key role in coalescing all NetFlow variants and equivalents as the standards process evolves the IPFIX specifications over time.

• sFlow is a similar but importantly different type of flow protocol and data record standard introduced and promoted by InMon Corp. sFlow does not sample all packets like NetFlow does, nor does it timestamp traffic flows. It relies on accurate and reliable statistical sampling methods for documenting flows, thereby reducing the amount of flow information that ultimately needs processing and analysis.

• Other vendor-specific flow record formats that are similar in nature to one of three most common variants listed above (in most cases these are either substitutions or very close adaptations) include:

  – J-Flow from Juniper Networks, which essentially conforms to NetFlow v5;

  – NetStream from 3Com/Huawei.

• xFlow, while not a variant, is the generic term often used to refer collectively to all flow record variants (NetFlow, sFlow, IPFIX, J-Flow, etc.).

1.9.4. NetFlow Versions

The fields that make up a NetFlow flow record depend on the version of NetFlow supported by the NetFlow exporter. Since the protocol was first introduced by Cisco in 1996 it has been updated numerous times in a series of backward-compatible versions, the most commonly used of which are versions 5 and 9:

• v1: First implementation, now obsolete, and restricted to IPv4 (without IP mask and AS Numbers).
• v2: Cisco internal version, never released.
• v3: Cisco internal version, never released.
• v4: Cisco internal version, never released.
• v5: First commonly deployed version, available (as of 2009) on many routers from different brands, but restricted to IPv4 flows.
• v6: No longer supported by Cisco.
• v7: Like version 5 with a source router field.
• v8: Several aggregation forms, but only for information that is already present in version 5 records
• v9: Template based, available (as of 2009) on some recent routers. Mostly used to report flows like IPv6, MPLS, or even plain IPv4 with BGP nexthop.Includes added fields for security and traffic analysis use cases

1.9.5. How Does NetFlow Work?

1.9.6. NetFlow Components

NetFlow monitoring solutions are typically comprised of three main components:

• Flow exporter: A NetFlow-enabled device that generates flow records and periodically exports them to a flow collector.
• Flow collector: A program running on a server that is responsible for receiving, storing, and pre-processing flow records received from NetFlow-enabled devices.
• Flow analyzer: An analysis application that processes flow records collected by the flow collector into reports, alerts, and other interpreted results.

1.9.7. Flow Records and Data

A NetFlow exporter (NetFlow-enabled device) identifies a flow as a unidirectional stream of packets having in common (at least) the following:

• Input interface port
• IP source address
• IP destination address
• Source port number
• Destination port number
• Layer 3 protocol field
• Type of service

These same attributes that define a given set of IP packets as a flow make up the core metadata (information about the flow rather than the information that’s actually in the packets) that is included in the NetFlow “flow record” for that flow. Each time a new unidirectional IP traffic flow starts traversing a device a new NetFlow flow record is created and tracked in the device’s on-board NetFlow cache. And of course, the function of NetFlow is to export this network flow data to a NetFlow collector for storage and analysis.

A flow record is ready for export when one of the following is true about the corresponding flow:

• The flow is inactive (no new packets received) for a duration defined in a timer. Timers are configurable but defaults are typically used.
• The flow is long-lived (active) but lasts for longer than the active timer (e.g., a long FTP download).
• A TCP flag (i.e., FIN, RST) indicates that the flow is terminated.

At export, the flow record is encapsulated in a UDP datagram and sent to a NetFlow collector that is typically external to a NetFlow-enabled device. The collector collects and stores the flow metadata in a record format that is defined by the protocol. The data points found in a NetFlow record typically include:

• Source and destination IP address
• Source and destination TCP/User Datagram Protocol (UDP) ports
• Type of service (ToS)
• Packet and byte counts
• Start and end timestamps
• Input and output interface numbers
• TCP flags and encapsulated protocol (TCP/UDP)
• BGP routing information (next-hop address, source autonomous system (AS) number, destination AS number, source prefix mask, destination prefix mask)

After the flow collector receives this flow record, it’s sent to the flow analyzer to extract the data for insights. Network admins use these insights to identify security threats and traffic patterns such as application performance and bandwidth usage that serve to improve the overall performance of a network.

1.9.8. Why use NetFlow: Key Benefits and Capabilities

By now it should be obvious that organizations can utilize flow-based analysis techniques with NetFlow to visualize traffic patterns for complete network visibility. This makes the responsibilities of the network admins considerably easier in many ways.

Some of the key benefits include:

• Optimized bandwidth usage and capacity planning
• Deeper network visibility
• Enabling of root cause analytics for application performance slowdowns
• Identification and investigation of security threats on the network

In addition, it has multiple use cases for network monitoring:

• Network, user, and application monitoring
• Capacity planning
• Cost reporting and analysis of network usage
• Network security
• Network troubleshooting

1.9.9. Related Resources about NetFlow

For an excellent overview of the origins, evolution and extensibility of NetFlow, see Kentik CEO Avi Freedman’s blog posts on NetFlow, sFlow, and Flow Extensibility: Part 1 and NetFlow, sFlow, and Flow Extensibility: Part 2.

More NetFlow resources:

• The Network Traffic Analytics that Enterprises Need: Learn why traditional network monitoring tools don’t scale to meet today’s network visibility needs, why more enterprises are moving from faults & packets to flow, and how Kentik can help.
  Related Kentipedia entries:

• NetFlow Analysis

• NetFlow Collector: What is a NetFlow Collector?

UPDATED: JULY 20, 2023