Comparing Netflow and SNMP data collection

MRTG

SNMP

Netflow

Netflow smoothed

We would be pleased if interface data reconstructed from netflow statistics was in all respects identical to interface data extracted with SNMP. Unfortunately, netflow has mechanisms that add what looks like noise to the reconstructed data.

The plot represented as 'MRTG' is a snapshot of an MRTG gif. In the second graph labeled 'SNMP', we used gnuplot to present 3 minute timed samples extracted directly with an SNMP client to prove that MRTG is not smoothing the data it collects. The third graph plots v8 netflow data for the same time period. And the fourth graph takes the netflow data from the third and applies a 9-point smoothing function to make comparison with the SNMP data easier.

Cisco Netflow aggregates packets into flows, reducing the bulk of the collected data but allocating all of it a single instant in time. These flows are then collected and aggregated again into Autonomous System flows. If a VIP-daemon process reaps flows on a time scale long relative to the sampling period, it would tend to add noise to the data by shifting bytes to time periods when the reaper is running. The daemon hypothesis might be tested by spectral analysis of the unsmoothed data to look for a peak which might be the reciprical of the daemon run rate.

This data is from the CALREN2-South interface facing Cable & Wireless. If the traffic dip at approx 22:20 (first occurance) was a real network hiccup we would have lost some number of UDP netflow datagrams. That would tend to amplify the depth of the notch. We do not know of a network event that matches this notch. There may be other explanations.