igb netstat input counters 2x?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

igb netstat input counters 2x?

John-Mark Gurney-2
I'm doing some perf testing on an APU4 board, and I noticed that
it looks like the input netstat counters are 2x than what they should
be.

I was seeing 60MiB/sec via netstat -w 1 -I igb1:
     40034     0     0   60760352       2538     0     177909     0
     40700     0     0   61776228       2574     0     180300     0

But the program was only reading 27MB/sec.  I decided to read the mac
stats directly via:
bytes=$(sysctl -n dev.igb.1.mac_stats.good_octets_recvd); while sleep 1; do
        nbytes=$(sysctl -n dev.igb.1.mac_stats.good_octets_recvd)
        echo $(($nbytes - $bytes)); bytes=$nbytes
done

and saw much more reasonable numbers:
31099740
30512488
30675974

Which is more in line w/ the 27MB/sec that the program reports.

I haven't looked at the code to see what could be causing the double
counting.  Also, the output numbers appear to be accurate.

This is with 13.0-CURRENT from the July 25th snapshot, which is r350322.

--
  John-Mark Gurney Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: igb netstat input counters 2x?

Andrey V. Elsukov
On 14.08.2019 03:27, John-Mark Gurney wrote:

> I'm doing some perf testing on an APU4 board, and I noticed that
> it looks like the input netstat counters are 2x than what they should
> be.
>
> I was seeing 60MiB/sec via netstat -w 1 -I igb1:
>      40034     0     0   60760352       2538     0     177909     0
>      40700     0     0   61776228       2574     0     180300     0
>
> But the program was only reading 27MB/sec.  I decided to read the mac
> stats directly via:
> bytes=$(sysctl -n dev.igb.1.mac_stats.good_octets_recvd); while sleep 1; do
> nbytes=$(sysctl -n dev.igb.1.mac_stats.good_octets_recvd)
> echo $(($nbytes - $bytes)); bytes=$nbytes
> done
>
> and saw much more reasonable numbers:
> 31099740
> 30512488
> 30675974
>
> Which is more in line w/ the 27MB/sec that the program reports.
>
> I haven't looked at the code to see what could be causing the double
> counting.  Also, the output numbers appear to be accurate.
>
> This is with 13.0-CURRENT from the July 25th snapshot, which is r350322.
Does this doubling happens only with IBYTES counter? What about
IPACKETS? Also I'd check L2/L3 addresses to be sure that they by
accident are not broadcast/multicast.

--
WBR, Andrey V. Elsukov


signature.asc (566 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: igb netstat input counters 2x?

John-Mark Gurney-2
Andrey V. Elsukov wrote this message on Thu, Aug 15, 2019 at 17:38 +0300:

> On 14.08.2019 03:27, John-Mark Gurney wrote:
> > I'm doing some perf testing on an APU4 board, and I noticed that
> > it looks like the input netstat counters are 2x than what they should
> > be.
> >
> > I was seeing 60MiB/sec via netstat -w 1 -I igb1:
> >      40034     0     0   60760352       2538     0     177909     0
> >      40700     0     0   61776228       2574     0     180300     0
> >
> > But the program was only reading 27MB/sec.  I decided to read the mac
> > stats directly via:
> > bytes=$(sysctl -n dev.igb.1.mac_stats.good_octets_recvd); while sleep 1; do
> > nbytes=$(sysctl -n dev.igb.1.mac_stats.good_octets_recvd)
> > echo $(($nbytes - $bytes)); bytes=$nbytes
> > done
> >
> > and saw much more reasonable numbers:
> > 31099740
> > 30512488
> > 30675974
> >
> > Which is more in line w/ the 27MB/sec that the program reports.
> >
> > I haven't looked at the code to see what could be causing the double
> > counting.  Also, the output numbers appear to be accurate.
> >
> > This is with 13.0-CURRENT from the July 25th snapshot, which is r350322.
>
> Does this doubling happens only with IBYTES counter? What about
> IPACKETS? Also I'd check L2/L3 addresses to be sure that they by
> accident are not broadcast/multicast.
as for L2 addresses, it shouldn't be, but:
root@gate3:~ # arp -an | grep igb1
? (69.x.x.1) at a0:xx:xx:xx:xx:xx on igb1 expires in 777 seconds [ethernet]
? (69.x.x.27) at f8:xx:xx:xx:xx:xx on igb1 permanent [ethernet]

It's solely the upload to my fiber, so there should only be two hosts
on it..  my machine, and the gateway, and things look that way...
Also, traffic was generated by https, and counters go to near zero when
not collecting stats, and then immediately jump to double when running
the fetch...

Looks like the pkts are doubled as well, which lends itself to the fact
that packets are being double counted...
            input           igb1           output
   packets  errs idrops      bytes    packets  errs      bytes colls
     39668     0     0   60193686       2507     0     175751     0
     39042     0     0   59247132       2469     0     173104     0

and I expanded the above to pull the pkts as well and got:

good_octets good mcast bcast total
29935544        19676   0       0       19672
29868462        19629   0       0       19629

So, looks like they are double counted.

The later was generated using:
bytes=$(sysctl -n dev.igb.1.mac_stats.good_octets_recvd);
mpkts=$(sysctl -n dev.igb.1.mac_stats.mcast_pkts_recvd);
bpkts=$(sysctl -n dev.igb.1.mac_stats.bcast_pkts_recvd);
gpkts=$(sysctl -n dev.igb.1.mac_stats.good_pkts_recvd);
tpkts=$(sysctl -n dev.igb.1.mac_stats.total_pkts_recvd);
while sleep 1; do
        nbytes=$(sysctl -n dev.igb.1.mac_stats.good_octets_recvd)
        nmpkts=$(sysctl -n dev.igb.1.mac_stats.mcast_pkts_recvd);
        nbpkts=$(sysctl -n dev.igb.1.mac_stats.bcast_pkts_recvd);
        ngpkts=$(sysctl -n dev.igb.1.mac_stats.good_pkts_recvd);
        ntpkts=$(sysctl -n dev.igb.1.mac_stats.total_pkts_recvd);
        printf "%d\t%d\t%d\t%d\t%d\n" $(($nbytes - $bytes)) $(($ngpkts - $gpkts)) $(($nmpkts - $mpkts)) $(($nbpkts - $bpkts)) $(($ntpkts - $tpkts))
        bytes=$nbytes
        mpkts=$nmpkts
        bpkts=$nbpkts
        gpkts=$ngpkts
        tpkts=$ntpkts
done

--
  John-Mark Gurney Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."

signature.asc (968 bytes) Download Attachment