I have a mrtg job that runs every 5 minutes on a FreeBSD-CURRENT box and
gets this randomly: SNMP Error: send_query: No buffer space available SNMPv1_Session (remote host: "main-switch.lerctr.org" [192.168.200.224].161) community: "lerctr" request ID: 1517898706 PDU bufsize: 8000 bytes timeout: 2s retries: 5 backoff: 1) at /usr/local/lib/perl5/site_perl/SNMP_util.pm line 492. SNMPGET Problem for ifInOctets.39 ifOutOctets.39 sysUptime sysName on [hidden email]::::::v4only at /usr/local/bin/mrtg line 2330. 2019-06-03 10:50:00: WARNING: Skipping host main-switch.lerctr.org as it does not resolve to an IPv4 address 2019-06-03 10:50:00: WARNING: no data for ifInOctets&ifOutOctets:[hidden email]. Skipping further queries for Host main-switch.lerctr.org in this round. 2019-06-03 10:50:00: WARNING: Skipping host sewingroomswitch.lerctr.org as it does not resolve to an IPv4 address 2019-06-03 10:50:00: WARNING: no data for ifInOctets&ifOutOctets:[hidden email]. Skipping further queries for Host sewingroomswitch.lerctr.org in this round. 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_39][_IN_] ' $target->[24]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_39][_OUT_] ' $target->[24]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_40][_IN_] ' $target->[25]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_40][_OUT_] ' $target->[25]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_41][_IN_] ' $target->[26]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_41][_OUT_] ' $target->[26]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_42][_IN_] ' $target->[27]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_42][_OUT_] ' $target->[27]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_43][_IN_] ' $target->[28]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_43][_OUT_] ' $target->[28]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_44][_IN_] ' $target->[29]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_44][_OUT_] ' $target->[29]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_45][_IN_] ' $target->[30]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_45][_OUT_] ' $target->[30]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_46][_IN_] ' $target->[31]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_46][_OUT_] ' $target->[31]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_47][_IN_] ' $target->[32]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_47][_OUT_] ' $target->[32]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_48][_IN_] ' $target->[33]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[main-switch.lerctr.org_48][_OUT_] ' $target->[33]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[sewingroomswitch.lerctr.org_1][_IN_] ' $target->[34]{$mode} ' did not eval into defined data 2019-06-03 10:50:01: ERROR: Target[sewingroomswitch.lerctr.org_1][_OUT_] ' $target->[34]{$mode} ' did not eval into defined data I have: kern.ipc.maxsockbuf=2147483648 in my sysctl.conf as suggested in a bunch of places, and the box has plenty (128G) of memory. How can I debug this? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: [hidden email] US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
03.06.2019 22:56, Larry Rosenman wrote:
> I have a mrtg job that runs every 5 minutes on a FreeBSD-CURRENT box and gets this randomly: > > SNMP Error: > send_query: No buffer space available [skip] > How can I debug this? Your outgoing network interface stalls for some reason. Where is this traffic directed to? Some kind of VPN tunnel? Physical NIC that looses link? Some Wifi or mobile network? _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
On 06/03/2019 12:23 pm, Eugene Grosbein wrote:
> 03.06.2019 22:56, Larry Rosenman wrote: > >> I have a mrtg job that runs every 5 minutes on a FreeBSD-CURRENT box >> and gets this randomly: >> >> SNMP Error: >> send_query: No buffer space available > > [skip] > >> How can I debug this? > > Your outgoing network interface stalls for some reason. > Where is this traffic directed to? Some kind of VPN tunnel? Physical > NIC that looses link? > Some Wifi or mobile network? > ⌂66% [[hidden email]:~] $ ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> bce0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> ether a4:ba:db:29:66:95 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> bce1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> ether a4:ba:db:29:66:97 media: Ethernet autoselect nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> bce2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> ether a4:ba:db:29:66:99 media: Ethernet autoselect nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> bce3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> ether a4:ba:db:29:66:9b media: Ethernet autoselect nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 02:d7:b8:51:f2:00 inet6 fe80::d7:b8ff:fe51:f200%bridge0 prefixlen 64 scopeid 0x6 inet6 2600:1700:210:b180:d7:b8ff:fe51:f200 prefixlen 64 autoconf inet 192.168.200.4 netmask 0xfffffc00 broadcast 192.168.203.255 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: bce0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 2 priority 128 path cost 55 groups: bridge nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> ⌂68% [[hidden email]:~] $ bridge0 via bce0 to the "main-switch" . and all the AP's,switches, etc. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: [hidden email] US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
04.06.2019 0:49, Larry Rosenman wrote:
> On 06/03/2019 12:23 pm, Eugene Grosbein wrote: >> 03.06.2019 22:56, Larry Rosenman wrote: >> >>> I have a mrtg job that runs every 5 minutes on a FreeBSD-CURRENT box and gets this randomly: >>> >>> SNMP Error: >>> send_query: No buffer space available >> >> [skip] >> >>> How can I debug this? >> >> Your outgoing network interface stalls for some reason. >> Where is this traffic directed to? Some kind of VPN tunnel? Physical >> NIC that looses link? >> Some Wifi or mobile network? >> > physical NIC on the same physical network (all Ubiquiti gear). > > > ⌂66% [[hidden email]:~] $ ifconfig > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 > options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 > inet 127.0.0.1 netmask 0xff000000 > groups: lo > nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> > bce0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> > ether a4:ba:db:29:66:95 > media: Ethernet autoselect (1000baseT <full-duplex>) > status: active > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > bce1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> > ether a4:ba:db:29:66:97 > media: Ethernet autoselect > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > bce2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> > ether a4:ba:db:29:66:99 > media: Ethernet autoselect > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > bce3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> > ether a4:ba:db:29:66:9b > media: Ethernet autoselect > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ether 02:d7:b8:51:f2:00 > inet6 fe80::d7:b8ff:fe51:f200%bridge0 prefixlen 64 scopeid 0x6 > inet6 2600:1700:210:b180:d7:b8ff:fe51:f200 prefixlen 64 autoconf > inet 192.168.200.4 netmask 0xfffffc00 broadcast 192.168.203.255 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: bce0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> > ifmaxaddr 0 port 2 priority 128 path cost 55 > groups: bridge > nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> > ⌂68% [[hidden email]:~] $ > > bridge0 via bce0 to the "main-switch" . and all the > AP's,switches, etc. You should search system logs for possible link problems and check counters shown by "netstat -idnh" (errs/drop/coll). Also check counters by "netstat -m" for mbufs/mbuf clusters. _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
On 06/03/2019 1:09 pm, Eugene Grosbein wrote:
> 04.06.2019 0:49, Larry Rosenman wrote: > >> On 06/03/2019 12:23 pm, Eugene Grosbein wrote: >>> 03.06.2019 22:56, Larry Rosenman wrote: >>> >>>> I have a mrtg job that runs every 5 minutes on a FreeBSD-CURRENT box >>>> and gets this randomly: >>>> >>>> SNMP Error: >>>> send_query: No buffer space available >>> >>> [skip] >>> >>>> How can I debug this? >>> >>> Your outgoing network interface stalls for some reason. >>> Where is this traffic directed to? Some kind of VPN tunnel? Physical >>> NIC that looses link? >>> Some Wifi or mobile network? >>> >> physical NIC on the same physical network (all Ubiquiti gear). >> >> >> ⌂66% [[hidden email]:~] $ ifconfig >> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 >> options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> >> inet6 ::1 prefixlen 128 >> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 >> inet 127.0.0.1 netmask 0xff000000 >> groups: lo >> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> >> bce0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> >> metric 0 mtu 1500 >> >> options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> >> ether a4:ba:db:29:66:95 >> media: Ethernet autoselect (1000baseT <full-duplex>) >> status: active >> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >> bce1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 >> >> options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> >> ether a4:ba:db:29:66:97 >> media: Ethernet autoselect >> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >> bce2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 >> >> options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> >> ether a4:ba:db:29:66:99 >> media: Ethernet autoselect >> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >> bce3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 >> >> options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE> >> ether a4:ba:db:29:66:9b >> media: Ethernet autoselect >> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> >> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 >> mtu 1500 >> ether 02:d7:b8:51:f2:00 >> inet6 fe80::d7:b8ff:fe51:f200%bridge0 prefixlen 64 scopeid 0x6 >> inet6 2600:1700:210:b180:d7:b8ff:fe51:f200 prefixlen 64 autoconf >> inet 192.168.200.4 netmask 0xfffffc00 broadcast 192.168.203.255 >> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 >> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 >> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >> member: bce0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> >> ifmaxaddr 0 port 2 priority 128 path cost 55 >> groups: bridge >> nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> >> ⌂68% [[hidden email]:~] $ >> >> bridge0 via bce0 to the "main-switch" . and all the >> AP's,switches, etc. > > You should search system logs for possible link problems > and check counters shown by "netstat -idnh" (errs/drop/coll). > > Also check counters by "netstat -m" for mbufs/mbuf clusters. ⌂79% [[hidden email]:~] $ netstat -idnh Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop lo0 16K <Link#1> lo0 27M 0 0 27M 0 0 0 lo0 - ::1/128 ::1 582k - - 582k - - - lo0 - fe80::%lo0/64 fe80::1%lo0 0 - - 0 - - - lo0 - 127.0.0.0/8 127.0.0.1 190k - - 26M - - - bce0 1.5K <Link#2> a4:ba:db:29:66:95 715M 186k 0 533M 0 0 15M bce1* 1.5K <Link#3> a4:ba:db:29:66:97 0 0 0 0 0 0 0 bce2* 1.5K <Link#4> a4:ba:db:29:66:99 0 0 0 0 0 0 0 bce3* 1.5K <Link#5> a4:ba:db:29:66:9b 0 0 0 0 0 0 0 bridg 1.5K <Link#6> 02:d7:b8:51:f2:00 715M 0 0 533M 15M 0 0 bridg - fe80::%bridge fe80::d7:b8ff:fe5 7.2k - - 7.2k - - - bridg - 2600:1700:210 2600:1700:210:b18 578M - - 476M - - - bridg - 192.168.200.0 192.168.200.4 161M - - 72M - - - ⌂86% [[hidden email]:~] $ netstat -m 3212/18193/21405 mbufs in use (current/cache/total) 2078/11604/13682/8175421 mbuf clusters in use (current/cache/total/max) 2045/8075 mbuf+clusters out of packet secondary zone in use (current/cache) 6/9355/9361/4087710 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/1211173 9k jumbo clusters in use (current/cache/total/max) 0/0/0/681285 16k jumbo clusters in use (current/cache/total/max) 4983K/65176K/70159K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 sendfile syscalls 0 sendfile syscalls completed without I/O request 0 requests for I/O initiated by sendfile 0 pages read by sendfile as part of a request 0 pages were valid at time of a sendfile request 0 pages were valid and substituted to bogus page 0 pages were requested for read ahead by applications 0 pages were read ahead by sendfile 0 times sendfile encountered an already busy page 0 requests for sfbufs denied 0 requests for sfbufs delayed ⌂83% [[hidden email]:~] $ Hrm. I may be able to move directly away from the bridge..... ⌂83% [[hidden email]:~] $ grep bce0 /var/log/messages ⌂80% [[hidden email]:~] 1 $ grep bridge /var/log/messages Jun 3 02:48:30 borg kernel: arp: 192.168.200.54 moved from 04:c9:d9:63:22:87 to 04:c9:d9:63:22:89 on bridge0 Jun 3 02:49:49 borg kernel: arp: 192.168.200.54 moved from 04:c9:d9:63:22:89 to 04:c9:d9:63:22:87 on bridge0 ⌂83% [[hidden email]:~] $ uptime 1:15PM up 1 day, 18:11, 1 user, load averages: 20.08, 18.82, 18.13 ⌂83% [[hidden email]:~] $ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: [hidden email] US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
04.06.2019 1:15, Larry Rosenman wrote:
> Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop > bce0 1.5K <Link#2> a4:ba:db:29:66:95 715M 186k 0 533M 0 0 15M Something really bad happening at driver level or physical level: way too many input errors (perhaps, bad CRC) and output drops (due to busy media?). I prefer to max out interface FIFO length while using bce(4) by means of /boot/loader.conf when link is heavily utilized: # max = 8*255 = 2040 hw.bce.tx_pages=8 Also, check driver-level stats with sysctl dev.bce.0 If possible, check link partner - managed switch port counters or what does it have. Maybe, replace twisted pair cable. _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
On 06/03/2019 1:27 pm, Eugene Grosbein wrote:
> 04.06.2019 1:15, Larry Rosenman wrote: > >> Name Mtu Network Address Ipkts Ierrs Idrop >> Opkts Oerrs Coll Drop >> bce0 1.5K <Link#2> a4:ba:db:29:66:95 715M 186k 0 >> 533M 0 0 15M > > Something really bad happening at driver level or physical level: > way too many input errors (perhaps, bad CRC) and output drops (due to > busy media?). > > I prefer to max out interface FIFO length while using bce(4) by means > of /boot/loader.conf > when link is heavily utilized: > > # max = 8*255 = 2040 > hw.bce.tx_pages=8 > > Also, check driver-level stats with sysctl dev.bce.0 > If possible, check link partner - managed switch port counters or what > does it have. > Maybe, replace twisted pair cable. Ok, I've removed the bridge (don't need it). Added the tunable and will check it after a few hours. I *REALLY* appreciate the help diagnosing this. (If need be I can replace the cable, but let's see what happens now). -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: [hidden email] US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
04.06.2019 1:42, Larry Rosenman wrote:
> Ok, I've removed the bridge (don't need it). Added the tunable > and will check it after a few hours. I *REALLY* appreciate > the help diagnosing this. (If need be I can replace the cable, but let's > see what happens now). You already run mrtg, so make sure you have graphs for bce0 usage. Maybe you have short-lived traffic storms. _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
On 06/03/2019 2:23 pm, Eugene Grosbein wrote:
> 04.06.2019 1:42, Larry Rosenman wrote: > >> Ok, I've removed the bridge (don't need it). Added the tunable >> and will check it after a few hours. I *REALLY* appreciate >> the help diagnosing this. (If need be I can replace the cable, but >> let's >> see what happens now). > > You already run mrtg, so make sure you have graphs for bce0 usage. > Maybe you have short-lived traffic storms. I upped the RX pages as well, and that seems(!) to help with having all 3 of my network backups running (bacula) which apparently was what was tripping it up. ⌂75% [[hidden email]:~] $ sysctl hw.bce hw.bce.rx_ticks: 18 hw.bce.rx_ticks_int: 18 hw.bce.rx_quick_cons_trip: 6 hw.bce.rx_quick_cons_trip_int: 6 hw.bce.tx_ticks: 80 hw.bce.tx_ticks_int: 80 hw.bce.tx_quick_cons_trip: 20 hw.bce.tx_quick_cons_trip_int: 20 hw.bce.strict_rx_mtu: 0 hw.bce.hdr_split: 1 hw.bce.tx_pages: 8 hw.bce.rx_pages: 8 hw.bce.msi_enable: 1 hw.bce.tso_enable: 1 hw.bce.verbose: 1 ⌂77% [[hidden email]:~] $ sysctl dev.bce.0 dev.bce.0.com_no_buffers: 0 dev.bce.0.stat_CatchupInRuleCheckerP4Hit: 0 dev.bce.0.stat_CatchupInMBUFDiscards: 0 dev.bce.0.stat_CatchupInFTQDiscards: 0 dev.bce.0.stat_CatchupInRuleCheckerDiscards: 0 dev.bce.0.stat_IfInRuleCheckerP4Hit: 10301 dev.bce.0.stat_IfInMBUFDiscards: 0 dev.bce.0.stat_IfInFTQDiscards: 0 dev.bce.0.stat_IfInRuleCheckerDiscards: 0 dev.bce.0.stat_IfInFramesL2FilterDiscards: 5741 dev.bce.0.stat_XoffStateEntered: 0 dev.bce.0.stat_MacControlFramesReceived: 0 dev.bce.0.stat_FlowControlDone: 0 dev.bce.0.stat_OutXoffSent: 0 dev.bce.0.stat_OutXonSent: 0 dev.bce.0.stat_XoffPauseFramesReceived: 0 dev.bce.0.stat_XonPauseFramesReceived: 0 dev.bce.0.stat_EtherStatsPktsTx1523Octetsto9022Octets: 0 dev.bce.0.stat_EtherStatsPktsTx1024Octetsto1522Octets: 7461404 dev.bce.0.stat_EtherStatsPktsTx512Octetsto1023Octets: 488 dev.bce.0.stat_EtherStatsPktsTx256Octetsto511Octets: 1331 dev.bce.0.stat_EtherStatsPktsTx128Octetsto255Octets: 139674 dev.bce.0.stat_EtherStatsPktsTx65Octetsto127Octets: 1492188 dev.bce.0.stat_EtherStatsPktsTx64Octets: 325 dev.bce.0.stat_EtherStatsPktsRx1523Octetsto9022Octets: 0 dev.bce.0.stat_EtherStatsPktsRx1024Octetsto1522Octets: 2890915 dev.bce.0.stat_EtherStatsPktsRx512Octetsto1023Octets: 1494 dev.bce.0.stat_EtherStatsPktsRx256Octetsto511Octets: 167073 dev.bce.0.stat_EtherStatsPktsRx128Octetsto255Octets: 4091 dev.bce.0.stat_EtherStatsPktsRx65Octetsto127Octets: 7602388 dev.bce.0.stat_EtherStatsPktsRx64Octets: 3913 dev.bce.0.stat_EtherStatsOversizePkts: 0 dev.bce.0.stat_EtherStatsUndersizePkts: 0 dev.bce.0.stat_EtherStatsJabbers: 0 dev.bce.0.stat_EtherStatsFragments: 0 dev.bce.0.stat_EtherStatsCollisions: 0 dev.bce.0.stat_Dot3StatsLateCollisions: 0 dev.bce.0.stat_Dot3StatsExcessiveCollisions: 0 dev.bce.0.stat_Dot3StatsDeferredTransmissions: 0 dev.bce.0.stat_Dot3StatsMultipleCollisionFrames: 0 dev.bce.0.stat_Dot3StatsSingleCollisionFrames: 0 dev.bce.0.stat_Dot3StatsAlignmentErrors: 0 dev.bce.0.stat_Dot3StatsFCSErrors: 0 dev.bce.0.stat_Dot3StatsCarrierSenseErrors: 0 dev.bce.0.stat_emac_tx_stat_dot3statsinternalmactransmiterrors: 0 dev.bce.0.stat_IfHCOutBroadcastPkts: 694 dev.bce.0.stat_IfHCOutMulticastPkts: 290 dev.bce.0.stat_IfHCOutUcastPkts: 9094426 dev.bce.0.stat_IfHCInBroadcastPkts: 3746 dev.bce.0.stat_IfHCInMulticastPkts: 6555 dev.bce.0.stat_IfHCInUcastPkts: 10659572 dev.bce.0.stat_IfHCOutBadOctets: 0 dev.bce.0.stat_IfHCOutOctets: 11458190364 dev.bce.0.stat_IfHCInBadOctets: 2384121 dev.bce.0.stat_IfHcInOctets: 5141824959 dev.bce.0.unexpected_attention_count: 0 dev.bce.0.dma_map_addr_tx_failed_count: 0 dev.bce.0.dma_map_addr_rx_failed_count: 0 dev.bce.0.mbuf_frag_count: 0 dev.bce.0.mbuf_alloc_failed_count: 0 dev.bce.0.l2fhdr_error_count: 0 dev.bce.0.%parent: pci1 dev.bce.0.%pnpinfo: vendor=0x14e4 device=0x1639 subvendor=0x1028 subdevice=0x0235 class=0x020000 dev.bce.0.%location: slot=0 function=0 dbsf=pci0:1:0:0 dev.bce.0.%driver: bce dev.bce.0.%desc: QLogic NetXtreme II BCM5709 1000Base-T (C0) ⌂78% [[hidden email]:~] $ We'll see over the next few hours. Thanks again. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: [hidden email] US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
On 06/03/2019 2:36 pm, Larry Rosenman wrote:
> On 06/03/2019 2:23 pm, Eugene Grosbein wrote: >> 04.06.2019 1:42, Larry Rosenman wrote: >> >>> Ok, I've removed the bridge (don't need it). Added the tunable >>> and will check it after a few hours. I *REALLY* appreciate >>> the help diagnosing this. (If need be I can replace the cable, but >>> let's >>> see what happens now). >> >> You already run mrtg, so make sure you have graphs for bce0 usage. >> Maybe you have short-lived traffic storms. > > I upped the RX pages as well, and that seems(!) to help with having > all 3 of my network backups > running (bacula) which apparently was what was tripping it up. > Definitely happier: [[hidden email]:~] 1 $ sysctl -d dev.bce.0.com_no_buffers dev.bce.0.com_no_buffers: Valid packets received but no RX buffers available [[hidden email]:~] $ [[hidden email]:~] $ sysctl dev.bce.0.com_no_buffers dev.bce.0.com_no_buffers: 0 ⌂63% [[hidden email]:~] $ netstat -iadh Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop lo0 16K <Link#1> lo0 2.1M 0 0 2.1M 0 0 0 lo0 - localhost localhost 150k - - 150k - - - ff02::202%lo0 ff01::1%lo0 ff02::2:a7c1:4da6 ff02::2:ffa7:c14d ff02::1%lo0 ff02::1:ff00:1%lo lo0 - fe80::%lo0/64 fe80::1%lo0 0 - - 0 - - - ff02::202%lo0 ff01::1%lo0 ff02::2:a7c1:4da6 ff02::2:ffa7:c14d ff02::1%lo0 ff02::1:ff00:1%lo lo0 - your-net localhost 6.3k - - 2.0M - - - 224.0.0.1 bce0 1.5K <Link#2> a4:ba:db:29:66:95 118M 0 0 92M 0 0 0 bce0 - 192.168.200.0 borg 53M - - 26M - - - bce0 - fe80::%bce0/6 fe80::a6ba:dbff:f 418 - - 428 - - - bce0 - 2600:1700:210 2600:1700:210:b18 68M - - 66M - - - bce1* 1.5K <Link#3> a4:ba:db:29:66:97 0 0 0 0 0 0 0 bce2* 1.5K <Link#4> a4:ba:db:29:66:99 0 0 0 0 0 0 0 bce3* 1.5K <Link#5> a4:ba:db:29:66:9b 0 0 0 0 0 0 0 ⌂62% [[hidden email]:~] $ ⌂71% [[hidden email]:~] $ sysctl hw.bce hw.bce.rx_ticks: 18 hw.bce.rx_ticks_int: 18 hw.bce.rx_quick_cons_trip: 6 hw.bce.rx_quick_cons_trip_int: 6 hw.bce.tx_ticks: 80 hw.bce.tx_ticks_int: 80 hw.bce.tx_quick_cons_trip: 20 hw.bce.tx_quick_cons_trip_int: 20 hw.bce.strict_rx_mtu: 0 hw.bce.hdr_split: 1 hw.bce.tx_pages: 8 hw.bce.rx_pages: 8 hw.bce.msi_enable: 1 hw.bce.tso_enable: 1 hw.bce.verbose: 1 ⌂74% [[hidden email]:~] -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: [hidden email] US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 _______________________________________________ [hidden email] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[hidden email]" |
Free forum by Nabble | Edit this page |