Problem
Mar 12 09:14:00 gateway kernel: NET: 697 messages suppressed.
Mar 12 09:14:00 gateway kernel: Neighbour table overflow.
Mar 12 09:14:05 gateway kernel: NET: 660 messages suppressed.
Mar 12 09:14:05 gateway kernel: Neighbour table overflow.
Mar 12 09:14:10 gateway kernel: NET: 682 messages suppressed.
Mar 12 09:14:10 gateway kernel: Neighbour table overflow.
Mar 12 09:14:15 gateway kernel: NET: 700 messages suppressed.
Mar 12 09:14:15 gateway kernel: Neighbour table overflow.
The same problem can sometimes manifest as a
No buffer space available error when using ping or arping against many hosts sequentially.
Solution
In my very large flat network, I have the following in /etc/sysctl.conf:
net.ipv4.neigh.default.gc_thresh1 = 4096
net.ipv4.neigh.default.gc_thresh2 = 8192
net.ipv4.neigh.default.gc_thresh3 = 8192
net.ipv4.neigh.default.base_reachable_time = 86400
net.ipv4.neigh.default.gc_stale_time = 86400
These are documented in arp(7).
You can also whack the running values:
# echo 4096 > /proc/sys/net/ipv4/neigh/default/gc_thresh1
# echo 8192 > /proc/sys/net/ipv4/neigh/default/gc_thresh2
# echo 8192 > /proc/sys/net/ipv4/neigh/default/gc_thresh3