Disable IPv6
(2014-09-16)
Problem
Things are slow. In our case, attempting to mount a previously-umounted filesystem would result in a five to seven second hang.
Solution
In /etc/sysconfig/network, add the line
NETWORKING_IPV6=no
See also below.
Disable IPv6 everywhere.
CentOS 6 FAQ: How do I disable IPv6?
Upstream employee Daniel Walsh recommends not disabling the ipv6 module, as that can cause issues with SELinux and other components, but adding the following to /etc/sysctl.conf:
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
To disable in the running system:
echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6
echo 1 > /proc/sys/net/ipv6/conf/default/disable_ipv6
or
sysctl -w net.ipv6.conf.all.disable_ipv6=1
sysctl -w net.ipv6.conf.default.disable_ipv6=1
Additional note #1: If problems with X forwarding are encountered on systems with IPv6 disabled, edit /etc/ssh/sshd_config and make either of the following changes:
(1) Change the line
#AddressFamily any
to
AddressFamily inet
(inet is ipv4 only; inet6 is ipv6 only)
or
(2) Remove the hash mark (#) in front of the line
#ListenAddress 0.0.0.0
Then restart ssh.
Additional note #2: If problems with starting postfix are encountered on systems with IPv6 disabled, either
(1) edit /etc/postfix/main.cf and comment out the localhost part of the config and use ipv4 loopback.
#inet_interfaces = localhost
inet_interfaces = 127.0.0.1
or
(2) take out the ipv6 localhost from /etc/hosts .
Additional Note #3 : To disable RPCBIND ipv6 (rpcbind, rpc.mountd, prc.statd) remark out the udp6 and tcp6 lines in /etc/netconfig:
udp tpi_clts v inet udp - -
tcp tpi_cots_ord v inet tcp - -
#udp6 tpi_clts v inet6 udp - -
#tcp6 tpi_cots_ord v inet6 tcp - -
rawip tpi_raw - inet - - -
local tpi_cots_ord - loopback - - -
unix tpi_cots_ord - loopback - - -
Wait, what?
Yeah, I know what you are thinking. "Why disable IPv6 just because nfs mounts are slow?"
Because when we did tcpdumps on the affected nodes during the waits, we found a DNS conversation like this:
- node: what's the A record for the nfs server?
- DNS: X.
- node: what's the AAAA record for the nfs server?
- DNS: no such number.
- node: what's the (subtly different way of requesting the) AAAA record for the nfs server?
- DNS: <silence for 7 seconds>
- node: what's the A record for the nfs server?
- DNS: X.
- node: hey nfs server...
Now in our case the DNS server is a Windows AD server, so the "correct" fix is probably "fix the AD server" but I'm a linux guy. Turning off IPv6 entirely means that the node never bothers asking for AAAA records, which means the node never has to wait through the DNS server not answering it.
The first solution mooted was to populate the NIS hosts table, and while this made mounts faster, the hangs would still come back periodically.