KeyWords: SysAdmin, NFS

NFS performance and the async export option

This morning I moved a large directory (Python 2.5.4 sources unpacked, 4200 files and directories for a total size of ~84M) from local disk on wn59 to my home directory, which is NFS-mounted. Just a simple:

  mv /tmp/Python-2.5.4 ~/

It took ~5 minutes.

This spawned an effort to understand where the performance bottleneck could possibly be, which kept Sergio and me busy until lunch.

Googling for NFS optimization tips, we tried every combination of client-side options: tcp=/=udp transport, increase and decrease rsize and wsize, NFS v2 and v3... to no avail: no significant change in performance. Eventually we found out that:

• Copying with scp was way faster: ~30 seconds for the whole tree.
• Just reading data from the NFS partition was very fast (cp     ~/Python-2.5.4 /tmp took <10 seconds, the expected performance) but moving it (mv ~/Python-2.5.4 ~/, which implies unlinking inodes, i.e., a write operation) was slow.

Bang! 8 seconds for writing the whole tree. Faster than the local disk!

So, the Linux man page says, regarding the async option:

This option allows the NFS server to violate the NFS protocol and reply to requests before any changes made by that request have been committed to stable storage (e.g. disc drive).

Therefore, since sync NFS does not affect read performance, and most of the traffic on /experiment-software is disk reads, with a few occasional writes by the sgm jobs, we decided to trade speed for safety and keep the sync option on the /experiment-software filesystem.

So, the async option is now on in the /home filesystem, and all other filesystems are still exported sync.

Readers' comments