<!-- keep this as a security measure: * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.LCGAdminGroup * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.LCGAdminGroup #uncomment this if you want the page only be viewable by the internal people #* Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.LCGAdminGroup --> KeyWords: SysAdmin, [[NFS]] ---+ NFS performance and the =async= export option This morning I moved a large directory (Python 2.5.4 sources unpacked, 4200 files and directories for a total size of ~84M) from local disk on =wn59= to my home directory, which is NFS-mounted. Just a simple: <verbatim> mv /tmp/Python-2.5.4 ~/ </verbatim> It took ~5 _minutes_. This spawned an effort to understand where the performance bottleneck could possibly be, which kept Sergio and me busy until lunch. Googling for NFS optimization tips, we tried every combination of client-side options: =tcp=/=udp= transport, increase and decrease =rsize= and =wsize=, NFS v2 and v3... to no avail: no significant change in performance. Eventually we found out that: * Copying with =scp= was _way_ faster: ~30 seconds for the whole tree. * Just _reading_ data from the NFS partition was very fast (=cp ~/Python-2.5.4 /tmp= took <10 seconds, the expected performance) but _moving it_ (=mv ~/Python-2.5.4 ~/=, which implies unlinking inodes, i.e., a write operation) was slow. So we tried the =async= _export_ option on the NFS _server_. Bang! 8 seconds for writing the whole tree. Faster than the local disk! So, the Linux man page says, regarding the =async= option: <blockquote> This option allows the NFS server to violate the NFS protocol and reply to requests before any changes made by that request have been committed to stable storage (e.g. disc drive). </blockquote> Therefore, since =sync= NFS does not affect _read_ performance, and most of the traffic on =/experiment-software= is disk _reads_, with a few occasional writes by the =sgm= jobs, we decided to trade speed for safety and keep the =sync= option on the =/experiment-software= filesystem. So, the =async= option is now on in the =/home= filesystem, and all other filesystems are still exported =sync=. ---++ Readers' comments %COMMENT{type="below"}%
This topic: LCGTier2
>
WebHome
>
PhoenixClusterBlog
>
PhoenixBlog20090227x1342
Topic revision: r1 - 2009-02-27 - RiccardoMurri
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback