TransferExercises < LCGTier2

Tags: view all tags

<!-- keep this as a security measure:
   * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.LCGAdminGroup
   * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.LCGAdminGroup
   #uncomment this if you want the page only be viewable by the internal people
   * #Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.LCGAdminGroup
-->

---+ !!Transfer Exercises

%TOC%

---++ Preparations
   1. Log into your account at =ui.lcg.cscs.ch=
   1. Get these two command reference files: [[%ATTACHURL%/srm-gridftp-reference.txt][srm-gridftp-reference.txt]] and [[%ATTACHURL%/fts-reference.txt][fts-reference.txt]] and take a look at them.
   1. Fetch and source the [[%ATTACHURL%/SEenv.sh][SEenv.sh]] file. It contains useful settings for working with our and FZK's SEs.
   1. Create some test files, e.g. by using the pseudo random numbers from =/dev/urandom=: <verbatim>
dd if=/dev/urandom of=mytestfile.dat count=100
</verbatim>

---++ WAN Transfers tools
---+++ !GridFTP
   1. Try to list the contents of our dCache's =/pnfs/lcg.cscs.ch/cms/local_tests/= directory
   1. Identify a smaller and a really big file (> 2GB).
   1. Copy the small file to your work area with =globus-url-copy=. Do it a second time and use the =-dbg= flag
   1. Copy the large file using =globus-url-copy=. Try to monitor the transfer and analyze the connectivity schema by using
      * [[http://storage01-lcg.projects.cscs.ch:2288/][dCache GUI]]
      * strace: =strace -fe trace=network globus-url-copy -p 10 srcfile tgtfile=
      * netstat: on a second console do a =netstat -ap= and try to identify your connections, _globus port range_ ports (usually 20000-25000) and the _ephemeral_ ports (usually ports > 30000).
      * our ganglia pages for [[http://mon-lcg.projects.cscs.ch/ganglia/?r=hour&c=PHOENIX-services&h=se02-lcg.projects.cscs.ch][se02-lcg]] and [[http://mon-lcg.projects.cscs.ch/ganglia/?r=hour&c=PHOENIX-services&h=se03-lcg.projects.cscs.ch][se03-lcg]] and [[http://mon-lcg.projects.cscs.ch/ganglia/?c=PHOENIX-services&h=ui-lcg.projects.cscs.ch&m=&r=hour&s=descending&hc=4][ui-lcg]]
   1. Copy the large file back to the SE using =globus-url-copy=. Again, try to use the above tools to monitor the transfers.
   1. Erase the file with =glite-gridftp-rm=

---+++ SRM
   1. Use =srmcp= to get the same files to your local area. Use the =-debug= flag to get an idea of the monitoring possibilities.
   1. Use =srmcp= to write a file to the SE. Notice that it will create any missing directories
   1. Use =srmcp= to get a file from FZK to our SE. A list of FZK files is here:[[%ATTACHURL%/FZKfiles.lst][FZKfiles.lst]]. The SRM directory at FZK is =srm://gridka-dCache.fzk.de:8443/srm/managerv1?SFN=/pnfs/gridka.de/cms/disk-only/phedex_monarctest/FZK-DISK2=
   1. Erase the file *on our SE* using =srm-advisory-delete=

---+++ FTS
   1. Place a =myproxy= certificate at the  *myproxy-fts.cern.ch* myproxy server and certify that it exists.
   1. Try to get a delegated certificate from the server
   1. Copy one of the FZK files to our SE using the FTS service. Try to monitor the transfer using =glite-transfer-list= and =glite-transfer-status=
   1. Try to copy several files in one job from FZK to SE using a copyjob file. You may use the [[%ATTACHURL%/generateTransferList.sh][generateTransferList.sh]] script to produce a copyjob file.

---++ LAN Transfers
---+++ dcap through dccp command
Refer to DcacheTestDescriptions

---+++ dcap through ROOT
Refer to DcacheTestDescriptions to find out how to access a file through the ROOT environment.

-- Main.DerekFeichtinger - 10 Jun 2007

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
lst	FZKfiles.lst	r1	manage	3.9 K	2007-06-10 - 19:46	DerekFeichtinger	List of FZK test transfer files
sh	SEenv.sh	r1	manage	0.9 K	2007-06-10 - 19:53	DerekFeichtinger	useful environment settings to work with our SE
txt	fts-reference.txt	r1	manage	4.5 K	2007-06-10 - 21:15	DerekFeichtinger	FTS command reference
sh	generateTransferList.sh	r1	manage	2.4 K	2007-06-10 - 19:47	DerekFeichtinger
txt	srm-gridftp-reference.txt	r1	manage	1.5 K	2007-06-10 - 21:15	DerekFeichtinger	SRM / GridFTP commands reference