(r63) CMSInfoPagesUsers < LCGTier2

Tags: view all tags
<!-- keep this as a security measure:
   * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.LCGAdminGroup,Main.CMSUserGroup
   * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.LCGAdminGroup
   #uncomment this if you want the page only be viewable by the internal people
   * #Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.LCGAdminGroup
-->

---+!! Information for CMS Users of the PHOENIX cluster

%TOC%

------------

---+++ Where can I obtain a Grid certificate?

If you have a CERN NICE account, it is best to get a certificate from CERN, just by filling out a form on the new [[https://ca.cern.ch/ca/][CERN CA web site]]. If not you can get a certificate form the Swiss Grid CA which is operated by SWITCH.

For people without CERN accounts:
SWITCH is acting as the Swiss Grid CA (Certification Authority). They are testing this service currently in a pilot phase.
In order to obtain a Grid certificate, got to this page on their site: https://www.switch.ch/grid/certificates/obtain/

Follow all the steps exactly as indicated on this page. Print out the registration document (best from the link in the return mail you receive upon
submission of the web forms, because most entries have allready been filled in there).
<br>%ICON{warning}% You have to give your *full* name exactly as it appears in your passport or ID
<br>%ICON{warning}% Sign the form *only at 6a)*, not at points b) and c)

You have to bring these documents to the responsible security manager of your insitute. He will sign the registration document
and send it on.

Institute responsible security manager:
   * For PSI: Tobias Marx (WHGA/U134)

---++++ How can I get the certificate out of the browser and into my Grid environment?

In the previous step you obtained your _grid certificate_, but actually your grid credentials consist of two parts. A _private key_ which was generated on your machine when you made your certificate request, and the certificate you got from the CA based on the _public key_ that was generated together with your private key. Most CAs (also CERN) issue the certificates through a web interface, and so the private key and the certificate end up in your browser's certificate store. The grid tools need the credentials in the form of two separate files in PEM format, so you will need to export them.  Look at [[GetKeyCertFromBrowser][this page]] for instructions on how to do this.

---+++ How to get access to CMS Grid resources: Authentication/Authorization using *VOMS*

%H% If you have not done so already, *you must register first* at [[https://lcg-voms.cern.ch:8443/vo/cms/vomrs?path=/RootNode][VOMRS-CMS]] (Also look at [[http://cmsdoc.cern.ch/cms/aprom/www/top/CMS_VO.html#registration][this page]]). You will need to have your certificate loaded in the browser, when you visit the page, or the server will refuse your connection (looks like if the server was dead - there will be no helpful error message).

This registration is needed for associating your grid certificate with CMS. Your certificate only confirms your identity, but to get authorized for
using a CMS service, your certificate must get registered with the CMS VOMS system. This system also allows you to request certain roles (similar to belonging to different UNIX groups), although for normal users, the defaults will be all right. After registering you can see the roles that have been assigned to you on the VOMRS server or also directly on the [[https://voms.cern.ch:8443/voms/cms/][CMS VOMS server page]].

Every time you work on the grid you will create a derivative certificate (and matching key) from your Grid certificate with a short live time of ca 12 h : the so called proxy certificate. If this temporary certificate gets stolen on the grid, a malicious user can impersonate you for just that limited time, but your real certificate is still safe.

Use the VOMS =voms-proxy-init= command to obtain a Grid proxy certificate. By this you will get a proxy certificate that not only contains authentication information (_who you are_) but also authorization information (_what you are allowed to do_, role information). To check what roles you are allowed to assume, visit the [[https://lcg-voms.cern.ch:8443/vo/cms/vomrs?path=/RootNode/MemberAction/SetMbrGroupsAndGroupRoles&action=execute][CMS VOMRS server]] (requires that your firewall does not block outgoing port 8443).

Currently, you should use the following command to obain a generic CMS valid proxy
<pre>
voms-proxy-init -voms cms
</pre>

To generate a generic proxy certificate with no roles information:
<pre>
voms-proxy-init
</pre>

To generate a proxy certificate for the production role:
<pre>
voms-proxy-init -voms cms:/cms/Role=production
</pre>

To look at the authorization information of your proxy certificate:
<pre>
voms-proxy-info -all
</pre>

---+++ Where on the SE can I save data produced by my jobs? %N%

Since early 2008 CMS has a regulation on the path to be used for user generated data. This is defined in the [[https://twiki.cern.ch/twiki/bin/view/CMS/DMWMPG_Namespace][CMS namespace policy]].

For the user space at T2_CH_CSCS this translates to (hn_username stands for your official CMS hypernews user name)
   * <pre>%SRM2%/cms/trivcat/store/user/$hn_username/*</pre> for harvested files residing at a "home" institute. This is custodial data for the home site. 
   * <pre>%SRM2%/cms/trivcat/store/temp/user/$hn_username/*</pre> for temporary files hosted on any site's SE. This portion of the namespace is treated as /tmp, e.g. is volatile.

Since CSCS does not feature tape backups of SE space, the files are only protected by a software RAID, and you should migrate your data off, if you want to have it more secure.

Our earlier convention on CSCS user space in <pre>%SRM2%/cms/local/$username</pre> is obsolete and should no longer be used.

</pre>

---+++ How can I get a list of data sets available at our site / on the Grid?

*Definitions*: The CMS _Dataset Bookkeeping System_ ([[https://twiki.cern.ch/twiki/bin/view/CMS/DBS-TDR][DBS]]) is a database and user API that indexes event-data data for the CMS Collaboration.
The _Data Location Service_ ([[https://twiki.cern.ch/twiki/bin/view/CMS/DLS][DLS]]) is a database that records which data sets are available at which sites (i.e. it lists the  of data sets).

*Howto locate data:* There is a discovery web page for both systems at this URL: http://cmsdbs.cern.ch/
A presentation on how to discover data can be found [[http://indico.cern.ch/getFile.py/access?sessionId=5&amp;resId=0&amp;materialId=0&amp;confId=7739][here]].
The direct use of DLS via a client installation is not recommended for now, but it's available on lxplus at CERN.

Information on datasets and their location is also stored in the !PhEDEx data transfer system, but it is only guaranteed to contain information about recently transported data (i.e. there is no guarantee that information will be dropped at some point. DLS is the authoritative source for location information).
Still, via !PhEDEx you can get information on which sets have been ordered by a center and whether they are arriving. Start from this page for our CSCS Tier-2:  
[[http://cmsweb.cern.ch/phedex/prod/Data::Replicas?view=global&rcolumn=Name&nvalue=Node+bytes&rows=interesting&dbs=7&dbs=11&dbs=14&dbs=15&dbs=10&dbs=16&dbs=12&dbs=2&dbs=1&dbs=3&dbs=9&dbs=13&dbs=6&dbs=4&dbs=5&dbs=22&dbs=8&node=27&filter=.*][PhEDEx replica page]] to locate data.

------------

---+++ How can I order official datasets to the CSCS Tier-2?

%STARTINCLUDE%
CMS uses the [[http://%PHEDEXWEBBASE%][PhEDEx]] system for its data management. The site's
local CMS administrator an order a data set for you.
   1. Identify the data by its official name (For current naming conventions, see [[https://twiki.cern.ch/twiki/bin/view/CMS/DBSNamingConvention][this CERN Wiki entry]]) . You can look up the available data sets via the [[http://cmsdbs.cern.ch/][DBS discovery]]. Make sure, that they exist. If a site has not correctly published the data, it cannot be fetched.
   1. Use the !PhEDEx [[https://%PHEDEXWEBBASE%/prod/Request::Create?type=xfer][create transfer request]] page to order the dataset. You need to log in via your _hypenews name/password_ (login link can be found in upper right screen corner ). 
      * *IMPORTANT:* Please indicate in the standard  *YYYY-MM or YYYY-MM-dd format* for how long we should host this sample for you. This we require to make management of the space easier. Example: <pre>
   %RED%retention time: 2009-02-12%ENDCOLOR% </pre> The set will not be automatically erased at that date, but you will be prompted at that time. If you would like to set a retention time longer than 6 months in the future please give a short explanation (either in the comments or via e-mail).
      * In the *DBS:* area, leave the preselected default (except you specifically know what you're doing)
      * Rules are different depending on where you need the data:
         * If you need data *only at PSI*, subscribe it *both to CSCS and PSI and indicate your phys group*. The data at CSCS will be deleted by us after transfer to PSI has been completed
         * If you want data *both at CSCS and PSI*, please make *two distinct subscriptions*
         * Furthermore, *when subscribing data to CSCS, indicate the *"local" group* instead of your phys group (this is for central accounting and space usage), unless you are *official physics groups data manager*
      * In the *destinations* area check the box for the T2_CH_CSCS and/or the T3_CH_PSI center.
      * In the *data items* area post a list of your datasets (one per line) like this: <pre>
   /Bs2MuMu/CMSSW_1_6_7-CSA07-1193558750/RECO
   /Bc2MuMuMuNu/CMSSW_1_6_7-CSA07-1193556423/RECO
   /Bc2JpsiMuNu/CMSSW_1_6_7-CSA07-1193556369/RECO </pre>
      * <strike>In the *Group* area you can indicate on behalf of which group you are making this request. Group data managers can order for their groups. *If you are no group data manager, always label your order with the* =local= *tag.* </strike>
      * For the *other options* you normally want to select _Replica, Growing, Normal Priority_.
   1. The request will be approved/disapproved by the local !PhEDEx data manager of the site. You will receive emails about the updates to your request.
   1. Please understand that *at some point we will start to refuse requests with undefined group or retention time information*, because the lack of this information severely impacts an efficient management of the storage space and will involve all of us in an email ping pong.
%STOPINCLUDE%
   1. The status of the transfers can be seen on the [[https://%PHEDEXWEBBASE%/prod/Data::Replicas?rcolumn=Name&dbs=7&dbs=11&dbs=14&dbs=15&dbs=10&dbs=16&dbs=12&dbs=2&dbs=1&dbs=3&dbs=9&dbs=13&dbs=6&dbs=4&dbs=5&dbs=22&dbs=8&node=27&nvalue=Node+bytes&rows=interesting&view=global][PhEDEx replica page for our site]]

-----------------

---+++ How can the data be accessed? What is the local path?

The generic CMS path (usually something starting with =/store/...=) is not the local path of the file, but almost.
CMS uses a *trivial file catalog* ([[https://twiki.cern.ch/twiki/bin/view/CMS/SWIntTrivial][TFC]]) implementation to define local filenames at a site, based on a rules file: Mostly, the rules just define a site specific prefix to be put in front of the generic CMS path, but they also can be more complex. The rules file is located at a standard location on every site, so it can be found automatically by CMS jobs. So, if you are using official CMS tools like CRAB to generate your jobs you can use the generic CMS names and the software will transparently translate them to site specific names. 

If you need direct access to files on our site, you can obtain the local filenames from the !PhEDEx web service, e.g. by using the following command

<pre>
wget -O- "http://cmsweb.cern.ch/phedex/datasvc/xml/prod/lfn2pfn?node=%RED%T2_CH_CSCS%ENDCOLOR%&protocol=%GREEN%dcap%ENDCOLOR%&lfn=%BLUE%/store/somefile%ENDCOLOR%" 2>/dev/null |sed -e "s/.*pfn='\([^']*\).*/\1\n/"

#Example output
dcap://storage01.lcg.cscs.ch/pnfs/lcg.cscs.ch/cms/trivcat/store/somefile
</pre>

In this way you can query for the correct URL on any CMS gridsite and with any of the supported protocols.


The following protocols and user tools can be used to access files:

---++++ SRMv2 (SRM version 2)
This should be the standard protocol for interactively accessing files.
There is an excellent [[http://www.ci.uchicago.edu/osgedu/schools/gridlab/srm.html][SRM howto guide from OSG]].

Use *srmcp* to move data from and to the SE. The accessible part of the namespace for CMS is under =%SRM2%/cms.=

Example:
<br>For easier readability, we define the variable SRMBASE first
<pre>
SRMBASE=%SRM2%/cms
srmcp -2 ${SRMBASE}/trivcat/store/CSA06/CSA06-102-os-minbias-0/somefile.root  file:////tmp/mytest.tst
srmcp -2 file:////tmp/mytest.tst  ${SRMBASE}/local_tests/mytestfile
</pre>

You can list directory contents and details with the =srmls= command:
<pre>
srmls -l ${SRMBASE}/trivcat/store
</pre>
Note, that the file sizes in the output of a directory listing may not be correct for files larger then 2GB. But using the =-l= flag option to the command will produce the correct size:

<pre>
srmls -l ${SRMBASE}/local_tests/big_file
2467069744 /pnfs/lcg.cscs.ch/cms/local_tests/big_file
</pre>

For deleting files, use the =srmrm= command. You can delete directories with =srmrmdir= which also supports a =-recursive= flag, but the recursive behavior will only remove empty directories for safety reasons. 

<pre>
srmrm ${SRMBASE}/local_tests/testdir/gaga
srmrmdir ${SRMBASE}/local_tests/testdir
</pre>

---++++ SRMv1 (SRM version 1)
You can use *srmcp* to move data from and to the SE. The accessible part of the namespace for CMS is under =%SRM1%/cms.=

Example:
<br>For easier readability, we define the variable SRMBASE first
<pre>
SRMBASE=%SRM1%/cms
srmcp ${SRMBASE}/trivcat/store/CSA06/CSA06-102-os-minbias-0/somefile.root  file:////tmp/mytest.tst
srmcp  file:////tmp/mytest.tst  ${SRMBASE}/local_tests/mytestfile
</pre>
Note that the *file protocol* specification needs *4 slashes* to refer to the local root file system "/". If you only use 3 slashes the file is staged relative to your current directory.

The =srm-get-metadata= command is very useful in getting information about a file or directory, e.g.
<pre>
srm-get-metadata ${SRMBASE}/trivcat/store
                     size :0
                     owner :cmssgm
                     group :cms
                     permMode :16893
                     ...
</pre>

Deleting files: Use the srm-advisory-delete command
<pre>
srm-advisory-delete ${SRMBASE}/local_tests/myfile.dat
</pre>

---++++ Grid FTP
Example:
<pre>
GSIFTPBASE=gsiftp://storage01.lcg.cscs.ch/pnfs/lcg.cscs.ch/cms
globus-url-copy ${GSIFTPBASE}/local_tests/testfile.tst file:/tmp/mytest.tst
globus-url-copy ${GSIFTPBASE}/trivcat/store/CSA06/CSA06-102-os-minbias-0/file.root  file:/tmp/mytest.tst
</pre>

You can use *edg-gridftp-ls* to look at files and directories, e.g.
<pre>
edg-gridftp-ls ${GSIFTPBASE}/trivcat/store
</pre>


---++++ dcap
This protocol can be used either from the local UI or by jobs running at the site, not over the WAN! It is a protocol that usually is limited to on-site machines. There is a command line copy comand =dccp=, but more frequently this is the protocol of choice for direct access of files through ROOT or CMSSW.

Example:
<pre>
DCAPBASE=dcap://storage01.lcg.cscs.ch:22125/pnfs/lcg.cscs.ch/cms
dccp $DCAPBASE/local_tests/mytesttree.dat /tmp/somefile
</pre>
%ICON{warning}% Note: dcap only allows read access to the SE (since this is a authenticationless protocol).


---++++ FTS

The FTS (File Transfer system) implements a queueing system for file transfers. It is useful for moving large data sets around, since it is able to control the bandwidth and other transfer specific parameters on predefined channels. Usage is complex and we do not currently recommend it for normal users. Contact us if you need to transfer lots of data that is not known to !PhEDEx.

<strike>
---++++ Xrootd
 
We offer *read only access* via [[http://xrootd.slac.stanford.edu/ xrootd]] (the Extended Root Daemon). This access method can
only be used from within the site and *does currently not require a grid proxy*. To use it in your jobs, follow these steps:

   * Set up your [[http://root.cern.ch ROOT]] environment 

<pre>
   export ROOTSYS=$VO_CMS_SW_DIR/../sitelocal/ROOT/ROOT_5.06.00
   export PATH=$PATH:$ROOTSYS/bin
   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROOTSYS/lib
</pre>

   * Convert the gsiftp-PFN into a root daemon PFN by changing the protocol to "root://" and *introducing another double slash (!!!)* after the host name: 
<pre>
   gsiftp://host/dir1/dir2/filename ---> root://host//dir1/dir2/filename 
</pre>
   You can use the following sed expression for the conversion (gsiftp TURL is contained in $name)  :-)
<pre>
   echo $name| sed -e 's/gsiftp:\/\/\([^/]*\)/root:\/\/\1\//'
</pre>

   * Use the *xrdcp* command line client to copy whole files. Naturally you can also access files directly from within a root job
<pre>
   xrdcp -s root://storage01.lcg.cscs.ch//storage/cms/local_tests/testfile.tst /tmp/mytest.tst
</pre>
Note: The *-s* flag chooses silent mode (no progress bar) since we use the command not in an interactive session
</strike>

------------

---+++ What CMS software is available at the site?
---++++ CMSSW
The "official" CMSSW software packages are maintained by a central CMS software manager who triggers installation on all sites. The "tags" of the officially available software are entered into the Grid information system used by the LCG software, from where they can be picked up by the framework. On any correctly configured LCG User Interface (UI) you can enter the following command to get information about the installations on all sites.
<pre>
   lcg-infosites --vo cms tag
</pre> 

Jobs find out about the location of the software on a given site by looking up the $VO_CMS_SW_DIR environment variable.

---++++ CRAB - out of date! needs to be revised!
<strike>
There is a CRAB ([[https://twiki.cern.ch/twiki/bin/view/CMS/CRAB][CERN info page]]) installation on the CSCS UI. So, if you have an UI account at CSCS you can use it to submit job ranges. This can be attractive, since you have more local space there as on CERN AFS.
The CRAB installations can be found under =/opt/CRAB/CRAB_[version]=. In order to use it, you need to source  e.g.
<pre>
. /opt/CRAB/CRAB_2_2_0/crab.sh
</pre>
</strike>
------------

---+++ links to external CMS related information
   * How to get [[http://cmsdoc.cern.ch/cms/aprom/www/top/CMS_VO.html][access to LCG as a CMS user]] (also has VOMS information)
   * [[https://hypernews.cern.ch/HyperNews/CMS/cindex CMS hyper news]] replaces the mailing lists and is one of the best sources for up-to-date information
   * Central [[https://twiki.cern.ch/twiki/bin/view/CMS/WebHome CMS twiki]]
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/WorkBook][CMS Workbook]] Documentation for CMSSW and the computing environment
      * [[http://cmsdoc.cern.ch/cms/production/www/html/general/index.html Production pages]]
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/GridServices][List of RB (resource brokers)]], etc. available for CMS users
   * [[http://cmsdoc.cern.ch/cms/cpt/tdr/index.html CMS computing TDR]]

---++++ How to troubleshoot Grid/CRAB jobs
   * [[https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookGridJobDiagnosisTemplate][CMS Workbook Grid Job Diagnosis Template]]


-- Main.DerekFeichtinger - 21 Jan 2008