<!-- keep this as a security measure: #uncomment if the subject should only be modifiable by the listed groups * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup #uncomment this if you want the page only be viewable by the listed groups # ######* Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup --> ---+!! Node Type: %CALC{"$SUBSTITUTE(%TOPIC%,NodeType,)"}% ---++!! Firewall requirements | *local port* | *open to* | *reason* | <!-- Example line #| 22/tcp | * | Example entry for ssh | --> ---------------- %TOC{title="Table of contents"}% ---+ Regular Maintenance work *Note that there are many checks performed by [[https://t3nagios.psi.ch/nagios/cgi-bin/status.cgi?navbarsearch=1&host=t3cmsvobox][t3nagios]]* ---++ Dataset cleaning This task must be done regularly (once every 2 months, for example), both for CSCS and PSI. *Getting the datasets list* Connect to t3cmsvobox as root and: <verbatim> su - phedex cd svn-sandbox/phedex/DB-query-tools/ ./ListSiteDataInfo.pl -w -t --db ~/config/DBParam.PSI:Prod/PSI -s "%CSCS%" | grep "eleted" ./ListSiteDataInfo.pl -w -t --db ~/config/DBParam.PSI:Prod/PSI -s "%CSCS%" | grep -vE "Dutta|Fanfani|Kress|Magini|Wuerthwein|Belforte|Spinoso|Ajit|DataOps|eleted|StoreResults|Argiro|Klute" ./ListSiteDataInfo.pl -w -t --db ~/config/DBParam.PSI:Prod/PSI -s "%PSI%" </verbatim> The *first* PERL command creates a list of datasets that can be safely deleted from CSCS, as they are just support requests for transfers to PSI (check that the transfer happened safely). The *second* command creates a list avoiding to include central requests, and the ones that can be deleted from CSCS. The *third* command produces a list for PSI. Datasets which are proposed for deletion are all the datasets which have an expired retention time. *Publishing the list and notify users* Due date for feedback is usually in a week. Lists must be published in [https://wiki.chipp.ch/twiki/bin/view/CmsTier3/DataSetCleaningQuery] (previous lists must be deleted). To get the information on the total size proposed for deletion, you can create a temporary text file with pasted list from the twiki and then do: <verbatim> cat tmp.list | awk 'BEGIN{sum=0}{sum+=$4}END{print sum/1024.}' </verbatim> This will give the total size in TB. A email like this must be sent to the cms-tier3-users@lists.psi.ch mlist: <verbatim> Subject: Dataset deletion proposal and request for User Data cleaning - Due date: 28 Oct 2011, 9:0 Dear all, a new cleaning campaign is needed, both at CSCS and PSI. You can find the list and the instructions on how to request to keep the data here: https://twiki.cscs.ch/twiki/bin/view/CmsTier3/DataSetCleaningQuery The data contained in the lists amount to 47TB (44TB) for CSCS (PSI). If you need to store a dataset both at CSCS and at PSI please also reply to this e-mail explaining why. Please remember to clean up your user folder at CSCS regularly; a usage overview can be found at [1]. Thanks, Daniel [1] http://ganglia.lcg.cscs.ch/ganglia/cms_sespace.txt </verbatim> ---++ Renew myproxy certificate for !PhEDEx transfers (once per month) *Nagios checks the [[https://t3nagios.psi.ch/nagios/cgi-bin/extinfo.cgi?type=2&host=t3cmsvobox02&service=CMS+VOMS+proxy+age][voms proxy lifetime]] used by PhEDEx; so far this proxy is a Fabio CMS proxy and so all the PhEDEx files uploaded in* =/pnfs/psi.ch/cms/trivcat/store/data= belong to him. The present myproxy servers have problems with host certificates for PSI from SWITCH, because they contain a "(PSI)" substring, and the parentheses are not correctly escaped in the regexp matching of the myproxy code. Therefore, the renewer DN (-R argument to myproxy-init below) and the _allowed renewers policy on the myproxy server_ need to be defined with wildcards to enable the matching to succeed. <pre> voms-proxy-init -voms cms myproxyserver=myproxy.cern.ch <strike>servicecert="/DC=com/DC=quovadisglobal/DC=grid/DC=switch/DC=hosts/C=CH/ST=Aargau/L=Villigen/O=Paul-Scherrer-Institut (PSI)/OU=AIT/CN=t3cmsvobox.psi.ch"</strike> servicecert='*/CN=t3cmsvobox.psi.ch' myproxy-init -s $myproxyserver -l psi_phedex -x \ -R "$servicecert" -c 720 scp ~/.x509up_u$(id -u) phedex@t3ui01:gridcert/proxy.cert # for testing, you can try myproxy-info -s $myproxyserver -l psi_phedex </pre> As the phedex user do <pre> chmod 600 ~/gridcert/proxy.cert </pre> You should test whether the renewal of the certificate works for the phedex user: unset X509_USER_PROXY # make sure that the service credentials from ~/.globus are used! <pre> voms-proxy-init # initializes the service proxy cert that is allowed to retrieve the user cert myproxyserver=myproxy.cern.ch myproxy-get-delegation -s $myproxyserver -v -l psi_phedex \ -a /home/phedex/gridcert/proxy.cert -o /tmp/gagatest export X509_USER_PROXY=/tmp/gagatest srm-get-metadata srm://t3se01.psi.ch:8443/srm/managerv1?SFN=/pnfs/psi.ch/cms rm /tmp/gagatest </pre> ---++ Storage Consistency Checks From time to time the transfer team will ask for input for their storage consistency check (so far only for T2); we need to complete the following steps: * make sure PhEDEx is updated to the latest version and the CVS config is committed to CVS * ask CSCS admins for a storage dump <pre>python chimera-dump.py -s /pnfs/lcg.cscs.ch/cms -c fulldump -g -o /tmp/outfile</pre> * convert the file using: <pre> sed -e 's#/pnfs/lcg.cscs.ch/cms/trivcat/store/\(mc\|data\|generator\|results\|hidata\|himc\|lumi\|relval\)/#/store/\1/#' \\ -e '/<entry name="\/pnfs\/lcg.cscs.ch\/cms\/.*<\/entry>/d' \\ -e 's#<dCache:location>.*</dCache:location>##' \\ outfile.xml | uniq > storagedump.xml </pre> * compress, store on AFS, and send path to transfer team * take the file you get back from the transfer team with the LFNs to be deleted <pre> for LFN in $(cat SCC_Nov2012_CSCS_LFNsToBeRemoved.txt); do lcg-del -b -D srmv2 -l srm://storage01.lcg.cscs.ch:8443/srm/managerv2?SFN=/pnfs/lcg.cscs.ch/cms/trivcat/$LFN; done </pre> ---+ Emergency Measures <!-- #List any measures that must be taken in case of some major incident, e.g. whether a mailing #list must be contacted or whether other services need to be shut down, etc. --> ---+ Installation Please look the Puppet file =t3vobox=. <!-- #Comment here on any peculiarities of the installation, e.g. on special packages needed, special setup #procedures which are not obvious --> add the following package to run our custom "accounting"-scripts: <pre>yum install perl-XML-Twig </pre> ---+ Services <!-- #List all the important services, their installation, configuration and how to start and stop them --> ---++ PhEDEx Refer to the description on the [[LCGTier2/CmsVObox][Tier-2 VOBox]]. There is one important difference: While we use FTS channels for the transfers to the Tier-2, we use the SRM backend for transfers to the Tier-3, because we do not have a FTS channel for PSI. This issue is linked to registering PSI as a regular grid site, which until recently was not possible, since we only sport a Grid SE, but no CE. So, there is no fts.map file in the configuration area for the !PhEDEx services. ---+ Backups OS snapshots are nightly taken by PSI VMWare Team ( like Peter Huesser ) + we have LinuxBackupsByLegato to recover a single file.
NodeTypeForm
Hostnames
t3cmsvobox ( t3cmsvobox02 )
Services
PhEDEx
4.1
Hardware
PSI DMZ VMWare cluster
Install Profile
vobox
Guarantee/maintenance until
VMWare PSI Cluster
This topic: CmsTier3
>
WebHome
>
AdminArea
>
CmsVoBox
Topic revision: r16 - 2013-09-18 - FabioMartinelli
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback