ServiceChallenges < LCGTier2

Tags: view all tags
<!-- keep this as a security measure:
   * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.LCGAdminGroup
   * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.LCGAdminGroup
   #uncomment this if you want the page only be viewable by the internal people
   * #Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.LCGAdminGroup
-->

---+!! Service Challenge Information

%TOC%

---++ Site logs for Phoenix
   * for [[CMSSiteLog][CMS]]
   * for [[SC4SiteLogATLAS][ATLAS]]

---++ CMS October Exercise 2009


---++ STEP09
   * [[http://indico.cern.ch/conferenceTimeTable.py?confId=56580][STEP09 Post Mortem workshop]]
   * [[https://twiki.cern.ch/twiki/bin/view/LCG/WLCGBaselineVersions][Baseline services versions]] for STEP09
      * dCache-1.9.2-6, but we should be able to live with the present system...
      * lcg-CE    3.1.25-0
      * WN    3.1.15-0
      * UI    3.1.30-0
      * ARC server    0.6

---+++ ATLAS / STEP09

   * Coordinator: Graeme Stewart, graeme.andrew.stewart@cernSPAMNOT.ch
   * Dates: May 25-31 Setup, June 1-14 Running.
   * Tier-2s will participate in MC simulation and in user analysis components of the challenge. User analysis in the EGEE clouds will happen through both WMS job submission and through pilot jobs. To facilitate this Tier-2s should configure their batch systems to accept the /atlas/Role=pilot VOMS role and set up the following shares:

| *Activity* | *VOMS Role* | *ATLAS Batch System Share* |
| Production | /atlas/Role=production | 50% |
| Analysis (pilot based) | /atlas/Role=pilot | 25% |
| Analysis (WMS submission) | /atlas | 25% |

   * Since we have both ARC and gLite front ends, they should both feed these shares.
   * [[http://gangarobot.cern.ch/][ganga job robot reports]]. This activity is led by Johannes Elmsheuser. Some of his jobs caused massive I/O problems through multiple dccp staging commands. This has been debugged by Riccardo, Fotis and me (added by Derek).
   * [[http://indico.cern.ch/contributionDisplay.py?contribId=28&sessionId=6&confId=56580][Post mortem talk by Dan van der Ster]]: CSCS-LCG2 was among the best sites (3rd place, page 11) in relation as measured by normalized throughput 
 
---+++ CMS / STEP09

   * [[https://twiki.cern.ch/twiki/bin/view/CMS/Step09][CMS STEP09 page on the central wiki]]
   * Coordinators: Oliver Gutsche, Daniele Bonacorsi
   * [[https://twiki.cern.ch/twiki/bin/view/CMS/Step09PostMortem][Post Mortem page centrally collecting findings]]
   * [[https://twiki.cern.ch/twiki/bin/view/CMS/Step09Operations][Daily activity logs (central wiki)]]
   * Tests scheduled for our Tier2
      * *June 1-26* [[https://twiki.cern.ch/twiki/bin/view/CMS/Step09AnalysisMetrics][T2 analysis exercise]]: We pledged 340 job cores for the current cluster. *50% of these (170 cores) should be available for analysis*. CMS will test whether this amount is achieved.
      * *June 4-16* [[https://twiki.cern.ch/twiki/bin/view/CMS/Step09Transfers#T1-T2_Transfer_tests][T1-T2 transfer exercise]]: latency to destination T2s, latency mss to buffer at T1s (T1 staging for transfer)
         * Phase 1: up to 10TB from FZK. *June 4 - 9*.
      * *I will track the performance of our site on* CMSSiteLog14
   * Info from meetings, etc.
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/Step09Meeting090528Minutes][Minutes from 2nd preparation meeting 2009-05-28]]
      * From a mail by Christoph Wissing: _There are no particular requirements to participate (except of fulfilling the commisioning matrix). Since many sites might upgrade to SL5 during STEP09, I asked if there is coordination needed. The clear answer from DataOps was that Tier2s can upgrade when the Tier2 wants to do it even during the STEP09 weeks._
      * [[%ATTACHURL%/CMS-STEP09-prelimin.pdf][CMS STEP09 preliminary planning (slides)]] from the [[http://indico.cern.ch/sessionDisplay.py?sessionId=5&slotId=0&confId=51957#2009-04-21][CMS Offline workshop in San Diego]]. Especially interesting: page 20 on the massive analysis exercices in May/June.

--------------------------------------------------------

---++ CCRC08
   * [[http://indico.cern.ch/conferenceTimeTable.py?confId=23563][CCRC08 Post Mortem Workshop]]
   * Central [[https://twiki.cern.ch/twiki/bin/view/LCG/WLCGCommonComputingReadinessChallenges][CCRC08 Information page]] for all experiments
   * [[https://twiki.cern.ch/twiki/bin/view/LCG/CCRC08Calendar][CCRC08 Calendar]]
   * central [[https://prod-grid-logger.cern.ch/elog/CCRC'08+Logbook/][CCRC08 Logbook]]
   * [[http://indico.cern.ch/conferenceDisplay.py?confId=29170][Talks]] from the CCRC08 !F2F meeting on March 4th 2008

---+++ ATLAS CCRC08

The Full Dress Rehearsal is part of CCRC08. This means that a realistic data sample according to ~1 day of data taking is distributed
according to the computing model. Phoenix (Sigve) has asked for our share. If a site is not ready with srmv2 and the space tokens,
centrally bulk deletion will not be used. Instead a manual deletion by cloud manager is performed.

   * [[https://twiki.cern.ch/twiki/bin/view/Atlas/Ccrc0801DDM][ATLAS CCRC08]] Information page
   * [[http://www.nikhef.nl/~bosk/presentations/2008/ATLAS-Presentation-for-FDR-and-CCRC.pdf][Recent talk]] by Kors.
   * [[https://twiki.cern.ch/twiki/bin/view/Atlas/SpaceTokens][Space tokens]] for ATLAS. Only ATLASDATADISK and ATLASMCDISK seem to be needed for CCRC08. For instance "token:ATLASDATADISK:srm://srm.grid.sara.nl:8443/srm/managerv2?SFN=/pnfs/grid.sara.nl/data/atlas/atlasdatadisk/" Only the production role should have write access. All tokens are of type "T0D1". ATLASDATADISK should be 2TB. ATLASMCDISK 10TB. *NB* Numbers for CCRC08. They will change. 
   * [[http://atlas.web.cern.ch/Atlas/GROUPS/DATABASE/project/ddm/releases/TiersOfATLASCache.py][TiersOfATLAS]] shows srm endpoints with space tokens for other sites. 

---+++ CMS CCRC08
   * CCRC08 Post Mortem [[http://indico.cern.ch/contributionDisplay.py?contribId=1&confId=23563][Talk by D. Bonacorsi]]
   * Our local page about [[CMSSiteLog11][CCRC08 at CSCS]]
   * Information
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/CCRC08-AnalysisAtTier2][Tier-2 analysis tests]] descriptions and requirements
         * [[https://hypernews.cern.ch/HyperNews/CMS/get/t2/123.html][description]] of the jobs that will be run on the Tier2s
         * [[https://twiki.cern.ch/twiki/bin/view/CMS/CCRC08-AnalysisAtTier2PhysicsGroup][Simulated analysis group testing]]
         * [[https://twiki.cern.ch/twiki/bin/view/CMS/CCRC08-AnalysisAtTier2PeopleAndAccessPolicy][people]] running the analysis and needing user storage access
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/CCRC08-PhaseI-TestTransfersOperations][PhaseI test transfers]] in February
      * Stefano Belforte's useful [[http://belforte.home.cern.ch/belforte/misc/CommissionedLinksStatus.html][status page of commissioned links]]
   * Tasks
      * Specify [[https://twiki.cern.ch/twiki/bin/view/CMS/CMSPledgeT2_CH_CSCS][analysis group pledges for CSCS]] %Y% 
      * Facilities Operations [[https://twiki.cern.ch/twiki/bin/view/CMS/CCRC08-PhaseI-TestTransfersOperations-StartUp][transfer check list]] %Y% 
      * Transfer commissioning:  new DDT2 metric: 20 MB/s over 24h (1.7TB)  for links starting from a T1, 5 MB/s over 24h (425 GB) for links starting from a T2 ([[https://hypernews.cern.ch/HyperNews/CMS/get/ddt-tf/585.html][hn message]])
         * [[https://hypernews.cern.ch/HyperNews/CMS/get/ddt-tf/627.html][hn message]] about steps to be taken by src./dest. sites %Y% 
         * [[https://twiki.cern.ch/twiki/bin/view/CMS/DDTLinkExercising][DDT link exercising]] information page
         * [[https://twiki.cern.ch/twiki/bin/view/CMS/DDTCommonProblems][DDT common problems page]]
      * Implement CMS [[https://twiki.cern.ch/twiki/bin/view/CMS/DMWMPG_Namespace][namespace policy]], e.g. for user space, temporary space %Y% (trivially solved. We do not make restrictions yet)
      * Get fully SRM2 compliant ([[https://twiki.cern.ch/twiki/bin/view/CMS/CMS-SRMv2-spacetokens][CMS instructions]]), also includes information on space tokens (but not needed for CMS T2) %Y% 



---++ CMS CSA07
   * Informational pages
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/CSA07][CSA07 central CERN twiki main page]]
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/CSA07Plan][CMS CSA07 planning page]]
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/Tier2Activities][CMS Tier2 activities]] (forum, contacts)
      * [[https://hypernews.cern.ch/HyperNews/CMS/get/skims/3.html][Msg about skims to be moved to CSCS]]. Note: We will probably only have a total of 22 TB available on our SE!
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideSkimCoordination][central skim coordination]]
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/ProdOps#Resources_in_use][central production information]]
      * [[http://jobrobot.web.cern.ch/JobRobot/cgi-bin/make_graph.py][Job Robot]] statistics page
   * Tasks for us to complete
      * [[CSA07SkimsForCSCS][Selection of Skims]] to be moved to CSCS
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/CSA07CheckListT2][CSA07 site checklist]]
      * [[http://io.hep.kbfi.ee:8889/app/][transfer link commissioning]]  ([[https://twiki.cern.ch/twiki/bin/view/CMS/DDTLinkStateChangeProcedure][procedure]])
   * Interesting links
      * [[http://hepuser.ucsd.edu/twiki/bin/view/UCSDTier2/DataTransferLatency][UCSD log]] on their data transfer latencies (Also see [[https://hypernews.cern.ch/HyperNews/CMS/get/t2/36/1/1/1/1.html][this HN message]]).

---++ Service Challenge 4 and CSA06 information

Service Challenge 4 (SC4) had begun on June 1st 2006 and ended in September. CMS is continuing
with the associated integration activities as long as they keep being useful.
During October and November of 2006 the [[https://twiki.cern.ch/twiki/bin/view/CMS/CSA06][Computing, Software, and Analysis challenge (CSA06)]]
is running.

---+++ Links to external information

   * [[https://twiki.cern.ch/twiki/bin/view/CMS/CSA06][CMS CSA06]] Computing, Software, and Analysis challenge
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/CheckListCSA06][CSA06 Site Checklist]]
   * [[https://twiki.cern.ch/twiki/bin/view/LCG/LCGServiceChallenges][LCG SC4 wiki]]
   * [[https://twiki.cern.ch/twiki/bin/view/CMS/SWIntegration][CMS SC4 pages]]
      * required [[https://twiki.cern.ch/twiki/bin/view/CMS/SWIntSC4Site][Site preparations]]
      * [[https://twiki.cern.ch/twiki/bin/view/CMS/SWIntSC4SiteStatus][SC4 site status]]
      * [[http://pcardabg.cern.ch:8080/dashboard/phedex][20 MB/s transfer status]] based on !LoadTests
      * [[http://cms-project-phedex.web.cern.ch/cms-project-phedex/cgi-bin/browser?page=qplot&db=sc&plot=l48h&entity=link&filter=CSCS&upto=][PhEDEx transfer quality plots]] for CSCS
      * [[http://indico.cern.ch/getFile.py/access?resId=9&materialId=slides&confId=1975][Talk by I. Fisk]], June 21st, [[http://indico.cern.ch/materialDisplay.py?materialId=slides&amp;confId=1975][SC4 Technical Meeting]]
   * ATLAS SC4 pages
      * [[https://twiki.cern.ch/twiki/bin/view/Atlas/ATLASServiceChallenges][ATLAS SC4 overview]]
      * [[https://twiki.cern.ch/twiki/bin/view/Atlas/GridKaCloud][ATLAS GridKA cloud (tier-2 centers)]]
      * [[https://uimon.cern.ch/twiki/bin/view/Atlas/DDMSc4][ATLAS DDM log]]
   * [[http://gridview.cern.ch/GRIDVIEW/][GridView GridFTP log]] (need to check why CSCS does not figure there)

-- Main.DerekFeichtinger - 18 Sep 2007