MeetingSwissGridOperations20170608 < LCGTier2

Tags: view all tags
&lt;!-- keep this as a security measure:<br /> * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.LCGAdminGroup,Main.EgiGroup<br /> * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.LCGAdminGroup<br /> #uncomment this if you want the page only be viewable by the internal people<br /> #* Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.LCGAdminGroup,Main.ChippComputingBoardGroup<br />--&gt;

---+ Swiss Grid Operations Meeting on 2017-06-08 at 14:00
   * *Place*: Vidyo (room: Swiss_Grid_Operations_Meeting, extension: 10537598)
   * *External link*: https://vidyoportal.cern.ch/flex.html?roomdirect.html&key=FAEn4zjAba7BqoQ11TGZu66VSDE
   * *Phone gate*: From Switzerland: 0227671400 (portal) + 10537598 (extension) + # (pound sign)
   * *IRC chat*: irc:gridchat.cscs.ch:994#lcg (ask pw via email)
   * *Switch Vidyo SIP IP*: 137.138.248.204
%TOC%

---++ Site status
---+++ CSCS
---++++ HEPiX short report:

TIMETABLE: https://indico.cern.ch/event/595396/timetable/#20170424.detailed

   * CSCS Sitereport:
      * https://indico.cern.ch/event/595396/contributions/2532583/attachments/1447693/2230947/cscs-lcg2_rprt_spring17.pdf
   * A around 120 participants
   * Many sites started using OpenStack
   * CERN Migration from AFS (impacted various sites) Replacement options (CERN Box, BNL Box, IHEP Box, EOS, CVMFS, etc)
   * Many sited reported consolidation and new facility
   * Increasing adoption of CentOS /SL7
   * Migration to Grafana for many sites
   * Increasing adoption of IPv6 dual-stack at many sites
   * Puppet migration from 3 to 4
      * mostly smooth
      * main issue Cron is now integrated in puppet (no more Cron moule is needed)
   * Very nice presentation on elasticsearch at CERN
   * Consolidation and unify CERN IT monitoring
   * Old school logging at Blabit and IN2P3
      * syslog-ng

---++++ Systems:
   * During the maintenance updated and cleaned all VM
   * Some swapping issues, working to track all jobs memory and cpu usage live
   * Working on a new node monitoring dashboard (CPU,Memeory,Swap,etc.. usage) with Grafana and Metricbeat
---++++ Storage:

*dCache*
   * - update from 2.13.50 to 2.13.58 issue
   * - migrated ~1.2PB and decommissioned the IBM DCS3700, deployed new ~1.6PB from NetApp E5600 systems
   * - working with CMS support for spacemon deploy
   * - working on pp dcache cluster (future upgrade)
*GPFS*
   * - new GPFS servers ready, will be deployed online
   * - GPFS new layout in place: Server-Cluster / WN-Cluster / Daint-Cluster

<img alt="newplot_3.png" height="450" src="%ATTACHURL%/newplot_3.png" title="newplot_3.png" width="700" />
---+++ PSI
   * We are currently in an emergency downtime (since 6.6.2017) due to a controller failure on our /home system server. The repair parts arrive this afternoon.
   * [[http://t3mon.psi.ch/ganglia/PSIT3-custom/accounting.txt][Accounting numbers (from scheduler) from last month]]

---+++ UNIBE-LHEP
   * New hardware, partially installed: 480 cores (E5-2630 v4 @ 2.20GHz) and 150 TB of storage<br /><br />
   * <b>HammerCloud status<br />100% UNIBE-LHEP, UNIBE-ID, UNIGE-DPNC, ~70% for CSCS*<br /></b><a href="http://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteviewhistorywithstatistics?columnid=562&view=Shifter%20view#time=720&start_date=&end_date=&use_downtimes=false&merge_colors=false&sites=multiple&clouds=all&site=ANALY_CSCS,ANALY_CSCS-HPC,ANALY_UNIBE-LHEP,ANALY_UNIBE-LHEP-UBELIX,CSCS-LCG2,CSCS-LCG2-HPC,CSCS-LCG2-HPC_MCORE,CSCS-LCG2_MCORE,UNIBE-LHEP,UNIBE-LHEP-UBELIX,UNIBE-LHEP-UBELIX_MCORE,UNIBE-LHEP_CLOUD,UNIBE-LHEP_CLOUD_MCORE,UNIBE-LHEP_MCORE,UNIGE-DPNC,UNIGE-DPNC_MCORE" target="_blank">http://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteviewhistorywithstatistics?columnid=562&view=Shifter%20view#time=720&start_date=&end_date=&use_downtimes=false&merge_colors=false&sites=multiple&clouds=all&site=ANALY_CSCS,ANALY_CSCS-HPC,ANALY_UNIBE-LHEP,ANALY_UNIBE-LHEP-UBELIX,CSCS-LCG2,CSCS-LCG2-HPC,CSCS-LCG2-HPC_MCORE,CSCS-LCG2_MCORE,UNIBE-LHEP,UNIBE-LHEP-UBELIX,UNIBE-LHEP-UBELIX_MCORE,UNIBE-LHEP_CLOUD,UNIBE-LHEP_CLOUD_MCORE,UNIBE-LHEP_MCORE,UNIGE-DPNC,UNIGE-DPNC_MCORE<img alt="" border="0" height="12" src="%PUBURL%/TWiki/TWikiDocGraphics/external-link.gif" width="13" /><br /><br /></a>
   * *Accounting numbers* <span style="background-color: transparent;"><span style="background-color: transparent;"> (from scheduler) from May 2017<br /><br />%EDITTABLE{}%</span></span> <table border="1" cellpadding="0" cellspacing="1"> <tbody> <tr><th>VO</th><th>Job Type</th><th>Produced WC core-hours</th> <td> </td> <td> </td> </tr> <tr> <td>ATLAS</td> <td>Any</td> <td>1034439</td> <td> </td> <td> </td> </tr> <tr> <td>ops</td> <td>Any</td> <td>14</td> <td> </td> <td> </td> </tr> <tr> <td>t2k.org</td> <td>Any</td> <td>0</td> <td> </td> <td> </td> </tr> <tr> <td>uboone</td> <td>Any</td> <td> <p>6795</p> </td> <td> </td> <td><br /><br /></td> </tr> </tbody> </table> <span style="background-color: transparent;"><br /></span>
   * *Accounting numbers from ATLAS dashboard* from last month (core-hours MAY 2017) [1]<br />CSCS / UNIBE: 56.7% / 43.3% (stable)<br /><br />
      * *Efficiency WT ok/fail* [2]<br />CSCS 78% (_was 93%_) - UNIBE 82% (_was 81%_) (_Ubelix getting better_)<br /><br />
      * *CPU/WT efficiency* [3]<br />CSCS 88% (_was 70%_) - UNIBE 83% (_was 81%_)
   * <br />[1] http://dashb-atlas-job.cern.ch/dashboard/request.py/dailysummary#button=cpuconsumption&sites%5B%5D=CSCS-LCG2&sites%5B%5D=UNIBE-LHEP&sitesCat%5B%5D=CH-CHIPP-CSCS&resourcetype=All&sitesSort=2&sitesCatSort=2&start=2017-05-01&end=2017-05-31&timerange=daily&granularity=8+Hours&generic=0&sortby=0&series=30&activities%5B%5D=all<br /><br />[2] http://dashb-atlas-job.cern.ch/dashboard/request.py/dailysummary#button=successfailures&sites%5B%5D=CSCS-LCG2&sites%5B%5D=UNIBE-LHEP&sitesCat%5B%5D=CH-CHIPP-CSCS&resourcetype=All&sitesSort=2&sitesCatSort=2&start=2017-05-01&end=2017-05-31&timerange=daily&granularity=8+Hours&generic=0&sortby=0&series=30&activities%5B%5D=all<br /><br />[3] http://dashb-atlas-job.cern.ch/dashboard/request.py/dailysummary#button=cpuefficiency&sites%5B%5D=CSCS-LCG2&sites%5B%5D=UNIBE-LHEP&sitesCat%5B%5D=CH-CHIPP-CSCS&resourcetype=All&sitesSort=2&sitesCatSort=2&start=2017-05-01&end=2017-05-31&timerange=daily&granularity=8+Hours&generic=0&sortby=0&series=30&activities%5B%5D=all
---+++ UNIBE-ID
   * Deploying singularity un Ubelix. Use case for ATLAS, since the cluster will transition to Centos7 by end of year and this OS is still a bti problematic for ATLAS. They will start soon testing using the homegrown image also used by UNIBE-LHEP for running on SwitchEngines. <p> </p>
---+++ UNIGE
   * Hardware: Upgraded 6 machines (in May 2017)
      * Old machines acquired in 2012, but upgraded in May 2017
      * Added 6x32 (192) cores and 6x96 (576) GB of RAM memory: 1 User Interface and 5 bacth nodes
      * It cooresponds to a ratio of 3 GB of RAM memory per core
   * Operations: May 2017 (CPU effciency):
      * http://dashb-atlas-job.cern.ch/dashboard/request.py/dailysummary#button=cpuefficiency&sites%5B%5D=UNIGE-DPNC&sitesCat%5B%5D=All+Countries&resourcetype=All&sitesSort=3&sitesCatSort=0&start=2017-05-01&end=2017-05-31&timerange=daily&granularity=Daily&generic=0&sortby=10&series=All
      * Currently, not getting ATLAS production jobs: Under investigation
   * Accounting numbers (from scheduler) from last month:
      * April: https://wiki.chipp.ch/twiki/pub/LCGTier2/MeetingSwissGridOperations20170608/g07.201704.log
      * May: https://wiki.chipp.ch/twiki/pub/LCGTier2/MeetingSwissGridOperations20170608/g07.201705.log <p> </p>
---+++ NGI_CH
   * Xxx
   * NGI-CH Open Tickets review <p> </p>
---++ Other topics
   * Topic1
   * Topic2

Next meeting date:

---++ A.O.B.

---++ Attendants
   * CSCS:
   * CMS:
   * ATLAS:
   * LHCb:
   * EGI: <p> </p>
---++ Action items
   * Item1
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who
png	newplot_3.png	r1	manage	29.7 K	2017-06-08 - 10:10	DinoConciatore
log	g07.201704.log	r1	manage	1.2 K	2017-06-08 - 12:19	LuisMarch
log	g07.201705.log	r1	manage	1.2 K	2017-06-08 - 12:19	LuisMarch