Swiss WLCG Operations Meeting on 2012-11-07

• Date and time: Wednesday 7th of November, at 14:00
• Place: EVO, password: chipp
• External link / EVO: http://evo.caltech.edu/evoNext/koala.jnlp?meeting=MvM2Ml2a2sDMDs9B98De9v

Agenda

• CSCS (reports Pablo):
  • Storage upgrade arrived. IBM DCS3700 x 6 boxes (60 x 3TB each), expected to provide 1.5 GB/s each (9 GB/s total). Still waiting for the IO servers to arrive, to install dCache and test performance.
  • Also waiting for Virtualization hardware to arrive (2 boxes with SandyBridge, 64 GB RAM, SSD drives, and 10 GbE cards). Will try RHEV 3.1 on them, or go back to Convirture.
  • Last maintenance
    • All nodes reinstalled with UMD1
    • Multipath fixed and gridftp door enabled on se[01-08]
  • Next maintenance
    • dCache 1.9.12 upgrade. Xrootd redirector via VOBOX.
    • Re-cable some ethernet and IB cables.
    • Move some compute nodes to different rack, for power consumption limits.
  • Atlasvobox still pending for reinstall (after virtualization is finished)
  • New power consumption monitoring graph
• PSI (reports Fabio):
  • SW: Found useful to use tcptrack to measure live the dCaps bandwidth usage and plan the storage upgrade; basically we're fine with the 2*10Gbit/s links we have today.
  • HW: In 2013, if we'll get the funds, we've decided to double the space of our SGI IS5500, so raising from 360TB raw to 720TB raw => ~500TB net ( RAID6 + global hot spares ) .
  • HW: In front of SGI IS5500 we're using 2 HP DL380 G7 that have room for 6+6 additional 1TB 2.5" SAS disks; we're going to buy these 12 disks to implement 2 read-only dCache pools because sometimes the batch/interactive jobs wait to much to access the same file.
  • HW: Insane amount of 1TB Seagate disks changed inside our Thors
  • SW: migrated t3bdii from SL5 gLite 3.2 to SL6 UMD2
  • SW: migrated t3cmsvobox from Phedex 3.1 to Phedex 4.1, also relocated from old HW to VMWare VM. During these days Daniel will upgrade the Phedex server @ CSCS.
  • SW: Nov 29-30 we're going to migrate PNFS to Chimera; perhaps also to 1.9.12. Follows the migrations plan:

    TODAY	Nov 29-30, PLAN A	Nov 29-30, PLAN A + 1.9.12-22 upgrade
    t3se01, SL4, 1.9.5-29, old HW	t3se01, SL6, 1.9.5-30, VMWare VM	t3se01, SL6, 1.9.12-22, VMWare VM
    t3dcachedb01, SL4, 1.9.5-29, PNFS, PG 8.2, old HW	t3dcachedb04, SL6, 1.9.5-30, Chimera, PG 8.4, VMWare VM	t3dcachedb04, SL6, 1.9.12-22, Chimera, PG 8.4, VMWare VM
    t3fs[13,14], SL6 pools and doors, 1.9.5-29	t3fs[13,14], SL6 pools and doors, 1.9.5-30	t3fs[13,14], SL6 pools and doors, 1.9.12-22
    t3fs[1-4,7-11], Solaris pools and doors, 1.9.5-29	t3fs[1-4,7-11], Solaris pools and doors, 1.9.5-30	t3fs[1-4,7-11], Solaris pools and doors, 1.9.12-22

  • SW: After the Chimera migration we'll be able to use my Nagios quota check for dCache; that needs min PG 8.4, but PG 8.4 is also an 1.9.12 requirement, so CSCS might use that SQL code after their 1.9.12 migration. Anyhow I'll report our production experiences before to encourage its usage in an other site.
• UNIBE (reports Gianfranco):
  • Accounting to central EGI portal in place. Historical records published too, but only back to Jan 2011. Will open a new ticket to ask for going further back
  • Pledged resources to ATLAS for 2013–14 as T2: 5k HEPSPEC06, 350TB for ATLAS-DATADISK
  • PhaseC SunBlades from CSCS commissioning ongoing: ~25% of nodes installed now (customised for ATLAS and also MPI for local users). Some delays due to resolving some ROCKS idiosyncrasies)
  • New CE with ARC 2.0.0 built (also a ROCKS 5.5 Front-End), Infiniband for LAN, 10GbE for WAN (arc01.lhep.unibe.ch, not tested, not in GOCDB yet)
  • Lustre MDS installed (SLC5.7 with Infiniband)
  • Starting on Lustre OSSs (thumpers)
  • KVM/Convirture server: progress still pending
  • Upgrade of DPM-mysql, DPM-disk,bdii-site to1UMD2 still pending: in downtime now (ongoing)
• Switch (reports Alessandro):
  • There is a problem with Nagios Configurator. All A/R figures will be adjusted accordingly.
  • DPM collaboration is about to start, to continue its support after EMI expires.

Other topics

• Meeting has been extended. Name, date, access rights, have to be discussed. Is the current format ok?
• EVO stops being free by the end of 2012. Start using Vidyo?
  • Fabio: for me Vidyo + our IRC #lcg chat is ok

Next meeting date: 10th of January 2013

Attendants

• CSCS: Pablo
• CMS: Fabio, Daniel
• ATLAS: Gianfranco
• LHCb:
• EGI: Alessandro

Action items

• Fabio will tell to CSCS if PSI wants the old Thors
• Pablo will report ( maybe a short Wiki page? ) the CSCS experiences about read-only dCache pools.

• DCS3700-PerfGuide_v33c-Paden.pdf: IBM_DCS3700_Presentation