MeetingSwissGridOperations20210708 < LCGTier2

<!-- keep this as a security measure:
* Set ALLOWTOPICCHANGE = TWikiAdminGroup,Main.LCGAdminGroup,Main.EgiGroup
* Set ALLOWTOPICRENAME = TWikiAdminGroup,Main.LCGAdminGroup
#uncomment this if you want the page only be viewable by the internal people
#* Set ALLOWTOPICVIEW = TWikiAdminGroup,Main.LCGAdminGroup,Main.ChippComputingBoardGroup
-->

---+ Swiss Grid Operations Meeting on 2021-07-08 at 14:00
   * *Zoom link:*
      * https://ethz.zoom.us/j/63018843316
*Next meeting:* 5 August 2021

---

<br />%TOC%

*Minutes:*

CSCS reorganisation:

- new WLCG platform integration lead by Miguel - replacing Nick’s at operations

- more details/ charts etc.. on the organisation will be posted by Pablo

ATLAS:

- Investigate the discrepancy in storage (1904 vs. 1896 vs. 2100 TB)
   * Rucio dump, we have all details, we need to verify them with Dario
   * maybe a problem with the token update ?  Derek seemed to confirm so
- DPM:
   * add all disks on the new server, install dpm stack, then transfer the jobs to the new server and then drop the old one
   * timescale for new disk installation by CSCS ~ next 4 to 8 weeks
   * ATLAS DPM work can begin following this
   * ATLAS would need 180-200 TB of new disk to be set aside for this change (similar to the current size).
LHCb:

- _GGUS need to be addressed faster (after 4 days the evidence of the problem is gone)_

- some cores got lost (dip in LHCb monitoring) afterwards discrepancy between #jobs seen by LHCb (~2500) and CSCS (~3000)
   * to be checked probably fixed over w/e by an ARC restart
CMS:

- in waiting room / SAM tests failing

_- need someone once a day to check GGUS ticket_

- CMS dcache fix implemented with Elia
   * CSCS needs to validate the solution
CSCS:

- all experiments exceeded the pledged

- DPM discussion --&gt; see ATLAS

Pledges:

- discuss the final numbers with Pablo the last week of July

- CHIPP agreed on a “scenario 6” in Pablos xls, meaning:
   * one boost on disk
   * compensated by CPU
   * precise numbers being finalized

AFTER THE SUMMER BREAK

Brainstorm meeting for:

- T2 requirements to run on ALPS

- Possible move of the PSI-T3 to ALPS (users access management)

---++ Followup from previous Action Items

[[ActionItems][Action items]]
---+++ ATLAS
   * [[%ATTACHURL%/ATLAST2ReportJun2021.pdf][ATLAST2ReportJun2021.pdf]]: ATLAS CH Tier2 report
---+++ CMS
---+++ LHCb

---++ T2 Sites reports
---+++ CSCS
   * [[%ATTACHURL%/CHIPPreportJul2021.pdf][CSCS Site Report]]
---+++ UNIBE
   * steady operation with 2.2x pledge delivery
   * lots of concurrent analysis jobs with thousands of files each caused high pressure on lustre. Decided on a full restart.
---++ T3 Sites reports
---+++ PSI
---+++ UniGe
---++ EGI / WLCG
   * APEL old Message Broker networks switched off on 8th July 2021. The new ARGO Message Service is used now to publish the accounting data. All CEs in CH have been updated to a version that supports AMS: 6.12.0<br /><br />
   * EGI ARGO Availability/Reliability report for NGI_CH os fopr June 2021<br /><br />
   * Issues with national funding to cover the EGI participation fee and the Operation co-ordination role. A discussion took place, no solutions identified yet.
---++ Review of open tickets
| *8 of 8 Tickets* |||||
| *Ticket-ID* | *Type* | *VO* | *Site* | *Priority* | *Resp. Unit* | *Status* | *Last Update* | *Subject* | *Scope* |
|  <a href="https://ggus.eu/index.php?mode=ticket_info&ticket_id=152847" target="_blank">152847 </a>  |  <img alt="" border="0" height="10" src="https://ggus.eu/index.php?mode=download&img=team_ticket.gif" width="38" />  | atlas | CSCS-LCG2 | top priority | NGI_CH | assigned | 2021-07-02 | DE CSCS-LCG2: High number of job faiures  | WLCG |
|  <a href="https://ggus.eu/index.php?mode=ticket_info&ticket_id=152819" target="_blank">152819 </a>  | | cms | CSCS-LCG2 | urgent | NGI_CH | assigned | 2021-07-07 | SAM tests for CE failing at T2_CH_CSCS  | WLCG |
|  <a href="https://ggus.eu/index.php?mode=ticket_info&ticket_id=152624" target="_blank">152624 </a>  |  <img alt="" border="0" height="10" src="https://ggus.eu/index.php?mode=download&img=team_ticket.gif" width="38" />  | lhcb | CSCS-LCG2 | urgent | NGI_CH | in progress | 2021-06-29 | Pilots Failed at CSCS-LCG2  | WLCG |
|  <a href="https://ggus.eu/index.php?mode=ticket_info&ticket_id=152070" target="_blank">152070 </a>  | | cms | CSCS-LCG2 | urgent | NGI_CH <img alt="" border="0" src="https://ggus.eu/index.php?mode=download&img=tri.gif" /> involved | assigned | 2021-06-14 | SAM tests failing at T2_CH_CSCS  | WLCG |
|  <a href="https://ggus.eu/index.php?mode=ticket_info&ticket_id=151997" target="_blank">151997 </a>  | | cms | CSCS-LCG2 | urgent | NGI_CH | assigned | 2021-05-14 | WebDAV protocol deployed (T2_CH_CSCS)  | WLCG |
|  <a href="https://ggus.eu/index.php?mode=ticket_info&ticket_id=151265" target="_blank">151265 </a>  | | cms | CSCS-LCG2 | less urgent | NGI_CH | on hold | 2021-04-09 | Enabling WebDAV on Production ...  | WLCG |
|  <a href="https://ggus.eu/index.php?mode=ticket_info&ticket_id=150373" target="_blank">150373 </a>  | | dune | UNIBE-LHEP | less urgent | NGI_CH | in progress | 2021-07-08 | Enable DUNE queue for CPU and future ...  | EGI |
|  <a href="https://ggus.eu/index.php?mode=ticket_info&ticket_id=144485" target="_blank">144485 </a>  | | none | CSCS-LCG2 | less urgent | NGI_CH <img alt="" border="0" src="https://ggus.eu/index.php?mode=download&img=tri.gif" /> assigned | in progress | 2021-04-14 | Upgrade to recent dCache release  | EGI |

   * Attendants
   * CSCS: Miguel, Nick, Pablo, Elia 
   * CMS: Derek, Mauro
   * ATLAS: Steven
   * LHCb: Roland
   * EGI:

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
pdf	ATLAST2ReportJun2021.pdf	r1	manage	396.5 K	2021-07-08 - 10:17	GianfrancoSciacca	ATLAS CH Tier2 report
pdf	CHIPPreportJul2021.pdf	r1	manage	1418.8 K	2021-07-08 - 07:23	NickCardo	CSCS Site Report
pdf	dcache-cms-access-changes-20210706.pdf	r1	manage	545.6 K	2021-07-08 - 11:28	DerekFeichtinger	dcache authz/authn change proposal
pdf	wlcg-swissops-cms-20210708.pdf	r2 r1	manage	2339.9 K	2021-07-08 - 11:56	DerekFeichtinger	CMS Report

This topic: LCGTier2 > WebHome > MeetingsBoard > MeetingSwissGridOperations20210708
Topic revision: r11 - 2021-07-09 - StevenSchramm