Go to
previous page /
next page of CMS site log
25. 09. 2009 Retransfer of missing blocks of Summer09 datasets
List of lost blocks:
lostBlocks_summer09.list: list of lost Summer09 datasets blocks
- h 11:42: Requested first part of lost Summer09 lost blocks, size ~1TB, for testing purposes. Request 62359:
https://cmsweb.cern.ch/phedex/prod/Request::View?request=62359&.submit=Submit
- h 12:00: Prod Agents stopped in order to manually clear lost pnsfs, as they are hanging jobs on dCache
- h 14:00 - 26/09/2009 h04:00: transfers happened, 11MB/s from various sources,~12h to be completed
-
[ phedex@cmsvobox ~]$ /home/phedex/PHEDEX/Utilities/InspectPhedexLog -es "2009-09-25 12:00:00" -t "2009-09-26 06:00:00" download.1
given starttime 2009-09-25 10:00:00
given endtime 2009-09-26 04:00:00
==============
ERROR ANALYSIS
==============
Data base Errors
==================
Expired tasks
==================
Total: 5
Error message statistics per site:
===================================
*** ERRORS from T1_IT_CNAF_Buffer:***
1 SOURCE error during TRANSFER_PREPARATION phase: [REQUEST_TIMEOUT] failed to prepare source file in 300 seconds
*** ERRORS from T1_ES_PIC_Buffer:***
7 DESTINATION error during TRANSFER_PREPARATION phase: [REQUEST_TIMEOUT] failed to prepare Destination file in 180 seconds
4 DESTINATION error during TRANSFER_PREPARATION phase: [USER_ERROR] failed to contact on remote SRM [httpg://storage01.lcg.cscs.ch:8443/srm/managerv2]. Givin' up after 3 tries
*** ERRORS from T1_TW_ASGC_Buffer:***
158 SOURCE error during PREPARATION phase: [GENERAL_FAILURE] Misconfiguration: No default service classes defined
4 transfer expired in the PhEDEx download agent queue after [hours] h
*** ERRORS from T1_US_FNAL_Buffer:***
21 TRANSFER error during TRANSFER phase: [GRIDFTP_ERROR] an end-of-file was reached globus_xio: An end of file occurred (possibly the destination disk is full)
20 Could not submit to FTS ~~
SITE STATISTICS:
==================
first entry: 2009-09-25 11:52:57 last entry: 2009-09-26 03:36:36
T1_ES_PIC_Buffer (OK: 191 Err: 11 Exp: 0 Canc: 0 Lost: 0) succ.: 94.6 % total: 189.5 GB ( 3.3 MB/s)
T1_FR_CCIN2P3_Buffer (OK: 1 Err: 0 Exp: 0 Canc: 0 Lost: 0) succ.: 100.0 % total: 1.0 GB ( 0.0 MB/s)
T1_IT_CNAF_Buffer (OK: 148 Err: 1 Exp: 0 Canc: 0 Lost: 0) succ.: 99.3 % total: 145.7 GB ( 2.6 MB/s)
T1_TW_ASGC_Buffer (OK: 315 Err: 158 Exp: 4 Canc: 0 Lost: 0) succ.: 66.6 % total: 311.9 GB ( 5.5 MB/s)
T1_US_FNAL_Buffer (OK: 261 Err: 41 Exp: 0 Canc: 0 Lost: 0) succ.: 86.4 % total: 386.9 GB ( 6.8 MB/s)
TOTAL SUMMARY:
==================
first entry: 2009-09-25 11:52:57 last entry: 2009-09-26 03:36:36
total transferred: 963.8 GB in 15.7 hours
avg. total rate: 17.4 MB/s = 139.5 Mb/s = 1470.8 GB/day
Opened ticket for errors in ASGC:
https://savannah.cern.ch/support/index.php?110207
29. 09. 2009 Retransfer of ALL missing blocks of Summer09 datasets
- h 10:50: subscription of all the remaining blocks
- ~h 11:00:transfers start, after deletions
- h 15:50: subscribed and approved JobRobot lost blocks:
/QCD_Pt_80_120/CMSSW_1_6_7-JobRobot-1201523639/GEN-SIM-DIGI-RECO#a1608ef9-6c0c-46e7-ae32-f4f99af21b39
/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO#1c34ef97-a4ff-4b6c-a5e1-401d3ec3a19c
/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO#272e6c35-435e-4fe9-9108-980761414eb8
/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO#8fcc5e95-583b-4658-8436-5d7addb27de6
/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO#9a26703c-acce-453b-a900-5c9782826831
/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO#a081486b-0296-46b4-9ca7-b1aba278407a
/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO#bca6223c-f060-4037-a36b-5761a8dec983
/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO#c15f9d45-da8a-46cd-af9f-b8873530711a
/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO#f477553f-2b34-4ca7-9659-663e59c80610
/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO#fe385fa5-5c51-4361-937f-ab164cfc7167
- 30/09/2009, h 09:45: approved AnalysisOps requests 63359 (req6) 63419 (req 9) and 64459 (afterglow). Set to High Priority
Go to
previous page /
next page of CMS site log
--
LeonardoSala - 2009-09-25