Tags:
create new tag
view all tags

1000 Recent Changes in CmsTier3 Web retrieved at 06:48 (GMT)

Statistics for CmsTier3 Web Month: Topic views: Topic saves: File uploads: Most popular topic views: Top contributors for topic save...
How to apply for an account at the CMS PSI Tier 3 To apply for a Tier 3 account please send an email to cms tier3@lists.psi.ch with Subject `account request`....
HowTo use EOS from the PSI Tier 3 Currently the local mounting of EOS has problems. But we can use the command line eos utility For authentication EOS accepts either...
Policies for the resource allocation on the PSI Tier 3 These policies were agreed upon in the first and second Steering Board meetings. They need revision in...
Slurm Batch system usage Simple job submission and the concept of queues (partitions) On the Tier 3 you run jobs by submitting them to the Slurm job scheduler. The...
Solutions to Operational Issues daily check of slurm Even though many problems are signalled through mails by Icinga, I also prefer to have a look each morning on...
How to access, set up, and test your account Mailing lists and communication with admins and other users cms tier3 users@lists.psi.ch : list through which we...
List of Monitorings Nagios Notifications on Icinga Server (for rhel7 nodes) Ganglia Report dCache State: interface Interface...
How to retrieve your corrupted or deleted /t3home and /work files Every user can retrieve from snapshots corrupted or deleted files from /t3home or /work...
How to work with Storage Element SE clients Storage data (based on dCache) located under directory /pnfs/psi.ch/cms/trivcat/store/user/username . Data are...
Welcome to the PSI 1 web News List all news T3 User information Request a T3 account Mailing Lists and Support To receive information about...
CMS data ordering by Rucio CMS is now using Rucio for the data management across sites. Even though users can submit own requests by Rucio, we currently need to duplicate...
Understanding Tier 3 storage The Tier 3 offers different kinds of storage and it is important that you understand how to use them to their best advantage. User home...
OBSOLETE INFORMATION basic understanding of the dCache for advanced user (archived pages) The Storage Element ( SE ) t3se01.psi.ch runs dCache, a Grid Storage...
CPU Example #!/bin/bash # #SBATCH p wn #SBATCH account t3 #SBATCH job name job name #SBATCH mem 3000M # memory 3GB (per job) #SBATCH time...
Tier 3 Steering Board Board Members ETHZ Mauro Donega (new CHIPP Computing board chair) Simone Pigazzini...
CMS Tier 3 News Item Summary Details I wish you a happy and productive new year. as every year, PSI is conducting the big compute center test (`Schaltertest`)...
Useful Slurm commands Overview command description sinfo monitor nodes and partitions queue information; check more info options by sinfo...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details These are the first nodes in our cluster based on the AMD EPYC architecture. For the first days we will just run them in a...
CMS Tier 3 News Item Summary Details Please look at these pages Tier3Storage HowToRetrieveBackupFiles
This page is an up to date version of ObtainingHostCertificates Obtaining host certificates for Grid servers Certificates now agaim come from QuoVadis. Their services...
Trust/Link from QuoVadis CA You are invited to use QuoVadis Trust/Link as a Subscriber to issue Grid Host certificates for the ScITS UniBe Account in the following...
ZFS snapshots backup configuration OBSOLETE Note by Derek at cluster takeover: This page contains obsolete information, since for the last months...
Monitoring OBSOLETE This page has been dismantled I leave it active for the moment since it contains some links that still may be interesting Networking and...
Slurm Utilisation useful commands to Monitor Slurm Plots of Slurm Partitions Usage WN and QUICK GPU and QGPU Load of WNs on t3ganglia...
Shutting down the Tier 3 Before Downtime 1. do announcement in : t3 user list: cms tier3 users #64;lists.psi.ch t3 admin wiki COGDB...
CMS Tier 3 News Item Summary Details
May 20 Security updates/measures set `nosuid` flag for shared file systems mount points (/t3home, /work and /pg backup) EGI Trust Anchor release...
Deployment of Storage Accounting EGI APEL dCache accounting done (and puppetized ) on t3se01 according to Quick check of publishing status: http...
Areca Support WebGUI usage steps to allow web access: # /root/Linuxhttp V2.5.1 180529/x86 64/archttp64 ArchHTTP for setup: http://hostname:81/ or directly the...
Grid Host Certificate instruction T3 Admin access registration on Certification Service Provider QuoVadis: done for the following common T3 address...
GPU Example #!/bin/bash # #SBATCH job name test job #SBATCH account gpu gres # to access gpu resources #SBATCH partition gpu...
Monitoring Storage Plots Slurm Utilisation Links to PhEDEx monitoring Link to Admin Monitoring
Slurm GPU Running Jobs Slurm GPU Running vs Pending Jobs Slurm QGPU (quick GPU Partition) Running Jobs Slurm QGPU (quick GPU Partition) Running vs Pending Jobs...
1 Web Create New Topic Index Search Changes Notifications Statistics Preferences User Pages Main Page...
Slurm WN Partition: Running Jobs Slurm WN Partition: Running vs Waiting Jobs Slurm QUICK Partition: Running Jobs Slurm QUICK Partition: Running vs Waiting...
Networking and File Transfers ( PhEDEx) Links: usage at T3 usage at T2 day of FTS3 jobs induced by CRAB3, transfers details day of FTS3 jobs induced...
Comparison of I/O Performance on /t3home, /work and /eos with simple dd benchmark measure of write performance to eos 7.9 MB/s dd if /dev/zero of /eos/home n...
PSI Tier 3 Physics Groups Overview Obsolete information, needs update Note: Each group has a home page on this wiki that can be reached by the link in the Name...
Submitting Multicore/Multithreading job using multiple processors/threads on a single physical computer (SMP parallel job) Your program might require a number of...
Basic job submission and monitoring on the Tier 3 The Tier 3 cluster is running the Sun Grid Engine batch queueing system. You can submit any shell script to the...
CMS Tier 3 News Item Summary Details
Hardware Care Hardware Serial and Warranties Storage Dalco JBOD Serves Support NinaLoktionova 2019 05 08
FW updates on DALCO JBOD servers SN F 18.05.101 104 and F 18.11.176 This is the latest firmware package: https://downloadcenter.intel.com/de/download/28696/Intel...
Dalco Storage servers with JBOD There are two raid controllers installed on t3fs07 10 servers Areca ARC 1883 combined 24 storage drives and MegaRAID...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details
Batch system policies These policies were discussed and endorsed in the steering board meeting of 2011 11 03 Aims The T3 CPUs and Storage resources must be shared...
T3 meeting with ETHZ CMS group NinaLoktionova 2019 12 16
CMS Tier 3 News Item Summary Details
(Derek) Notice that this machine still reboots unexplainedly about once per month. The machine comes up well, but the dcache pools are not started and need manual...
MegaRAID Support To check RAID1 raid controller of the two OS SSDs on t3fs07 11 use StorCLI Standalone Utility: https://downloadcenter.intel.com/download/27654/StorCLI...
CMS Tier 3 News Item Summary Details
2019/Q1 plan 1. new boot/pxe server done 1. home migration deadline Feb 2019 1. slurm (for GPU resources, rhel7) 1. 2d GPU procurement/deployment...
CMS Tier 3 News Item Summary Details
Sun Grid Engine on CMS Tier3 This document describes the planning, installation, and basic configuration of SGE on the PSI Tier3 cluster. The advanced configuration...
CMS Tier 3 News Item Summary Details
PhEDEx PhEDEx is the data transfer service used by CMS to transfer data sets beween sites. Central Monitoring Plots for last 24h of Agents as seen by the...
CMS Tier 3 News Item Summary Details
Thumper local port to switch port mappings and trunking tests Local port to switch port mappings These mappings are needed for correctly configuring the switch...
Burn in Tests The system boots to the Pc Check main menu. Choose `Immediate Burn in Testing option` in the main menu. The Immediate Burn in Testing option...
CMS Tier 3 News Item Summary Details
9th Tier 3 Steering Board meeting Venue When: Friday, 7th Dec 2018, 10:30 13:30 Location: ETH H...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details
Cluster Overview OLD Cluster Overview Cluster Composition and Services Cluster Specs Original Requirements Our requirements document (q.v. our internal CMS Tier...
CmsTier3 Web Preferences The following settings are web preferences of the CmsTier3 web. These preferences overwrite the site level preferences in...
CMS Tier 3 News Item Summary Details
Obtaining host certificates for Grid servers Our host certificates are issued by the SWITCH CA (using the QuoVadis service). Go here for the web interface for the...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details
Node Type: Until today 20th June 2013 t3bdii.psi.ch is currently an alias for t3bdii02.psi.ch that`s a SL6 UMD 2 VM Firewall requirements local port open...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Tier3 Security Update Downtime on 21.07 at 14:00 Summary Dear all, we expect about 2 hours interruption of services. Details
All News DerekFeichtinger 12 May 2009
CMS Tier 3 News Item Summary Details Message from 7. 7. 2017 Dear Users We need to replace a HW component for the central home storage server. The parts are ordered...
Node Type: Firewall requirements local port open to reason Regular Maintenance work contact the system integrator : francois.dallmann #64;dalco...
Node Type: Firewall requirements local port open to reason Regular Maintenance work Emergency Measures HP Support data: Service Agreement...
Node Type: Firewall requirements local port open to reason 2811/tcp gridftp control connection 22125/tcp unauthenticated dcap (read...
8th Tier 3 Steering board meeting Venue When: Friday, 12th May 2017, 14h 16h (can prolong till 17h if needed) Location: ETH H...
CMS Tier 3 News Item Summary Details We would like to carry out this operation on Monday afternoon, 13:30h. It will require a complete shutting down of the storage...
CMS Tier 3 News Item Summary Details Dear Users Since the too frequent taking of snapshots has led time and time again to problems with users not being able to...
Node Type: Firewall requirements local port open to reason Installation Official Doc ( pretty chaotic ) https://twiki.cern.ch/twiki/bin/view/CMSPublic...
Node Type: Admin info on this node type Firewall requirements local port open to reason Regular Maintenance work Nagios Check out our t3nagios...
CMS Tier 3 News Item Summary Details As every year, PSI will perform a power test in the beginning of January 2017, exactly on 07/08. This means all T3 services...
Node Type: Firewall requirements local port open to reason Regular Maintenance work Emergency Measures Installation 10Gbs net The following 10Gbs...
Node Type: Firewall requirements local port open to reason Regular Maintenance work Nagios t3nagios SuperMicro IPMI tools In Band and Out Band...
Node Type: Firewall requirements local port open to reason 20000 25000/tcp Globus port range for gridftp data streams Regular Maintenance...
Go to previous page / next page of Tier3 site log 10. 09. 2013 SGI IS5500SP broken drawer Today we got, again, a broken drawer in the SGI IS5500SP Expansion: Summary...
Node Type: Firewall requirements local port open to reason Regular Maintenance work Nagios UIs on t3nagios User quota on /tmp /scratch ( but...
Node Type: Firewall requirements local port open to reason 2811/tcp gridftp control connection 22125/tcp unauthenticated dcap (read...
Go to previous page / next page of Tier3 site log 26. 12. 2012 t3fs14 reboot on Dec 25th On Dec 25th around 9 a.m. CET there was a Nagios warning about the host...
HP ProLiant DL380 G7 iLO3 Intro The 2 HP DL380 G7 servers t3fs 13,14 can be managed by the specific HP Service Processor called iLO3 that basically is a dedicated...
Node Type: Firewall requirements local port open to reason Emergency Measures Too many frequent or heavy writes You can identify which user files...
CMS Tier 3 Upgrade Planning Page Summary Details t3fs05 emergency OS reinstallation (Solaris 10 u8) In this emergency downtime, the main effort had to be spent...
CMS Tier 3 Upgrade Planning Page Summary All user account management via LDAP Major reconfiguration of the cluter`s user and authentication management (see...
Node Type: Serves home directories and experiment`s software directories to t3wn and t3ui servers. Firewall requirements local port open to reason...
Go to previous page / next page of Tier3 site log 2016 Derek`s handover 1st meeting Make a round in the Data Centre to see the new 10Gbs setup new HPs new...
Searching into /pnfs by dc find To search into /pnfs the T3 users can use dc find with the limitation that the information retrieved will be in the worst case...
CMS Tier 3 News Item Summary Details On 27.10.2016, from 12:30 till 13:30 the /shome file service will be fully stopped and eventually rebooted in order to upgrade...
How to work in a CMS environment Setting up the CMS environment by /cvmfs /cvmfs is an IT technology developed at CERN to easily distribute the CMSSW releases around...
CMS Tier 3 News Item Summary Details The new 2016 UIs have been installed and can be immediately used by the whole T3 community ; they feature much faster HW than...
How to interactively debug jobs on a worker node Important note The queue debug.q allows to debug the running jobs on the worker nodes. Please do not abuse this...
IssueSRMcallsFailOrGetUnresponsive Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2016 06 09 user complaint...
IssueT3NFS01halt Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2016 09 22 Observations 7185554.009100...
IssueT3WN51reboots Symptoms Summary: cut here kernel BUG at mm/truncate.c:489! invalid opcode: 0000 #1 SMP last sysfs file: /sys/devices/system/cpu/online...
CMS Tier 3 News Item Summary Details In order to upgrade both the /pnfs and the /shome file service the T3 will be in scheduled downtime on Fri 23.09....
T3 CH PSI and T2 CH CSCS data sets to be cleaned up Datasets cleaned on Sep 14th, we got back ~40TB between T3 and T2 Please indicate in the leading Keep column...
CMS Tier 3 News Item Summary Details General Network maintenance Slurm upgrade to 16.04 GPFS metadata migration from SSD to flash Compute Nodes...
Go to previous page / next page of Tier3 site log 06. 07. 2016 dCache new SRM limits got enforced Xrootd t3dcachedb03 t3dcachedb03 Domain xrootd...
CMS Tier 3 News Item Summary Details On Friday 1st Jul 2016 from 14:00 till 15:00 the /shome file service is going to be updated ; both the UIs and the WNs will...
Go to previous page / next page of Tier3 site log 15. 06. 2016 Major upgrades performed on t3nfs02 Today I`ve updated on t3nfs02 : Kernel ver. to 3.10....
How to manage jobs with SGE Utilities SGE provides many command line utilities and a GUI program to Interact With the Sun Grid Engine Software. For the information...
Go to previous page / next page of Tier3 site log 09. 06. 2016 dCache 2.15 stuck on t3se01 BE AWARE OF THE LATEST 2.15 Derek`s tools https://github.com/fabiomartinelli...
CMS Tier 3 Upgrade Planning Page Summary Details Since some of our Switches will get SW upgrades, we will also lose all T3 intranet connectivity for a short while...
CMS Tier 3 Upgrade Planning Page Summary Details before downtime: preparation work 1 Dowtime Announcement to users through mailing list and news...
How to run local Tier 3 jobs by CRAB2 If you are running on a dataset, it must be stored locally at PSI Tier 3. CRAB3 is the new CRAB !!! Be aware that CRAB2 isn...
CMS Tier 3 News Item Summary Details The T3 users mailing list portal in now on https://psilists.ethz.ch/sympa/info/cms tier3 users ; all the past user emails...
CMS Tier 3 News Item Summary Details On Friday 6th May 2016 from 10:00 till 17:00 we`re both upgrading the /pnfs file service and installing some 10Gbs cards...
CMS Tier 3 Upgrade Planning Page Summary Details Change t3dcachedb04 by t3dcachedb03 , and t3se02 by t3se01 : root@t3dcachedb04 ~ # psql U postgres...
CMS Tier 3 News Item Summary Details Lately we`ve deployed 9 new t3wn5 1 9 servers featuring CPU cores / 128GB RAM / 10Gbs cards ; they`re already fully exploited...
Go to previous page / next page of Tier3 site log 15. 04. 2016 Old /swshare dirs to be removed 1.1G anaconda 267G cms 62G cms old 786M CRAB 115M DCACHE GUI 24G...
How to run the HEP SPEC2006 CPU benchmark Introduction How To by HEPIX To get an objective measure of the CPU power of a WLCG site the Hepix community has created...
CMS Tier 3 News Item Summary Details Dear T3 users the current T3 shome crashes are becoming too much frequent so we`ve decided to release the New T3 shome in...
Go to previous page / next page of Tier3 site log Problems with myproxy renewal from PSI vobox and CSCS vobox 04. 02. 2016 t3cmsvobox.psi.ch myproxy issue...
6th T3 Steering board pre meeting ( Virtual ) Followup SteerBoardMeeting07 Meeting Facts When : on 2015 10 27 at 15:00 16:00 ) as agreed by http://doodle...
CMS Tier 3 News Item Summary Details From Friday 8th Jan 2016 till Saturday 9th Jan 2016 PSI is going to test its electrical systems and perform several HW upgrades...
2015 12 16 Sixth Steering Board Meeting Preparatory information can be found in the documentation and minutes of the virtual preparation meeting for Steering...
Go to previous page / next page of Tier3 site log 29. 09. 2014 CMS dcaps average MB/s at T3 tcptrack Command on t3fs13 was: tcptrack p r 60 f i...
News Items Form Name Type Size Values Tooltip message Attributes Title text 50 M Summary textarea 250 M User...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details Dear T3 Users from Friday 24/07/2015 @ 16:00 till Monday 27/07/2015 @ 14:00 the T3 will be completely stopped due to...
CMS Tier 3 News Item Summary Details On Friday 11/09/2015 from 13:00 till 20:00 we`re upgrading the /pnfs files service ; accordingly no /pnfs file reads...
CMS Tier 3 News Item Summary Details On Friday 25/09/2015 from 13:30 till 14:30 we`re going to upgrade the /pnfs file service ; accordingly no /pnfs...
IssueCvmfsFailsToMount Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2015 07 01 Observations cvmfs on t3wn...
CMS Tier 3 News Item Summary Details Dear T3 Users next Friday 12/06/2015 from 13:00 till 14:00 the T3 /pnfs file service will be unavailable ; please don`t try...
Node Type: Firewall requirements local port open to reason 8649/udp ganglia collector 8670/udp ganglia collector 8671/udp...
Node Type: Firewall requirements local port open to reason 3401/udp 128.142.0.0/16 , 188.185.0.0/17 CMS central Squid monitoring based on SNMP...
CMS Tier 3 News Item Summary Details SearchingIntoSlashPNFS
Go to previous page / next page of Tier3 site log 03. 05. 2015 Son of Grid Engine 8.1.8 cpuset error and fix Ref: http://linux.oracle.com/documentation/EL6/Red Hat...
CMS Tier 3 News Item Summary Details Dear T3 users due to an extraordinary electrical maintenance at PSI there is a chance to get a power cut today 30th Apr from...
Go to previous page / next page of Tier3 site log 30. 04. 2015 17TB of files read last time before 01 01 2012 1 GB nmohr 3 GB lamm h 9 GB leo 21 GB casal 26 GB bortigno...
Introduction We are exploiting the official PSI backup system `Legato` by installing and configuring the Linux client like WELL described in this How To. Please...
Go to previous page / next page of Tier3 site log 24. 03. 2015 ./check http json.py nagios@t3dcachedb04 nagios $ ./check http json.py H t3dcachedb:2288/info/poolgroups...
CMS Tier 3 News Item Summary Details The current T3 dCache 2.6 is getting unsupported in months so we have to update to dCache 2.10 as several CMS T2s already...
CMS Tier 3 News Item Summary Details The current T3 dCache 2.6 is getting unsupported in months so we have to update to dCache 2.10 as several CMS T2s already...
CMS Tier 3 News Item Summary Details The amount of T3 users still running SL5 code is pretty low ; these T3 users will have to migrate to t3ui05.psi.ch by the...
Go to previous page / next page of Tier3 site log 13. 03. 2015 t3wn 30 40 RAM errors EDCA RAM errors in one server sframe main 156845 : segfault at...
Sherpa Integration Project Technical Documentation Introduction This page documents the use of the T3 for the SherpaNLO validation and integration project. This...
CMS Tier 3 News Item Summary Details From Friday 9th Jan 2015 at 19:00 till Saturday 10th Jan 2015 at 19:00 PSI is testing its electrical systems and performing...
CMS Tier 3 News Item Summary Details During the recent T3 Steering Board meeting held in Dec `14 at ETHZ the T3 Institute representatives agreed about the complete...
CMS Tier 3 News Item Summary Details The T3 admins have deployed /cvmfs, the standard technology used in the CMS Grid to transparently and efficiently distribute...
NOTE: Installations are best done directly on the NFS server t3nfs01. They must be done as user cmssgm!!! CMSSW installation on the Tier 3 Central CMS documentation...
Fourth PSI Tier 3 Steering board meeting Venue Date and time: Monday, Feb 11, 14 16h (Page) Location: ETH Hoenggerberg (HPK D32) Slides: by Fabio Martinelli Meeting...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details Dear T3 users by decommissioning 3ui01 and getting its disks plus recycling some other spare disk we can now increase each...
Go to previous page / next page of Tier3 site log 20. 11. 2014 t3fs06 RPC error Nagios warning Nagios Notification Type: PROBLEM Service: NFS registered...
CMS Tier 3 News Item Summary Details Please read HowToAccessSe#gfalFS
Go to previous page / next page of Tier3 site log 11. 10. 2014 T3 VMs rebooted Hi Daniel ( Webster ) I think you`re already aware, these T3 VMs have been forcefully...
CMS Tier 3 News Item Summary Details The new SL6 UIs providing 1.7TB /scratch can be used according to the following policy Tier3Policies#User interface policies...
CMS Tier 3 News Item Summary Details On Friday 12th Sep `14 from 13:00 until ~19:00 the T3 /pnfs file service will be unavailable or unreliable because we`re...
CMS Tier 3 News Item Summary Details Since 15.08.2014 the T3 batch queues have been extended by four new SL6 WNs t3wn 41,43,44,50 to allow the T3 users to submit...
Installing ROOT ROOT installations are located in /swshare/ROOT , which also contains the init files linked to the latest installed version. Installations are performed...
Custom YUM repositories for the Tier3 Introduction PSI runs a customized version of Linux some local RPMs repositories ( e.g. including IBM GPFS ) ; the T3 uses...
Go to previous page / next page of Tier3 site log 19. 12. 2012 Largest directories via Chimera New version Simply consult the daily report: http://t3mon.psi...
Node Type: Firewall requirements local port open to reason Regular Maintenance work n.a. Emergency Measures If this VM is really causing issues...
dCache documentation snippets from mails, etc. Good external links to documentation and tools FAQ on the official trac wiki notes on dCache ItalianT2Tools...
CMS Tier 3 News Item Summary Details The new gfal CLIs and APIs to interact with the Grid SEs have been deployed at T3.
Go to previous page / next page of Tier3 site log 19. 03. 2014 t3fs05 unresponsive Symptoms The t3fs05 fileserver hosting /swshare became unresponsive during...
Go to previous page / next page of Tier3 site log 04. 03. 2014 t3vmui01 lcg cp 18GB file Not OK / OK GGUS Ticket and solution https://ggus.eu/index.php?mode ticket...
Go to previous page / next page of Tier3 site log 10. 03. 2014 Lost Sensor n.60 on each SUN Thor fileserver For some reason today in parallel we got this error;...
Go to previous page / next page of Tier3 site log 14. 02. 2014 Missing Billing DB and Billing logs entries about t3fs14 files Yesterday I ran https://bitbucket.org...
Working with graphical applications with the NX client application Introduction The NX server/client suite enables you to very efficiently work over the network with...
CMS Tier 3 News Item Summary Details dCache Upgrade Because the several middlewares employed in the Grid are migrating towards a safer usage of the digital certificates...
Go to previous page / next page of Tier3 site log 28. 01. 2014 A Python Pandas Postgresql example http://pastebin.com/enSQd7ip FabioMartinelli 2014 01 28...
Go to previous page / next page of Tier3 site log 14. 01. 2014 dCache 2.6.19 pool (Too many open files) error 13 Jan 2014 21:47:57 (t3fs13 cms 1) door:GFTP t3se...
Go to previous page / next page of Tier3 site log 20. 12. 2013 Installing Puppet 2.7.21 on each T3 Solaris server ver. 10/13 Today I was upgrading puppet to version...
CMS Tier 3 News Item Summary Details As every year, PSI has a forced shutdown of IT services over the first working week weekend in January for the yearly system...
CMS Tier 3 News Item Summary Details The T3 will be either unavailable or unreliable during that time interval; there will be no emails announcing the downtime...
Go to previous page / next page of Tier3 site log 10. 12. 2013 dCache Authentication failed: Certificates does not conform to algorithm constraints In /var/log...
CMS Tier 3 Upgrade Planning Page Summary Details Planned / Status Planned Status UIs UMD3 upgrade UIs latest dCache SRM RPM UIs...
Go to previous page / next page of Tier3 site log 27. 10. 2013 t3fs05 swap full t3fs05.psi.ch console login: t3fs05.psi.ch console login: root Oct 27 15:05:21 t3fs...
CMS Tier 3 News Item Summary Details Today the nodes t3wn 30 40 are limited to 10MB/s in respect of their 100MB/s usual bandwidth, the PSI network team...
Solaris Installations collection of Solaris commands by zagbot We use Solaris 10 as our default. We had tested OpenSolaris installations in 2008 (OpenSolaris...
Go to previous page / next page of Tier3 site log 17. 09. 2013 t3fs07 hangs because of a broken disk bash 4.1# Sep 17 10:53:43 t3fs07 scsi: WARNING: /pci@3c,0/pci10de...
CMS Tier 3 News Item Summary Details On the next Friday 13 09 2013 9:30 am the /pnfs files service are going to be stopped for ~1h to perform a major reboot...
Go to previous page / next page of Tier3 site log 05. 09. 2013 Evacuating t3fs13 cms 1 to get it from 22TB to 17TB t3dcachedb03.psi.ch (t3fs13 cms 1) admin migration...
Go to previous page / next page of Tier3 site log 05. 09. 2013 Solaris t3fs disks catalogue Totals Disk Model Occurrences Total SEAGATE ST31000524NS...
CMS Tier 3 News Item Summary Details To allow to the T3 users to use a recent version of the gcc compilers on SL5 without altering the SL5 system gcc compilers...
Go to previous page / next page of Tier3 site log 04. 09. 2013 Introduced CERN/REDHAT devtools 1.1 Installed on t3ui ,t3wn servers the CERN/REDHAT devtools...
Working on the PSI CMS Tier 3 The PSI Tier 3 encompasses the following services/machines User Interface: This is a fully operational LCG UI. It enables you...
MAY 2013 To get the Solaris broken disks statistics run: root@t3nagios ~ # /usr/local/bin/disks failure statistics.sh Disk problems on the Thumper/Thor Fileservers...
Go to previous page / next page of Tier3 site log 22. 08. 2013 Updated dcap binaries on SL5 UIs and WNs From http://koji.fedoraproject.org/koji/buildinfo?buildID...
Go to previous page / next page of Tier3 site log 22. 08. 2013 Swapping our first broken disk in the E5460 enclosure Broken disk is type HITACHI; NetApp sent us...
Go to previous page / next page of Tier3 site log 21. 08. 2013 t3fs13 dCache pools found unresponsive Today since ~19:30 the t3fs13 were unresponsive, I noticed...
CMS Tier 3 News Item Summary Details On Friday 16th morning we`ll stop the /pnfs file services to quickly introduce the new Storage System; we apologize for...
Remote Repository Backup Admin Software The software consists of custom made python scripts that can be found at https://www.github.com/daniel meister/rrbackup...
Remote Repository Backup This service is only running in beta mode at the moment and does not yet have production level monitoring set up. Purpose Since the shutdown...
CMS Tier 3 Upgrade Planning Page Summary Details It`s needed the Peter presence and his cooperation; VMs to be migrated: t3se01 t3frontier01...
16th July 2013 OUTDATED, BUT KEPT FOR REFERENCE This document describes the experiences we made during the upgrade of the SGE installation from 6.1 to 6.2u5, the...
Collection of useful information about git ToDo for everyone (working on analysis code) 1 create an account on GitHub 1 map your GitHub account with your CERN...
Go to previous page / next page of Tier3 site log 11. 06. 2013 Constantly more than 250 PG DB connections Today I`m trying to understand why there are so many PG...
Go to previous page / next page of Tier3 site log 04. 06. 2013 t3dcachedb03 again frozen This early morning t3dcachedb03 hanged for ~20 mins, SRM recovered, according...
Go to previous page / next page of Tier3 site log 31.05.2013 VOMS Server Issue Problem Initial Trigger The search for this issue was triggered by a user request...
Go to previous page / next page of Tier3 site log 16. 05. 2013 t3dcachedb03 frozen Go to previous page / next page of Tier3 site log t3dcachedb03.frozen.png...
Disk quota on /tmp and /scratch Introduction Please have a look to our usage policies. Note that on t3ui08 Urs wanted /scratch thresholds 98% 99% Puppet module...
CMS Tier 3 News Item Summary Details The Grid middleware used inside the WNs to support your jobs is getting outdated and we have to migrate to the release; to...
CMS Tier 3 News Item Summary Details A new and powerful VMWare cluster has been installed at PSI with 2013 HW plus a recent VMWare installation; as a consequence...
CMS Tier 3 News Item Summary Details We`ll upgrade: dCache to ver 2.2.9 1 Postgresql to ver. 9.2.3 kernels to the latest version. All of these...
Go to previous page / next page of Tier3 site log 04. 03. 2013 SRM Error: Already have 1 record(s) with pnfsPath Got this e mail from a T3 user Dear admins, I am...
T3 Groups Purpose For multiple user to be able to collaborate on an analysis we offer the possibility to create dedicated analysis group folders on the SE in /store...
IssueDcacheTooManySQLConnections Symptoms Summary: SE cannot be accessed through srm. In t3se01:catalina.out: createConnection(): Got exception org.postgresql...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details Details reported in the user SE NFSv3 section.
Go to previous page / next page of Tier3 site log 03. 01. 2013 dCache file deletion File Deletion Test with old file (i.e. short PNFSID) 0002000000000000069DE5B...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details Dear T3 users because I did not get any complain about the new servers t3ui01 t3ui08 t3ui09 then I assume that they`re working...
CMS Tier 3 News Item Summary Details As every year, PSI has a forced shutdown of IT services over the first working week weekend in January. This includes complete...
Complete Upgrade planning and Upgrade logging history Note: The Upgrade planning twiki functionality is based on the UpgradePlanningTemplate and UpgradePlanningForm...
Crab Installation NOTE: PROBABLY IN PART OBSOLETE The installation mostly follows the from the CERN wiki. Take a look there to see whether the installation instructions...
Go to previous page / next page of Tier3 site log 28. 12. 2012 t3fs07,t3fs08 went down bit of theory about ACPI states; from that I understand that S5/G2: soft...
Interacting with the ELOM/ILOM ELOM is the older version of the service processor`s firmware while ILOM is the name of the current versions. reference sheet by...
Go to previous page / next page of Tier3 site log 13. 12. 2012 SMARTd configuration change requested after an Hitachi 1TB swap The 1TB Hitachi disks that we inherited...
CMS Tier 3 News Item Summary Details Main purpose of this downtime is the upgrade of some dCache components that got very old in respect of the others CMS sites...
Go to previous page / next page of Tier3 site log 29. 11. 2012 146GB disks to be absorbed by UIs Unused Servers server unused disks ex t3wn08...
Go to previous page / next page of Tier3 site log 16. 11. 2012 Restarted pnfsd Thanks for the info! M. From: Fabio Martinelli Organization: Paul Scherrer Institut...
Contacts with SUN/ORACLE for Hardware Problems This page is OBSOLETE because apart from our WNs all the SUN HW is out of warranty. To still get statistics about the...
Go to previous page / next page of Tier3 site log 26. 07. 2012 Enforcing flexible memory limits on SGE Change proposal Up to now we didn`t apply memory limits...
Go to previous page / next page of Tier3 site log 30. 07. 2012 Dealing with the fallout from the new memory limits Analyzing maximal memory usage of the last few...
CMS Tier 3 News Item Summary Details To prevent a user to kill a server beacuse a wrong memory usage each Job will get 3GB of RAM by default, details here.
IssueNewWNsKernelPanic Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2012 07 19/22 Observations Out of...
CMS Tier 3 News Item Summary Details We spent the latest month installing, configuring and testing the new 11 Intel servers type S2600JF each equipped with 48GB...
Physic Group page for the: group
Issueeth0DetectedHardwareUnitHang Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): root@t3service01 ~ # grep Detected...
Physic Group page for the: group
Physic Group page for the: group
Title From the old glite 3.1 (?) node /opt/glite/etc/vomses `vo.nedm.cyfronet.pl` `voms.cyf kr.edu.pl` `15007` `/C PL/O GRID/O Cyfronet/CN voms.cyf kr.edu.pl` `vo...
IssueSenseKey Symptoms Summary: root@t3fs13 ~ # grep sdd /var/log/messages Mar 27 16:28:31 t3fs13.psi.ch kernel: sd 5:0:0:3: sdd Sense Key : Recovered Error...
Node Type: Firewall requirements local port open to reason 22/tcp 129.129.194.77/16 ssh 1514/tcp 192.33.123.29/24 syslog ng...
IssueWhiteScreenNXClient Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2012 03 23 Leo and Fronga report about...
Some help on IPMI commands External documentation blog article by Ben Rockwood Sensor readings speeding up commands by first recording status to a cache Commands...
Go to previous page / next page of Tier3 site log 14. 03. 2012 Network tests of the new 10GbE attached file servers for the IS5500 Tests between t3fs13 and t3fs...
Go to previous page / next page of Tier3 site log 22. 02. 2012 Qlogic FW update on t3fs13,14 SGI case about RDAC FW update I`ve found a more updated FW for our...
CMS Tier 3 News Item Summary Details Dear T3 users Major upgrades to the storage and compute infrastructure of the T3 require a complete exchange of the current...
SGI IS5500 / NetApp E5400 HW issues 26 01 2012 Drawer 2 `Lost of redundancy` event We`ve installed the new SGI/NetApp system and suddenly we got an error on the...
CMS Tier 3 News Item Summary Details the T3 policies.
CMS Tier 3 News Item Summary Details
IssueNxSessionStartupFailure Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2011 12 09 Observations NX...
CMS Tier 3 News Item Summary Details New walltime limits for all.q and long.q The ALL.Q will be our queue for MEDIUM LENGTH jobs, and it will have a runtime limit...
PSI Tier 3 Steering board meeting Meeting Date and Location Time and date: Thu, Nov 3rd, 10:30 12:30h (poll) Location: ETH Hoenggerberg, HPK D32 (map) Proposed...
Go to previous page / next page of Tier3 site log 25. 10. 2011 Workernode to fileserver throughput. About 8MB/s per job Go to previous page / next page of Tier...
CMS Tier 3 News Item Summary Details Details on the new policies you can find here.
IssueDcacheCorruptOSM Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2011 06 16 Observations NOTE: We are...
Node Type: Firewall requirements local port open to reason Regular Maintenance work Emergency Measures Installation The Atomicorp RPMs repository...
Go to previous page / next page of Tier3 site log 01. 06. 2010 Implementing the ZFS incremental snapshot backup The naive send/recv approach has terrible performance...
Physic Group page for the group Links to subgroup pages Bs2MuMu
Table with chain files MC chain files Name Chain file configuration files Chain Events Processed Events Lumi B0 mumu...
SGE 6.1 Interactive Queue on t3ce01 It`s useful introduce an queue in the t3ce01 SGE configuration for 2 main purposes; to allow users to develop SW exploiting...
Signup procedure to GOC as a ROC DECH member NOTE: THIS IS OBSOLETE INFORMATION WHICH IS MAINLY KEPT FOR HISTORICAL REASONS. EGI replaces most of these addresses...
Second Steering Board Meeting Information and Slides The meeting takes place on Tue, Feb 22nd, 14 16h original poll Location: ETHZ, Building/Room...
E mails for HW alerts To properly get the Sun ILOM HW e mail alerts we had to configure each ILOM service vs the PSI e mail GW 192.33.120.33 psquad.psi.ch and the...
LSI RAID1 status To monitor the RAID1 status on the servers t3admin01, t3ce01 and t3se01 I installed the utlity mpt status, manually loaded the driver `mptctl`, updated...
CMS Tier 3 Upgrade Planning Page Summary Details Fileserver install status t3fs01 SunOS 5.10 Generic 141445 09 t3fs02 SunOS 5.10 Generic...
Go to previous page / next page of Tier3 site log 15. 01. 2011 Three breakdowns of t3ui01 in 2 days t3ui01 went for the second time into a hanging state. The SP...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details
Physic Group page for the: group
red { background color:#DD2233; } blue { background color:#4433FF; } green { background color:#22AA11; } black { color:#000000; font weight:bold; } section { color...
CMS Tier 3 News Item Summary Details
Table with Candidates in the NTuple Plugin Bplus to J/Psi Kplus Candidate ID Daughters Fit package Description 100521 Kalman Bplus...
Under construction How to run on data with crab Setting up the environment Check out some recent CMSSW version Setup the CRAB environment. Using c shell...
Mass production of ntuples on data and (min bias) MC Release The working releases will change over time. The following have been tested to work. CMSSW 3...
Go to previous page / next page of Tier3 site log 06. 07. 2010 Worker nodes reading from two Thors at about 800 MB/s Some jobs managed to almost get the theoretical...
Analysis of the `Ntuples` This page describes the setup and some tools for analyzing the `ntuples`. Quick and dirty plots directly from the ntuple setenv LD LIBRARY...
Go to previous page / next page of Tier3 site log 28. 06. 2010 Bandwidth measurement between CSCS and PSI Tier 3 Using the following machines: se39.lcg.cscs...
CMS Tier 3 News Item Summary Details
Go to previous page / next page of Tier3 site log 11. 06. 2010 Second breakdown of t3ui01 in 7 days 808 Fri Jun 11 11:51:41 2010 IPMI Log minor...
CMS Tier 3 News Item Summary Details The worker nodes t3wn11 t3wn29 have been upgraded to 24 GB memory per node (3GB per CPU core). The cluster queuing system...
Howto look at files in Manno srmls srm://storage01.lcg.cscs.ch:8443/srm/managerv2\?SFN /pnfs/lcg.cscs.ch/cms/trivcat/store/user/ursl/production/Winter10/...
Links to external documentation on the Tier 3 Hardware Blade 6000 Modular System http://docs.sun.com/app/docs/prod/blade.x6270?l en a view Sun Blade...
Documentation on CMSSW Tested Versions Setup a developers release area to work, just to get you started on t3ui02.psi.ch: cmsrel cd /src cmsenv cvs co r...
Pool Migration External documentation dcache book: http://www.dcache.org/manuals/Book/cookbook/cb pool migration.shtml Example for copying pool t3fs05 cms to...
Go to previous page / next page of Tier3 site log 17. 03. 2010 Pool migration from t3fs05 cms to t3fs07 cms migration commands Actuall, I will carry out a pool...
Go to previous page / next page of Tier3 site log 10. 03. 2010 Testing the SEs We want to test different SE setup, and their behaviour under a real user case. For...
Introduction The production is not based on crab, but on 1 3 scripts. It can be run either on the grid or on the T3 batch system. The output is (by default) stored...
CMS Tier 3 Upgrade Planning Page Summary dCache upgrade to 1.9.5 16 1 stop phedex 1 UI: Prevent user login and reboot to get rid of all logged in users...
CMS Tier 3 News Item Summary Details Dear users We finally were able to solve the problems for the major upgrade of our storage management software in a virtual...
IssueSolarisFileServerPerformance Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2010 03 05 Observations...
CMS Tier 3 Upgrade Planning Page Summary Details Preparation: download dcache software to t3se01:/root/download Make a backup of the postgresql data...
IssueZfsNfsExportAcls Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2010 02 18 Bug report from Andrey Starodumov...
Go to previous page / next page of Tier3 site log 04. 02. 2010 SunBlade X620 Burning tests (wn10 19) Test bed Based on CPT, it is located on the admin node: /home...
Go to previous page / next page of Tier3 site log 28. 01. 2010 Test of data transfer latency at PSI Method : subscribe one by one the following datasets:...
Informations on X4500 and X4540 Solaris X4540 setup use of the flash card Interesting discussion where it is argumented that by using ZFS on the flash card...
Physic Group page for the: group
CMS Tier 3 News Item Summary Details In short: all the worker nodes have been upgraded to SL5 the old SL4 UI on t3ui01 stays the same, so that people...
PSI Tier 3 phase B upgrade slides for ETH meeting on 2010 01 27 (Derek) DerekFeichtinger 2010 01 27
IssueX4500ControllerFailure This issue is treated in the following Sunsolve article: http://sunsolve.sun.com/search/document.do?assetkey 1 66 233341 1 Solaris 10 x...
Issue Tracker Obsolete Items: Note IssueForm IssueTemplate DerekFeichtinger 2009 11 19
IssueSolarisPatchFail1 Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2008 12 19 Observations On two x...
IssueLEDproblems Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2009 12 09 2010 01 06 Observations t3fs...
CMS Tier 3 News Item Summary Details
PSI Tier 3 Ganglia installation The installation follows closely the description of the Tier 2 ganglia installation with slightly modified configuration files. The...
IssueDcachePoolHangs Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2009 12 09 2009 12 10 Observations...
IssueSrmWrongTurl Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2009 05 11 Observations SRM read requests...
CMS Tier 3 News Item Summary Details The downtime was used to reinstall a current Solaris 10 version on t3fs05. We used the downtime furthermore for a number of...
Physic Group page for the: group
Steering Board Meeting 2009 12 01 Meeting Materials: Slides (D. Feichtinger) Agenda 1 Welcome 1 Information about present status of T3...
Physics Group Form Name Type Size Values Tooltip message Attributes Name text 50 official CMS group name M Responsible User...
Physic Group page for the: group
IssueNscdAliasedHostsNotCached Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2009 06 17 Observations The...
IssueDcacheSrmRead2 Symptoms Summary: Case 1 Observations srmcp fails because the clients receives a wrong TURL srmcp debug srm://t3se01.psi.ch:8443/srm/managerv...
Title DerekFeichtinger 2009 11 18
This page is out of date. I will update it soon. If set `scheduler sge` in crab.cfg, get following error. chen z@t3ui01 test $ crab create crab.cfg crab. crab...
CMS T3 CH PSI User Meeting The meeting will take place on Mon, Sep 14 2009, approx. 9:30h An EVO Room in the CMS community will be reserved for external listeners...
CMS Tier 3 News Item Summary Details
CMS Tier 3 News Item Summary Details The details for this upgrade can be seen in UpgradePlanning200905141314
Solaris UFS partitioning and RAID mirroring for the X4500 My first try at getting a Solaris partitioning (most of this is taken from an howto by www.gravity.phy.syr...
Go to previous page / next page of Tier3 site log 29. 06. 2009 This is a log as temporary notes for Zhiling. 29.06.2009 Home was nearly full. Phedex instances...
ILOM upgrade problem of X4150 machines 2009 05 27 Upgrade from Also refer to the last update (FirmwareBiosUpdate) SP firmware 2.0.2.6 SP firmware build number:...
dCache configurator We use a script create PoolConf.pl which generates most of the basic configuration for the Pools. The script can be found under https://svn.cscs...
Go to previous page / next page of Tier3 site log 25. 05. 2009 OS patching and reconfiguration of some X4500s All machines have difficulties in getting new patches...
CMS Tier 3 Upgrade Planning Page Summary Details
Go to previous page / next page of Tier3 site log 25. 02. 2009 Worker Node to Fileserver write throughput We currently have three Thumpers active in dcache. T. Punz...
dCache Upgrade Problem 1.8.12 to 1.9.5 gridFTP transfers work, but on every SRM interaction I find in the /opt/d cache/libexec/apache tomcat 5.5.20/logs/catalina...
IssueDcacheRead Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): 2009 05 07 Observations This happened on a...
CERNLIB installation Get CERNLIB from http://cernlib.web.cern.ch/cernlib/ There exist binary tarballs and primitve RPMS (not based on SRPMs that compile, they just...
CMS Tier 3 News Item Summary Details Filesystem quotas have been enforced to protect the system. Each user may normally use 15GB of /shome directory space. He...
Go to previous page / next page of Tier3 site log 25. 11. 2008 Thumper Fileserver t3fs03 problems Dcache pools on the node were marked as unavailable. A listing...
Go to previous page / next page of Tier3 site log Log: CRAB SGE Task: Merging modified code to CRAB 2.4.1 Create jobs Solved Submit jobs Solved...
CMS Tier 3 News Item Summary Details Dear Swiss CMS colleagues The PSI CMS Tier 3 cluster is now online. This is the common cluster for CMS members of ETHZ, University...
Go to previous page / next page of Tier3 site log 06. 10. 2008 Test user feedback In the week of Oct 6th we are working with test users to get the user environment...
CMS Tier 3 News Item Summary Details Running Jobs Please read the HowToSubmitJobs topic which also gives some details about the status of CRAB jobs. Ordering...
CRAB SGE Toubleshooting CARB crashed when querying status of glite jobs When I was trying to get the status of glite jobs from CRAB, I got following error: chen...
Go to previous page / next page of Tier3 site log 16. 10. 2008 Out of memory problem impacting t3wn06 Ganglia and other services have been impacted since a few days...
Go to next page of Tier3 site log 25. 09. 2008 Testing of PhEDEx service for the PSI Tier 3 I activated the PhEDEx service yesterday after removing some difficulties...
Site logs for CMS PSI Tier3 DerekFeichtinger 25 Sep 2008
Go to previous page / next page of Tier3 site log Title Go to previous page / next page of Tier3 site log
Test simple CMSSW batch submission on local files/SE Using CMSSW The CMS software environment is set up with: source /swshare/cms/cmsset default.sh prepare...
CMS Tier 3 News Item Summary Details Storage : The OpenSolaris problems are now understood, and we can work around them. We will start the service with just...
IssueDcacheSrmRead Problem Description SRMv1 read from the SE fails, even though globus url copy of the same file from the SE as well as from the pool node directly...
IssueTemplate Symptoms Summary: Occurrences At what times did this problem occur (used to estimate frequency): Observations Solution or Workaround Monitoring...
Name Type Size Values Tooltip message Attributes Affected Service text 100 M Symptom summary text 100 M Reason Understood...
CMS Tier 3 News Item Summary OpenSolaris problems An update to OpenSolaris rendered the systems unbootable. This is a major point against using this OS, even though...
Node Type: Firewall requirements local port open to reason Regular Maintenance work Emergency Measures Installation Services Backups
CMS Tier 3 News Item Summary Details
TWiki`s CmsTier3 web The 1 web of TWiki. TWiki is a Web Based Collaboration Platform for the Enterprise.
See also the faster 1
See also the verbose 1 .
This is a subscription service to be automatically notified by e mail when topics change in this 1 web. This is a convenient service, so you do not have to come...
TWiki`s CmsTier3 web /CmsTier3 The 1 web of TWiki. TWiki is a Web Based Collaboration Platform for the Enterprise.
Number of topics: 364

Show recent changes with 50, 100, 200, 500, 1000 topics, all changes

Related topics: RSS feed, rounded corners RSS feed, ATOM feed, WebNotify, site changes, site map

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r2 - 2005-03-28 - TWikiContributor
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback