Sherpa Integration Project - Technical Documentation

Introduction

This page documents the use of the T3 for the SherpaNLO validation and integration project.

This is evolving documentation; if something is unclear or missing please contact dmeister@phys.ethz.ch

Purpose

The T3 should be used as a common test-bed for all interested people within CMS working on the validation of SherpaNLO and its integration into CMSSW. The fact that software can be installed centrally helps others users to reproduce results.

Constraints

Please remember this is a secondary use of the cluster which should in no way have an impact on the normal work of its regular users; try not to overload the system and have a look at the monitoring page before starting a large number of jobs

Accounts

New Accounts

To get a guest account for the T3 for the purpose of this Sherpa project please contact dmeister@phys.ethz.ch.

User Interface

Please use the t3ui03.psi.ch machine to log in; this is a user interface machine and should not be used for heavy processing; in case this should be unavailable try t3ui17.psi.ch or t3ui19.psi.ch.

Troubleshooting

If you have any problem with your account please have a look at the documentation in this TWiki (especially look at the page User Information section on the home page); if this does not help contact cms-tier3@lists.psi.ch and do not forget to mention that you have a Sherpa Guest Account.

Software

CMSSW versions

Both CMSSW_5_3_9_sherpa2beta2 and CMSSW_5_3_11_sherpa2beta2 are available.

New Install

New (or newly configured) versions of Sherpa can be installed centrally in /swshare/Sherpa/; to get a folder there that is writable with your user account, please contact dmeister@phys.ethz.ch; note that you are expected to provide a complete installation/configuration manual for your version afterwards (example see below).

Available Versions

• 1.4.3 versions in /swshare/Sherpa/1.4.3
• 2.0-beta2 versions (from tarball) in /swshare/Sherpa/2.0-beta2
• 2.0-beta2-svn versions (latest beta2 from SVN includes fixed interface to OpenLoops 0.3 in /swshare/Sherpa/2.0-beta2-svn
• 2.0.0 versions in /swshare/Sherpa/2.0.0

Batch System

We have 3 dedicated queues on the batch system that should be used for Sherpa related jobs; please do not use any of the other regular queues (we can of course also technically restrict you from using them but that is some additional work on our side).

For general information about batch job submissions please read the how to submit jobs page.

Queues

• sherpa.gen.q for event generation (low priority queue)
• sherpa.int.long.q for integration (with some time limits)
• sherpa.int.vlogn.q for integration (without any time limits)

Limits

1. Queue job runtime limits
   • sherpa.gen.q: 12h
   • sherpa.int.long.q: 72h
   • sherpa.int.vlong.q: none
2. Queue job limits. How many jobs can be running in each of the queues:
   • sherpa.gen.q: 50 jobs
   • sherpa.int.long.q: 32 jobs
   • sherpa.int.vlong.q: 16 jobs
3. Per user job limits. Defines the maximum number of jobs a user can have running in each queue:
   • sherpa.gen.q: 32 jobs
   • sherpa.int.long.q: 32 jobs
   • sherpa.int.vlong.q: 8 jobs
4. The policies are relaxed at night and on weekends, so that the cluster can be taken by a bigger number of longer jobs:
   • Night time defined as weekdays from 19h - 4h, weekend time defined as Sat 4h - Mon 4h (timezone is Europe/Zurich)
   • sherpa.gen.q: 100 jobs
   • sherpa.int.long.q: 64 jobs
   • sherpa.int.vlong.q: 16 jobs

Example job script

#!/bin/bash                                                                                                                                                                                                                
#$ -cwd                                                                                                                                                                                                                    
#$ -o /shome/dmeister/data/comp/mpi/tests/log/                                                                                                                                                                             
#$ -e /shome/dmeister/data/comp/mpi/tests/log/                                                                                                                                                                             
#$ -N mpi_sherpa_fu                                                                                                                                                                                                    
#$ -q sherpa.int.long.q                                                                                                                                                                                                    
#$ -pe mpi_fu 32                                                                                                                                                                                                           
source /var/lib/mpi-selector/data/openmpi-1.4-gcc-x86_64.sh
mpirun -np 32 /swshare/Sherpa/2.0.0/131201-openmpi/bin/Sherpa [params]

Questions/Ideas

Feel free do drop me an e-mail at dmeister@phys.ethz.ch, contact me on Skype at daniel-meister, or come by my CERN office at 32/2-C13 any time if you have questions or input about the setup.

Additional Material

• https://indico.cern.ch/getFile.py/access?contribId=0&resId=0&materialId=slides&confId=269517

-- DanielMeister - 2013-07-18