Test simple CMSSW batch submission on local files/SE

Using CMSSW

• The CMS software environment is set up with:

  source /swshare/cms/cmsset_default.sh

• prepare to run CMSSW

  cd CMSSW_Version/src; eval `scramv1 runtime -(c)sh`

  or

  cmsenv

Run CMSSW interactively on a file stored on t3se01.psi.ch

• To run CMSSW interactively on a file stored on the t3se01.psi.ch, modify the "PoolSource" parameters in the CMSSW config file as follows

  source = PoolSource {
      untracked vstring fileNames = {
        'dcap://t3se01.psi.ch:22125/pnfs/psi.ch/cms/catalog/store/user/.../yourfile'
      }
      untracked int32 maxEvents = 10
    }
  
       

SGE batch job script

• For SGE, Batch jobs are shell scripts. Job submission is telling SGE to execute a shell script. You normally start your program from within that script. You don't need to create old-school job description files since you can talk with SGE from within the job script in case you need. Lines starting with #$ are interpreted by SGE. Below is a sample job script for submitting CMSSW job to SGE:

  #!/bin/bash
  #
  #$ -cwd
  #$ -j y
  #$ -S /bin/bash
  #
  date
  hostname
  pwd
  source /swshare/cms/cmsset_default.sh 
  eval `scramv1 runtime -sh`
  cmsRun yourCMSSWconfig.cfg
     

Submit it with command

qsub test.job

. In the job script, the lines starting with a #$ are SGE commands. Above, -cwd tells sge to cd to the directory job has been submitted when it starts: running(e.g. your cmssw project directory), "-j y" is to transfer the current environment variable state to the shell starting the job and -S tells which shell to dispatch the job with.

Demo on Submitting CMSSW jobs to SGE

Change to your

src

directory:

mkdir demo
cd Demo
mkedanlzr DemoAnalyzer

Compile the code:

cd DemoAnalyzer
scramv1 b

The mkedanlzr script has generated an example configuration file demoanalyzer.cfg in the DemoAnalyzer directory. Edit this file to change the data source file myfile.root to an event sample file containing muons and add a parameter limiting the number of events which located in local filesystem(using "file:myfile.root" or our SE(using "dcap://t3se01.psi.ch:22125/pnfs/psi.ch/cms/catalog/store/....". Now the configuration file should read like that.

import FWCore.ParameterSet.Config as cms

process = cms.Process("Demo")

process.load("FWCore.MessageService.MessageLogger_cfi")

process.maxEvents = cms.untracked.PSet( input = cms.untracked.int32(-1) )

process.source = cms.Source("PoolSource",
    # replace 'myfile.root' with the source file you want to use
    fileNames = cms.untracked.vstring(
        'dcap://t3se01.psi.ch:22125/pnfs/psi.ch/cms/catalog/store/user/zlchen/test.root'
    )
)

process.demo = cms.EDAnalyzer('DemoAnalyzer'
)


process.p = cms.Path(process.demo)

Create a job script"demoCMSSW.job" with following lines:

#!/bin/bash
#
#$ -cwd
#$ -j y
#$ -S /bin/bash
#
date
hostname
pwd
source /swshare/cms/cmsset_default.sh 
eval `scramv1 runtime -sh`
cmsRun demoanalyzer_cfg.py

Submit job to SGE:

$ qsub demoCMSSW.job 
Your job 24 ("demoCMSSW.job") has been submitted
$ qstat -g c
CLUSTER QUEUE                   CQLOAD   USED  AVAIL  TOTAL aoACDS  cdsuE  
-------------------------------------------------------------------------------
all.q                             0.00      1     31     56      0     24 
......
$ qstat -g c
CLUSTER QUEUE                   CQLOAD   USED  AVAIL  TOTAL aoACDS  cdsuE  
-------------------------------------------------------------------------------
all.q                             0.00      0     32     56      0     24 

The output of job script and your CMSSW program is in demoCMSSW.job.o24 -- ZhilingChen - 04 Sep 2008