Monitoring
Batch jobs (queuing system)
Current queue / accounting / CMS Dashboard
Number of running and queued jobs:
Ganglia WN page
Storage
/pnfs
dir
Show space graphs for
Links:
/pnfs
dir I/O queues
-
regular
I/O queue movers = dcap/gsidcap movers (heavy random IO for internal analysis) ; MAX 100 ACTIVE movers per file server, other requests will get QUEUED
-
wan
I/O queue movers = SRM/gridftp movers (transfers of whole files also from outside) ; MAX 2 ACTIVE movers per file server, other requests will get QUEUED
-
xrootd
I/O queue movers = transfers of files by xrootd ; MAX 2 ACTIVE movers per file server, other requests will get QUEUED
-
[t3uiXY]$ watch -n 1 elinks 'http://t3dcachedb:2288/queueInfo'
to check by CLI the regular
, wan
, xrootd
I/O queues status ( even though this is seldom needed )
ACTIVE movers:
QUEUED movers ( the associated I/O queue is exceeding the max amount of allowed
ACTIVE movers ) :
PENDING requests (these are hanging file transfers, almost always an error state if they persist):
/shome
and /swshare
dirs
/shome space usage
Networking and File Transfers (+ PhEDEx)
Links:
Plotting interval:
Availability reports
These tests are run by the centralized Grid monitoring services and they determine whether the T3 or the T2 are considered to be working correctly:
Computer Room Temps
private link