T3 Downtime due to PSI yearly Compute Center Maintenance -- 05. 01. 2022 DerekFeichtinger
Downtime will last from 16:00h Fri, 7. Jan until 10:00h on Mon, 10. Jan

Monitoring

Batch jobs (queuing system)

Current queue / accounting / CMS Dashboard

Number of running and queued jobs:


Ganglia WN page

Storage

/pnfs dir

Show space graphs for

Links:

/pnfs dir I/O queues

  • regular I/O queue movers = dcap/gsidcap movers (heavy random IO for internal analysis) ; MAX 100 ACTIVE movers per file server, other requests will get QUEUED
  • wan I/O queue movers = SRM/gridftp movers (transfers of whole files also from outside) ; MAX 2 ACTIVE movers per file server, other requests will get QUEUED
  • xrootd I/O queue movers = transfers of files by xrootd ; MAX 2 ACTIVE movers per file server, other requests will get QUEUED
  • [t3uiXY]$ watch -n 1 elinks 'http://t3dcachedb:2288/queueInfo' to check by CLI the regular, wan , xrootd I/O queues status ( though this is seldom needed )

ACTIVE movers:

QUEUED movers ( the associated I/O queue is exceeding the max amount of allowed ACTIVE movers ) :

PENDING requests (these are hanging file transfers, almost always an error state if they persist):

/shome and /swshare dirs

/shome space usage

Networking and File Transfers (+ PhEDEx)

Links:

Plotting interval:


Availability reports

These tests are run by the centralized Grid monitoring services and they determine whether the T3 or the T2 are considered to be working correctly:
  • CMS Nagios : T3 , T2
  • German Nagios : T3 , T2
  • Gstat : T3 , T2

Computer Room Temps

private link
Edit | Attach | Watch | Print version | History: r121 | r75 < r74 < r73 < r72 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r73 - 2015-06-09 - FabioMartinelli
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback