Difference: Tier3Monitoring (1 vs. 117)

Revision 1172019-06-27 - NinaLoktionova

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

/pnfs dir I/O queues

Changed:
<
<
  • regular I/O queue movers = dcap/gsidcap/LAN xrootd movers (heavy random IO for internal analysis) ; MAX 100 ACTIVE movers per file server, others will get QUEUED
  • wan I/O queue movers = SRM/gridftp movers (transfers of whole files also from outside) ; MAX 2 ACTIVE movers per file server, others will get QUEUED
  • xrootd I/O queue movers = WAN xrootd movers ; MAX 2 ACTIVE movers per file server, others will get QUEUED
  • To check by CLI the I/O queues run from a UI watch -n 1 -d  lynx --dump --width=200 'http://t3dcachedb:2288/queueInfo' e.g. if your jobs are not progressing it might be due to a file server with too many queued movers ; in this case you can inform by email the T3 users ( the T3 admins will get it too )
>
>
  • regular I/O queue movers = dcap/gsidcap/LAN xrootd movers (heavy random IO for internal analysis) ;
  • wan I/O queue movers = SRM/gridftp movers (transfers of whole files also from outside) ;
  • xrootd I/O queue movers = WAN xrootd movers ;
  • To check by CLI the I/O queues run from a UI watch -n 1 -d  lynx --dump --width=200 'http://t3dcachedb:2288/queueInfo' e.g. if your jobs are not progressing it might be due to a file server with too many queued movers ; in this case you can inform by email T3 admins.
 

Revision 1162018-08-23 - NinaLoktionova

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1152018-05-09 - NinaLoktionova

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1142018-05-04 - NinaLoktionova

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1132018-05-04 - NinaLoktionova

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
 

Batch jobs (queuing system)

Changed:
<
<
Current queue / accounting
>
>
Current queue / accounting
  Number of running and queued jobs:
Line: 72 to 79
 
Changed:
<
<
>
>
 
Changed:
<
<
<--   * show user/dataset space usage -->
>
>
<--   * show user/dataset space usage -->
 

Show storage space graphs for

Line: 128 to 135
  -->

/mnt/t3nfs01/data01/{shome,swshare} dirs NEW

Changed:
<
<
User Space Report
>
>
User Space Report
 

Line: 155 to 162
 
Changed:
<
<
>
>
 

Revision 1122018-04-26 - NinaLoktionova

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
 

Revision 1112018-04-26 - NinaLoktionova

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
 

/pnfs dir

Links:

Line: 77 to 78
 
<--   * show user/dataset space usage -->
Changed:
<
<

Revision 1102018-04-24 - NinaLoktionova

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
 

/pnfs dir I/O queues

  • regular I/O queue movers = dcap/gsidcap/LAN xrootd movers (heavy random IO for internal analysis) ; MAX 100 ACTIVE movers per file server, others will get QUEUED
Line: 82 to 88
 
  • wan I/O queue movers = SRM/gridftp movers (transfers of whole files also from outside) ; MAX 2 ACTIVE movers per file server, others will get QUEUED
  • xrootd I/O queue movers = WAN xrootd movers ; MAX 2 ACTIVE movers per file server, others will get QUEUED
  • To check by CLI the I/O queues run from a UI watch -n 1 -d  lynx --dump --width=200 'http://t3dcachedb:2288/queueInfo' e.g. if your jobs are not progressing it might be due to a file server with too many queued movers ; in this case you can inform by email the T3 users ( the T3 admins will get it too )
Added:
>
>
 
Added:
>
>
 

/mnt/t3nfs01/data01/{shome,swshare} dirs NEW

User Space Report

Revision 1092016-11-25 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1082016-11-24 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1072016-11-06 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1062016-10-20 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1052016-09-29 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1042016-09-13 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1032016-09-07 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1022016-09-05 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1012016-09-03 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1002016-08-30 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 992016-08-26 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 982016-08-05 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 972016-07-13 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 962016-07-04 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 952016-07-03 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 942016-07-01 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 932016-06-24 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 922016-06-16 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 912016-06-13 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 902016-06-10 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 892016-06-10 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 882016-06-02 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 872016-05-30 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 862016-05-25 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 852016-05-07 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 842016-04-12 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 832016-03-31 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 822016-03-31 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 812016-03-29 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 802016-03-24 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 792016-03-23 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 782015-09-22 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 772015-09-08 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 762015-09-08 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 752015-09-07 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 742015-07-31 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 732015-06-09 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 722015-05-28 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 712015-05-22 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 702015-05-20 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 692015-05-20 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 682015-04-27 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 672015-04-22 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 662015-03-13 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 652015-03-12 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 642015-03-11 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 632015-03-10 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 622014-12-15 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 612014-12-12 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 602014-11-17 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 592014-05-29 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 582014-05-19 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 572014-05-13 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 562014-05-12 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 552014-04-07 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 542014-04-04 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 532014-04-03 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 522014-02-17 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 512014-02-06 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 502014-02-05 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 492014-02-05 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 482014-02-05 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 472014-01-13 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 462013-11-14 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 452013-06-25 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 442013-01-14 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 432013-01-11 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 422012-11-29 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 412012-10-11 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 402012-07-16 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 392012-01-31 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 382012-01-19 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 372012-01-09 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 362011-10-25 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 352011-10-18 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 342011-10-10 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 332011-10-07 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 322011-07-14 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 312011-07-13 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 302011-07-06 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
<-- -->
 

Worker nodes load monitoring

Ganglia WN page

Revision 292011-07-06 - FabioMartinelli

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
<-- -->
 

Worker nodes load monitoring

Ganglia WN page

Revision 282011-04-01 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 272011-03-29 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 262011-03-17 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 252011-03-09 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 242011-01-28 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 232010-06-28 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 222010-06-16 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 212010-05-09 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 202010-05-07 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 192010-05-04 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 182010-04-28 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 172010-02-12 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 162009-12-11 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Added:
>
>

| welcome to test Slurm GPU and CPU nodes -- 10. 05. 2019 NinaLoktionova
Dear T3 Users,

please let us know if you are interested in testing new Batch System and SW on RHEL7:

https://wiki.chipp.ch/twiki/bin/view/CmsTier3/SlurmUsage

|

 

PSI Tier-3 Monitoring

Revision 152009-11-06 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 142009-09-19 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 132009-05-20 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 122009-05-10 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 112009-05-07 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 102009-01-27 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 92009-01-22 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 82009-01-22 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 72009-01-21 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 62009-01-19 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 52008-11-28 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 42008-11-28 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 32008-11-27 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 22008-11-21 - DerekFeichtinger

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 12008-11-12 - DerekFeichtinger

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"
<-- keep this as a security measure:
   #uncomment if the subject should only be modifiable by the listed groups 
   # * Set ALLOWTOPICCHANGE = TWikiAdminGroup,Main.CMSAdminGroup
   # * Set ALLOWTOPICRENAME = TWikiAdminGroup,Main.CMSAdminGroup
   #uncomment this if you want the page only be viewable by the listed groups
   # * Set ALLOWTOPICVIEW = TWikiAdminGroup,Main.CMSAdminGroup,Main.CMSAdminReaderGroup
-->

PSI Tier-3 Monitoring

Batch jobs on the CE

Number of running and queued jobs:

Worker nodes load

Ganglia WN page

Storage Element

Free storage space:

-- DerekFeichtinger - 12 Nov 2008

 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback