Tier3Overview < CmsTier3

Tags: view all tags
<!-- keep this as a security measure:
   #uncomment if the subject should only be modifiable by the listed groups 
   * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup
   * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup
   #uncomment this if you want the page only be viewable by the listed groups
   # * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup
-->

---+!! Cluster Overview

%TOC%

---+!! OLD Cluster Overview



---++ Cluster Composition and Services

%SEARCH{"META:FORM{name=\"NodeTypeForm\""  type="regex" nosearch="on" nototal="on" excludetopic="Web*" header="| *Type* | *Hosts* | *Hardware* | *Services* |" format="| !$percntCALC{$SUBSTITUTE($topic,NodeType,)}$percnt | $formfield(Hostnames) | $formfield(Hardware) | $formfield(Services) |"}%

---++ Cluster Specs
---+++ Original Requirements

Our requirements document (q.v. our internal CMS-Tier3-Project-description.doc) specified the following for the two install phases.

| *Phase* | *Year* | *CPU / kCINT2000* | *Disk / TB* |
| A | 2008 | 180 | 75 |
| B | 2009 | 500 | 250 |
| C | 2012 | ?? | ?? |

---+++ Phase A - CPU

CINT = SPECint benchmark value (average baseline value taken from the [[http://www.spec.org/benchmarks.html#cpu][SPEC published results]]). For 1000 CINT2000 one frequently uses the abbreviation *kSI2k*.

| *No. of WNs* | *Processors* | *Cores/node* | *CINT2006/core* | *kCINT2000/core* | *No. of Cores* | *CINT2006* | *kCINT2000* |
| 8 | 2*Xeon E5410 | 8 | 18.8 | 3.34 | %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% | %CALC{"$EVAL($T(R$ROW(0):C4) * $T(R$ROW(0):C6))"}% | %CALC{"$EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C6))"}% |

*Note:* We use this conversion for the untabled CINT2000 values: =CINT2000base=-1373.9673+250.8226*CINT2006base=

---+++ Phase A - Storage
| *No. of Fileservers* | *Type* | *Space/Node (TB)* | *Total Space (TB)* |
| 6 | [[http://en.wikipedia.org/wiki/Sun_Fire_X4500][SUN X4500]] | 17.5 | %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |

*Note:* 48 500 GB disks per fileserver:
   * 4 raidz pools with 9 disks, 1 raidz pool with 8 disks: (4*8 + 1*7) * 500 GB = 21 TB usable raw: measured with fs overhead: ca 17.5 (83%)
   * 2 disks mirrored OS
   * 2 disks as hot spare

---+++ Phase B - CPU

For 1000 CINT2000 one frequently uses the abbreviation *kSI2k*.

%STARTSECTION{name="Cpu" type="section"}% 
%TABLE{caption="T3_CH_PSI CPU resources and performance benchmarks 2010/2011"}%
| *No. of WNs* | *Processors* | *Cores/node* | *HS06/node* | *HS06/core* | *kCINT2000/core* | *total No. of Cores* | *total HS06* | *total kCINT2000* |
|  7 | 2*Xeon E5410 |  8 |  %CALC{"$ROUND($EVAL(131.38 * 0.7),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  20 | 2*Xeon X5560 |  8 |  117.53 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | | | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |

*Note:* [[https://twiki.cern.ch/twiki/bin/view/FIOgroup/ProcRefHepSpec2006#Si2k_to_HEPSPEC06_conversion][hepspec06/kSi2k = 3.9 ± 0.2 =&gt; kSi2k = hepspec06/3.9]]

The PSI Tier-3 has 48 active users (August 2011). 
%ENDSECTION{name="Cpu" type="section"}%

---+++ Phase B - Storage

%STARTSECTION{name="Storage" type="section"}% 
%TABLE{caption="T3_CH_PSI SE Storage recources 2010/2011"}%
| *No. of Fileservers* | *Type* | *Space/Node (TB)* | *Total Space (TB)* |
|  4 | [[http://en.wikipedia.org/wiki/Sun_Fire_X4500][SUN X4500]] |  16.10 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |
|  5 | SUN X4540 |  34.09 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | |  *%CALC{"$SUM($ABOVE())"}%* |

*Note:* 48 1TB disks per X4540 fileserver:
   * 5 raidz pools with 9 disks: 5*9 * 1TB = 45 TB usable raw. df yields 34093714636800 bytes = 34.09 TB = 31.008 TiB 
      * Current dcache config: 2 pools of 14000 GiB = 15.03 TB
   * 3 disks as hot spare
   * OS sits on flash storage
   * Note: Two of our 6 X4500 are now used for home directories, SW areas and backup and therefore were taken out of this table.
   * For old X4500: 16097967341568 bytes = 16.10 TB = 14.64 TiB 
      * Current dcache config: 1 pool of 14000 GiB = 15.03 TB
%ENDSECTION{name="Storage" type="section"}%

---+++ Phase C - CPU

%STARTSECTION{name="Cpu" type="section"}% 
%TABLE{caption="T3_CH_PSI CPU resources and performance benchmarks 2012"}%
| *No. of WNs* | *Processors* | *Cores/node* | *HS06/node* | *HS06/core* | *kCINT2000/core* | *total No. of Cores* | *total HS06* | *total kCINT2000* |
|  20 | 2*Xeon X5560 |  8 |  117.53 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  11 | 2*E5-2670 2.60GHz |  16 |  263 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | | | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |

*Note:* [[https://twiki.cern.ch/twiki/bin/view/FIOgroup/ProcRefHepSpec2006#Si2k_to_HEPSPEC06_conversion][hepspec06/kSi2k = 3.9 ± 0.2 =&gt; kSi2k = hepspec06/3.9]]

The PSI Tier-3 has 53 active users (June 2012). 
%ENDSECTION{name="Cpu" type="section"}%

---+++ Phase C - Storage

%STARTSECTION{name="Storage" type="section"}% 
%TABLE{caption="T3_CH_PSI SE Storage recources 2012"}%
| *No. of Fileservers* | *Type* | *Space/Node (TB)* | *Total Space (TB)* |
|  4 | [[http://en.wikipedia.org/wiki/Sun_Fire_X4500][SUN X4500]] |  16.10 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |
|  5 | SUN X4540 |  34.09 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |
|  2 | [[http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-3328412-241644-241475-4091412.html][HP Proliant DL380 G7]] |  130 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | |  *%CALC{"$SUM($ABOVE())"}%* |

*Note 1:* 48 1TB disks per X4540 fileserver:
   * 5 raidz pools with 9 disks: 5*9 * 1TB = 45 TB usable raw. df yields 34093714636800 bytes = 34.09 TB = 31.008 TiB 
      * Current dcache config: 1 unique pool of ~31 TiB
   * 3 disks as hot spare
   * OS sits on flash storage
   * Note: Two of our 6 X4500 are now used for home directories, SW areas and backup and therefore were taken out of this table.
   * For old X4500: 16097967341568 bytes = 16.10 TB = 14.64 TiB 
      * Current dcache config: 1 pool of 14000 GiB = 15.03 TB
*Note 2:* 120 3TB disks hosted in [[http://www.sgi.com/products/storage/raid/5500.html][SGI IS5500]] per both [[http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-3328412-241644-241475-4091412.html][HP Proliant DL380 G7]] fileserver:
   * [[http://www.sgi.com/products/storage/raid/5500.html][SGI IS5500]] formatted with 12 Raid6, each 8+2 disks, 6 Raid6 offered to first [[http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-3328412-241644-241475-4091412.html][HP Proliant DL380 G7]], 6 to the other.
   * So 6 dCache cms pools each 22TB per [[http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-3328412-241644-241475-4091412.html][HP Proliant DL380 G7]] fileserver =&gt; 130TB per fileserver =&gt; 260TB both.
   * *2* hot spares , even if we have the Raid6 protection.
   * Generally speaking, the [[http://www.sgi.com/products/storage/raid/5500.html][SGI IS5500]] can be expanded by simply adding disk trays.
   * Just an idea, the cheapest way to expand this storage could be add an other expansion to [[http://www.sgi.com/products/storage/raid/5500.html][SGI IS5500]] with 12*5*3TB disks =&gt; organized like 6 Raid6 =&gt; we gain 130TB net =&gt; we attach these new 6 volumes to t3fs13 and we make a bonding of its 2*10Gbit/s Ethernets, we connect t3fs13 to the last 10Gbit/s uplink available in our 3 network switches.
%ENDSECTION{name="Storage" type="section"}%
---+++ Phase D - CPU

On 10th Oct 2013 the CPU resources are the same of 2012 =Phase C - CPU=.
---+++ Phase D - Storage

%STARTSECTION{name="Storage" type="section"}% 
%TABLE{caption="T3_CH_PSI SE Storage Resources 2013"}%
| *No. of Fileservers* | *Type* | *Space/Node (TB)* | *Total Space (TB)* | *Total Space (TiB)* |
|  4 | [[http://en.wikipedia.org/wiki/Sun_Fire_X4500][SUN X4500]] |  16 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3) * 0.90949470177293 ),0)"}% |
|  5 | SUN X4540 |  33 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3) * 0.90949470177293 ),0)"}% |
|  2 | [[http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-3328412-241644-241475-4091412.html][HP Proliant DL380 G7]] |  282 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3) * 0.90949470177293 ),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |

*Note 1:* During the Summer 2013 we connected a new [[http://www.netapp.com/us/products/storage-systems/e5400/index.aspx][NetApp E5400 360TB raw]] to the 2012 [[http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-3328412-241644-241475-4091412.html][HP Proliant DL380 G7]], so the final net storage has been easily doubled.
---+++ Phase E - CPU

On 18th Aug 2014 the CPU resources are: 
%STARTSECTION{name="Cpu" type="section"}% 
%TABLE{caption="T3_CH_PSI CPU resources and performance benchmarks 2014"}%
| *No. of WNs* | *Processors* | *Cores/node* | *HS06/node* | *HS06/core* | *kCINT2000/core* | *total No. of Cores* | *total HS06* | *total kCINT2000* |
|  20 | 2*Xeon X5560 |  8 |  117.53 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  11 | 2*E5-2670 2.60GHz |  16 |  263 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  4 | 2*AMD 6272 2.40GHz |  32 |  241 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | | | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |

| *No. of UIs* | *Processors* | *Cores/node* | *HS06/node* | *HS06/core* | *kCINT2000/core* | *total No. of Cores* | *total HS06* | *total kCINT2000* |
|  6 | 2*AMD 6272 2.40GHz |  32 |  241 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | | | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |

---+++ Phase E - Storage

Same storage as Tier3Overview#Phase_D_Storage
---+++ Phase F - CPU
In April 2016 the CPU resources are : 
%STARTSECTION{name="Cpu" type="section"}% 
%TABLE{caption="T3_CH_PSI CPU resources and performance benchmarks 2014"}%
| *No. of WNs* | *Processors* | *Cores/node* | *HS06/node* | *HS06/core* | *kCINT2000/core* | *total No. of Cores* | *total HS06* | *total kCINT2000* |
|  9 | 2*Xeon E5-2698v3 |  64 |  700 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |   %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  20 | 2*Xeon X5560 |  8 |  117.53 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  11 | 2*E5-2670 2.60GHz |  16 |  263 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  4 | 2*AMD 6272 2.40GHz |  32 |  241 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | | | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |

| *No. of UIs* | *Processors* | *Cores/node* | *HS06/node* | *HS06/core* | *kCINT2000/core* | *total No. of Cores* | *total HS06* | *total kCINT2000* |
|  6 | 2*AMD 6272 2.40GHz |  32 |  241 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | | | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |
---+++ Phase F - Storage
Same as Phase E
---+++ Phase G - CPU %N%
In Nov 2016 the CPU resources are : 
%STARTSECTION{name="Cpu" type="section"}% 
%TABLE{caption="T3_CH_PSI CPU resources and performance benchmarks 2016"}%
| *No. of WNs* | *Processors* | *Cores/node* | *HS06/node* | *HS06/core* | *kCINT2000/core* | *total No. of Cores* | *total HS06* | *total kCINT2000* |
|  9 | 2*Xeon E5-2698v3 |  64 |  700 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |   %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  20 | 2*Xeon X5560 |  8 |  117.53 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  11 | 2*E5-2670 2.60GHz |  16 |  263 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  10 | 2*AMD 6272 2.40GHz |  32 |  241 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | | | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |

| *No. of UIs* | *Processors* | *Cores/node* | *HS06/node* | *HS06/core* | *kCINT2000/core* | *total No. of Cores* | *total HS06* | *total kCINT2000* |
|  3 | 2*E5-2697 v4 @ 2.30GHz |  72 |  700 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C4) / $T(R$ROW(0):C3)),2)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) / 3.9),2)"}% |  %CALC{"$EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3))"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C5) * $T(R$ROW(0):C7)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C6) * $T(R$ROW(0):C7)),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | | | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |
---+++ Phase G - Storage %N%
%STARTSECTION{name="Storage" type="section"}% 
%TABLE{caption="T3_CH_PSI SE Storage Resources Nov 2016"}%
| *No. of Fileservers* | *Type* | *Space/Node (TB)* | *Total Space (TB)* | *Total Space (TiB)* |
|  4 | SUN X4500 %RED%Only Redundant Copies%ENDCOLOR% |  16 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3) * 0.90949470177293 ),0)"}% |
|  5 | SUN X4540 %RED%Only Redundant Copies%ENDCOLOR% |  33 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3) * 0.90949470177293 ),0)"}% |
|  2 | [[http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-3328412-241644-241475-4091412.html][HP Proliant DL380 G7]] |  282 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3) * 0.90949470177293 ),0)"}% |
|  1 | [[https://www.hpe.com/us/en/product-catalog/servers/proliant-servers/pip.hpe-proliant-dl380-gen9-server.7271241.html][HP Proliant DL380 G9]] |  200 |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3)),0)"}% |  %CALC{"$ROUND($EVAL($T(R$ROW(0):C1) * $T(R$ROW(0):C3) * 0.90949470177293 ),0)"}% |
|  *%CALC{"$SUM($ABOVE())"}%* | | |  *%CALC{"$SUM($ABOVE())"}%* |  *%CALC{"$SUM($ABOVE())"}%* |
---++ Rack layout

   * Rack layout: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}% <br /><img width="1179" alt="T3-Racklayout.png" src="%ATTACHURLPATH%/T3-Racklayout.png" height="870" />
%ENDTWISTY%

---++ T3 HW Pictures

---+++ Phase A:
   * Photo: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /><img alt="22072008074.jpg" src="%ATTACHURLPATH%/22072008074.jpg" />
%ENDTWISTY%

---+++ Phase B:
   * a lot of crates and packing material <br /><img width="728" alt="" src="%ATTACHURLPATH%/16122009529-small.jpg" height="546" />
   * Tier-3 phase B system: <br /> <img width="971" alt="20100514_005.jpg" src="%ATTACHURLPATH%/20100514_005.jpg" height="546" />

---+++ 2 * [[http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-3328412-241644-241475-4091412.html][HP Proliant DL380 G7]]
   * Front Side Right: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IMG_0494.JPG" src="%ATTACHURLPATH%/IMG_0494.JPG" height="764" />
   * Front Side Left: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IMG_0493.JPG" src="%ATTACHURLPATH%/IMG_0493.JPG" height="764" />
   * Chassis Lock: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="800" alt="IMG_0492.JPG" src="%ATTACHURLPATH%/IMG_0492.JPG" height="600" />
   * Internal: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IMG_0487.JPG" src="%ATTACHURLPATH%/IMG_0487.JPG" height="764" />
   * RAM Left Side: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IMG_0488.JPG" src="%ATTACHURLPATH%/IMG_0488.JPG" height="764" />
   * RAM Right Side: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IMG_0489.JPG" src="%ATTACHURLPATH%/IMG_0489.JPG" height="764" />
   * Back Plane: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="800" alt="IMG_0491.JPG" src="%ATTACHURLPATH%/IMG_0491.JPG" height="600" />
   * Back Side with 4 1Gbit/s E, 4 8Gbit/s FC, 2 10Gbit/s E: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IMG_0486.JPG" src="%ATTACHURLPATH%/IMG_0486.JPG" height="764" />
   * 1 Hot Swap Power Supply: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="800" alt="IMG_0484.JPG" src="%ATTACHURLPATH%/IMG_0484.JPG" height="600" />
   * 1 Hot Swap Fan: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="800" alt="IMG_0490.JPG" src="%ATTACHURLPATH%/IMG_0490.JPG" height="600" />
   * 1 Hot Swap SAS disk Up Side: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IMG_0495.JPG" src="%ATTACHURLPATH%/IMG_0495.JPG" height="764" />
   * 1 Hot Swap SAS disk Back Side: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IMG_0496.JPG" src="%ATTACHURLPATH%/IMG_0496.JPG" height="764" />
---+++ 1 * [[http://www.sgi.com/products/storage/raid/5500.html][SGI IS5500]]
   * Front: %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IS5500-front.jpg" src="%ATTACHURLPATH%/IS5500-front.jpg" height="764" />
   * Back ( but don't consider the Infiniband expansion ): %TWISTY{showlink="Show" hidelink="Hide" showimgleft="%ICONURLPATH{toggleopen-small}%" hideimgleft="%ICONURLPATH{toggleclose-small}%"}%<br /> <img width="1024" alt="IS5500-back.jpg" src="%ATTACHURLPATH%/IS5500-back.jpg" height="764" />
   * SGI IS5500 360TB front picture + 2 HP Proliant: <br /> <img width="600" alt="IMG_0605.JPG" src="%ATTACHURLPATH%/IMG_0605.JPG" height="764" />
---+++ 11 * [[http://www.intel.com/content/www/us/en/motherboards/server-motherboards/server-board-s2600jf.html][Intel S2600JF]]

   * 11 Intel S2600JF installed by mdadm raid1 (OS) and mdadm raid0 =/scratch= : <br /> <img width="900" alt="IMG_0600_2.png" src="%ATTACHURLPATH%/IMG_0600_2.png" height="600" />
---+++ 10 * [[Supermicro1uH8DGU-F]] %N%
   * 6 SL6 UIs + 4 SL6 WNs installed by mdadm raid1+0 - 32 cores 100GB RAM 1700 GB =/scratch=: <br /> <img width="700" alt="New10UIsSummer2014.JPG" src="%ATTACHURLPATH%/New10UIsSummer2014.JPG" height="1000" />
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
jpg	16122009529-small.jpg	r1	manage	142.1 K	2009-12-23 - 09:53	DerekFeichtinger	phase B crates
jpg	20100514_005.jpg	r1	manage	736.6 K	2010-05-14 - 13:20	DerekFeichtinger	Tier-3 phase B
jpg	22072008074.jpg	r1	manage	515.6 K	2008-08-23 - 09:55	DerekFeichtinger	cluster photo
odp	CHIPP-Meeting-20080909.odp	r1	manage	965.2 K	2008-09-01 - 16:18	DerekFeichtinger
JPG	IMG_0484.JPG	r1	manage	94.5 K	2012-01-11 - 10:23	FabioMartinelli	1 Hot Swap Power Supply
JPG	IMG_0486.JPG	r1	manage	129.0 K	2012-01-11 - 10:30	FabioMartinelli	Back Side with 4 1Gbit/s E, 4 8Gbit/s FC, 2 10Gbit/s E
JPG	IMG_0487.JPG	r1	manage	116.5 K	2012-01-11 - 10:33	FabioMartinelli	Internal
JPG	IMG_0488.JPG	r1	manage	127.6 K	2012-01-11 - 10:34	FabioMartinelli	RAM Left Side
JPG	IMG_0489.JPG	r1	manage	149.8 K	2012-01-11 - 10:34	FabioMartinelli	RAM Right Side
JPG	IMG_0490.JPG	r1	manage	77.7 K	2012-01-11 - 10:17	FabioMartinelli	1 Hot Swap Fan
JPG	IMG_0491.JPG	r1	manage	93.1 K	2012-01-11 - 10:20	FabioMartinelli	Back Plane
JPG	IMG_0492.JPG	r1	manage	83.6 K	2012-01-11 - 10:20	FabioMartinelli	Chassis Lock
JPG	IMG_0493.JPG	r1	manage	125.3 K	2012-01-11 - 10:41	FabioMartinelli	Front Side Left
JPG	IMG_0494.JPG	r1	manage	131.1 K	2012-01-11 - 10:40	FabioMartinelli	Front Side Right:
JPG	IMG_0495.JPG	r1	manage	86.4 K	2012-01-11 - 10:43	FabioMartinelli	1 Hot Swap SAS disk Up Side
JPG	IMG_0496.JPG	r1	manage	70.9 K	2012-01-11 - 10:45	FabioMartinelli	1 Hot Swap SAS disk Back Side:
png	IMG_0600_2.png	r2 r1	manage	5765.2 K	2012-05-30 - 08:04	FabioMartinelli	11 Intel S2600JF + mdadm raid 0/1
JPG	IMG_0605.JPG	r1	manage	2599.2 K	2012-05-24 - 13:54	FabioMartinelli	SGI IS5500 360TB front picture + 2 HP Proliant
jpg	IS5500-back.jpg	r1	manage	2531.9 K	2012-02-14 - 09:32	FabioMartinelli	SGI IS5500 360TB back picture
jpg	IS5500-front.jpg	r1	manage	2955.6 K	2012-02-14 - 09:32	FabioMartinelli	SGI IS5500 360TB front picture
JPG	New10UIsSummer2014.JPG	r1	manage	1325.9 K	2014-07-27 - 08:27	FabioMartinelli	6 UIs + 4 WNs 32cores 100GB RAM 1700 GB scratch
png	T3-Racklayout.png	r3 r2 r1	manage	37.1 K	2010-07-09 - 12:10	DerekFeichtinger	Tier3 Rack layout
JPG	photo_1.JPG	r1	manage	2690.8 K	2015-02-23 - 14:28	FabioMartinelli	6 UIs + 4 WNs 32cores 100GB RAM 1700 GB scratch
Topic revision: r50 - 2018-08-21 - NinaLoktionova
CmsTier3
User Pages
Main Page
Policies
Physics Groups
Steering Board Meetings
Admin Pages
AdminArea
Cluster Specs