Tags:
create new tag
view all tags

Benchmarks

Hardware

CPU * Executeable HEPSPEC06* Spec2k* Clock speed [MHz] L2+L3 cache size [grand total, KB] Cores (runs) HT on/off Memory [GB] Mainboard type Power (W) W/SPEC SPEC/core
Intel Xeon E5540 32-bit 104.27   2533 2048+16384 8 off 24 (6 modules) SUN ASSY,BLADE/IB,X6275      
Intel Xeon E5540 32-bit 117.78   2533 2048+16384 12 on 24 (6 modules) SUN ASSY,BLADE/IB,X6275      
Intel Xeon E5540 32-bit 127.26   2533 2048+16384 16 on 24 (6 modules) SUN ASSY,BLADE/IB,X6275      
AMD Opteron 6172 32-bit 196.58   2100 12288+20480 24 not supported 72 (9 modules) Supermicro H8DGU      
AMD Opteron 6272 (Interlagos) 32-bit 222   2100 16384+12288 32 not supported 96 Supermicro H8DGU 400 1.8  
Intel Xeon E5-2670 (Sandy Bridge) 32-bit 333   2600 2048+20480 32 on 64 DALCO Twin2 (Intel S2600JF) 410 1.23 10.4
greina23 32-bit 384   2300 ? 40 on 64 -      
Intel Xeon E5540 64-bit 122.45   2533 2048+16384 8 off 24 (6 modules) SUN ASSY,BLADE/IB,X6275      
Intel Xeon E5540 64-bit 148.50   2533 2048+16384 16 on 24 (6 modules) SUN ASSY,BLADE/IB,X6275      
Intel Xeon E5450 64-bit 82.60   3000 24576+0 8 off 32 (16 modules) Sun SUN FIRE X4250      
AMD Opteron Processor 32-bit 119,95 15,7k 3000 24576+0 16 off 32 (32 modules) Sun Sun Blade X8440 Server Module      
AMD Opteron 6172 32-bit 196.58   2100 12288+20480 24 not supported 72 (9 modules) Supermicro H8DGU      
AMD Opteron 6172 64-bit 224.85   2100 12288+20480 24 not supported 72 (9 modules) Supermicro H8DGU      
Intel Xeon X5650 64-bit 206.31   2670 512+24576 24 on 32 (8 modules) IBM 69Y5698      
Intel Xeon E5-2680 v2 32-bit 442.48   2800   40 on 128       11.1
Intel Xeon E5-2680 v2 64-bit 505.04   2800   40 on 128       12.6
BRISI - Intel Xeon E5-2690 v3 64-bit (48 core) 211   2600   8 on 64        
BRISI - Intel Xeon E5-2690 v3 64-bit (48 core) 594   2600   32 on 64        
BRISI - Intel Xeon E5-2690 v3 32-bit (48 core) 511.06   2600   32 on 64        

* Performance per node.

The green values are measured according to CERN procedures.

Virtual Machines

CPU Executeable HEPSPEC06 Clock speed [MHz] Cores (runs) Memory [GB] Virtualization
Intel Xeon E5450 64-bit 25.90 3000 2 2 (1 modules) Xen HVM domU

Available Resources

This are the CPU recourses available on Phoenix. HEP-SPEC06 is measured according to the CERN procedures.

CPU # of cores HEP-SPEC06 kSI2k*
Intel Xeon E5-2670 0 2080 21632 5408
Intel Xeon E5-2680 v2 1920 24192 6048
Total 4000 45824 11456

* 4 HEP-SPEC06 = 1 kSI2k

How to Update Published Values

  • On ARC CEs it is enough to update the corresponding lines in arc.conf

[root@arc01 ~]# vim /etc/arc.conf
[...]                                                                                                     
jobreport_options="archiving:<%= scope.lookupvar('arc_ce::archiving_dir') %>,topic:/queue/global.accounting.cpu.central,gocdb_name:CSCS-LCG2,benchmark_type:HEPSPEC,benchmark_value:11.46,use_ssl:true"             
[...]                                                                                                     
cpu_scaling_reference_si00="2600"                                                                         
processor_other_description="Cores=35.84, Benchmark=11.46-HEP-SPEC06"                                     
[...]                                                                                                     
opsys="6.7"                                                                                               
[...]                                                                                                     
benchmark="SPECINT2000 2600"                                                                              
[...]                                                                                                     

[root@arc01 ~]# /etc/init.d/nordugrid-arc-inforeg stop      
[root@arc01 ~]# /etc/init.d/nordugrid-arc-ldap-infosys stop 
[root@arc01 ~]# /etc/init.d/nordugrid-arc-ldap-infosys start
[root@arc01 ~]# /etc/init.d/nordugrid-arc-inforeg start     

  • On CREAM CEs the correct values should be set in site-info.def and then Yaim run to apply the new values into GIP configuration

[root@cfengine ~]# vim -R /srv/cfengine/files/ANY/opt/cscs/siteinfo/site-info.def
[...]
CE_CPU_SPEED=2600                                                                           
CE_OTHERDESCR="Cores=35.84,Benchmark=11.46-HEP-SPEC06"                                      
CE_CAPABILITY="CPUScalingReferenceSI00=2600 Share=atlas:40 Share=cms:40 Share=lhcb:20 glexec
CE_SI00=2600                                                                                
CE_MINPHYSMEM=32768                                                                         
CE_MINVIRTMEM=96256                                                                         
CE_OS_="6.7"                                                                         
[...]

[root@cream02:~]# /opt/glite/yaim/bin/yaim -r -s /opt/cscs/siteinfo/site-info.def -n creame
 -n SLURM_utils -f config_bdii_5.2 -f config_cream_gip_info_dynamic_setenv -f config_cream_gip_info_dyna
c -f config_cream_gip_setenv -f config_cream_gip -f config_cream_gip_glue2_setenv -f config_cream_gip_gl
2 -f config_cream_gip_scheduler_plugin_setenv -f config_cream_gip_scheduler_plugin -f config_cream_gip_s
tware_plugin_setenv -f config_cream_gip_software_plugin -f config_cream_vo_tag_dir -f config_info_servic
cream_ce_setenv -f config_info_service_cream_ce -f config_info_service_cemon_setenv -f config_info_servi
_cemon -f config_cream_info_service_rtepublish_setenv -f config_cream_info_service_rtepublish        

 # if this does not work, just modify by hand

[root@cream01:~]# vim /var/lib/bdii/gip/ldif/static-file-CE.ldif     
[root@cream01:~]# vim /var/lib/bdii/gip/ldif/static-file-Cluster.ldif
[root@cream01:~]# vim /var/lib/bdii/gip/ldif/ComputingManager.ldif   
 
 # and restart bdii service

After a while the new values should be published by the site BDII:

Feb 11 15:08 [rgianni@ui:~]$ ldapsearch -x -LLL -h bdii.lcg.cscs.ch -p 2170 -b 'o=grid'

Possible Improvements

Not all of the jobs are CPU bound. Many of them are waiting for data. Therefore we could enable Hyper Threading and allow 12 jobs per WN. The other 4 cores will be reserved for the OPS/SGM and CSCS queue only.

  • Advantages
    • More CPU slots 1152 vs. 768
    • Better utilization of CPU
    • Better overall performance
    • Better match of CPU pledges for 2010 (about 13% more performance, see below)
    • Cores of CSCS queue available for testing; faster detection/debugging of problems
  • Disadvantages
    • Possibly less performance per single job slot
    • Accounting less accurate
      • Performance of a single job depends a lot on the number of running jobs / WN

CPU # of cores HEP-SPEC06 kSI2k*
Intel Xeon E5540 1152 11306,88 2826,72

* 4 HEP-SPEC06 = 1 kSI2k

Resources

-- PeterOettl - 2010-02-18
Edit | Attach | Watch | Print version | History: r21 < r20 < r19 < r18 < r17 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r21 - 2016-02-11 - GianniRicciardi
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback