How to run the HEP-SPEC2006 CPU benchmark
Introduction
Official How To by HEPIX
To get an objective measure of the CPU power of a WLCG site the Hepix community has created custom
SPEC CPU 2006 configurations for OS
Scientific Linux 4 and 5 and related gcc compilers; if you're lucky the CPU power of your CPUs has been already computed and reported in this
HEP-SPEC2006 results table. Conversely you need to compute the value as described in this page.
At PSI T3 we measured the following 2 HEP-SPEC2006 values:
Intel Xeon CPU X5560 2.80GHz @ PSI =>
117.53 =>
14.70 per job ( used 8 cores on t3wn29 )
Intel Xeon CPU E5-2670 2.60GHz @ PSI:
Jobs |
Node |
32bit HT OFF |
32bit HT ON |
64bit HT OFF |
64bit HT ON |
16 |
wn30 |
262.90 |
264.01 |
|
|
16 |
wn31 |
|
262.85 |
|
|
16 |
wn32 |
|
262.11 |
|
|
16 |
wn33 |
|
263.41 |
|
302.74 |
16 |
wn34 |
|
261.54 |
|
|
16 |
wn35 |
|
264.03 |
|
268.75 |
16 |
wn36 |
|
262.78 |
|
|
16 |
wn37 |
|
262.58 |
|
|
16 |
wn38 |
|
263.57 |
|
|
16 |
wn39 |
|
262.48 |
|
|
16 |
wn40 |
|
261.59 |
|
|
20 |
wn33 |
|
|
|
358.75 |
24 |
wn30 |
|
312.00 |
|
|
24 |
wn31 |
|
313.33 |
|
|
24 |
wn32 |
|
318.84 |
|
|
24 |
wn33 |
|
315.91 |
|
363.23 |
24 |
wn34 |
|
316.27 |
|
|
24 |
wn35 |
|
318.06 |
|
|
24 |
wn36 |
|
310.64 |
|
359.73 |
24 |
wn37 |
|
311.52 |
|
|
24 |
wn38 |
|
314.59 |
|
|
24 |
wn39 |
|
316.89 |
|
|
24 |
wn40 |
|
314.77 |
|
|
32 |
wn33 |
|
329.06 |
|
378.94 |
32 |
wn37 |
|
326.54 |
|
371.85 |
32 |
wn38 |
|
327.81 |
|
372.00 |
32 |
wn39 |
|
322.56 |
|
|
32 |
wn40 |
|
325.92 |
|
|
Intel Xeon CPU E5-2670 2.60GHz @ an UK site:
Intel Xeon CPU E5-2670 2.60GHz @ GridKA in Germany:
How to run HEPSPEC2006
In a nutshell you need:
- An idle server, if it's a WN attached to a batch system then disable that node from the batch system and simply wait until the node is idle to be tested.
- Buy or get your Institute copy of the SPEC CPU 2006 benchmark.
- Build the SPEC CPU 2006 benchmark.
- Run the SPEC CPU 2006 benchmark with the specific Scientific Linux 4 and 5 configuration.
- Run a script vs the results created from the benchmark to get the HEP SPEC CPU 2006 value of your CPU.
At PSI T3 the previous steps mean that you need to mount
t3nfs01:/mnt/t3nfs01/data01/swshare
and then ( probably change that
seq 32
to
seq #cores
):
rsync --delete -av /mnt/t3nfs01/data01/swshare/HEPSPEC2006/ /scratch/specint2006
cd /opt
ln -s /scratch/specint2006/
cd /scratch/specint2006/SPEC_CPU2006v1.1-ORIGINAL_DVD/
source shrc
# check that there is not a dir /scratch/specint2006/SPEC_CPU2006v1.1-ORIGINAL_DVD/result ; if so remove it
# pay attention to that linux32-gcc_cern.cfg, if you want 64bit it's linux64-gcc_cern.cfg
for i in $(seq 32) ; do nohup runspec --config linux32-gcc_cern.cfg --nobuild --noreportable all_cpp & done
# a new dir /scratch/specint2006/SPEC_CPU2006v1.1-ORIGINAL_DVD/result has been created
cp /scratch/specint2006/SPEC_CPU2006v1.1-ORIGINAL_DVD/THE_SCRIPT_TO_RUN_ONCE_runspec_FINISCHED.sh /scratch/specint2006/SPEC_CPU2006v1.1-ORIGINAL_DVD/result
cd /scratch/specint2006/SPEC_CPU2006v1.1-ORIGINAL_DVD/result
./THE_SCRIPT_TO_RUN_ONCE_runspec_FINISCHED.sh
How to compute HEPSPEC2006
A debug mode example:
[martinelli_f@t3wn29 result]$ ./THE_SCRIPT_TO_RUN_ONCE_runspec_FINISCHED.sh
+ RUNS=
+ for n in 'CPU2006.*.log'
++ echo CPU2006.035.log
++ sed 's/^.*CPU2006\.\(\w\+\)\.log/\1/'
+ RUNS=' 035'
+ for n in 'CPU2006.*.log'
++ echo CPU2006.036.log
++ sed 's/^.*CPU2006\.\(\w\+\)\.log/\1/'
+ RUNS=' 035 036'
+ for n in 'CPU2006.*.log'
++ echo CPU2006.037.log
++ sed 's/^.*CPU2006\.\(\w\+\)\.log/\1/'
+ RUNS=' 035 036 037'
+ for n in 'CPU2006.*.log'
++ echo CPU2006.038.log
++ sed 's/^.*CPU2006\.\(\w\+\)\.log/\1/'
+ RUNS=' 035 036 037 038'
+ for n in 'CPU2006.*.log'
++ echo CPU2006.039.log
++ sed 's/^.*CPU2006\.\(\w\+\)\.log/\1/'
+ RUNS=' 035 036 037 038 039'
+ for n in 'CPU2006.*.log'
++ echo CPU2006.040.log
++ sed 's/^.*CPU2006\.\(\w\+\)\.log/\1/'
+ RUNS=' 035 036 037 038 039 040'
+ for n in 'CPU2006.*.log'
++ echo CPU2006.041.log
++ sed 's/^.*CPU2006\.\(\w\+\)\.log/\1/'
+ RUNS=' 035 036 037 038 039 040 041'
+ for n in 'CPU2006.*.log'
++ echo CPU2006.042.log
++ sed 's/^.*CPU2006\.\(\w\+\)\.log/\1/'
+ RUNS=' 035 036 037 038 039 040 041 042'
+ SUM=0
+ for n in '$RUNS'
+ partial=0
+ count=0
++ sed -n -e '/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p' CFP2006.035.ref.txt CINT2006.035.ref.txt
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8)'
+ count=1
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.9)'
+ count=2
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.9) + l(21.1)'
+ count=3
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.9) + l(21.1) + l(19.4)'
+ count=4
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.9) + l(21.1) + l(19.4) + l(12.7)'
+ count=5
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.9) + l(21.1) + l(19.4) + l(12.7) + l(11.6)'
+ count=6
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.9) + l(21.1) + l(19.4) + l(12.7) + l(11.6) + l(19.6)'
+ count=7
+ [[ 0 + l(14.8) + l(23.9) + l(21.1) + l(19.4) + l(12.7) + l(11.6) + l(19.6) != 0 ]]
++ echo 'scale=8; e((0 + l(14.8) + l(23.9) + l(21.1) + l(19.4) + l(12.7) + l(11.6) + l(19.6)) / 7)'
++ bc -l
+ SUM='0 + 17.04546268'
+ for n in '$RUNS'
+ partial=0
+ count=0
++ sed -n -e '/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p' CFP2006.036.ref.txt CINT2006.036.ref.txt
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.7)'
+ count=1
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.7) + l(23.3)'
+ count=2
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.7) + l(23.3) + l(18.4)'
+ count=3
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.7) + l(23.3) + l(18.4) + l(20.2)'
+ count=4
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.7) + l(23.3) + l(18.4) + l(20.2) + l(11.8)'
+ count=5
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.7) + l(23.3) + l(18.4) + l(20.2) + l(11.8) + l(12.2)'
+ count=6
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.7) + l(23.3) + l(18.4) + l(20.2) + l(11.8) + l(12.2) + l(17.3)'
+ count=7
+ [[ 0 + l(14.7) + l(23.3) + l(18.4) + l(20.2) + l(11.8) + l(12.2) + l(17.3) != 0 ]]
++ echo 'scale=8; e((0 + l(14.7) + l(23.3) + l(18.4) + l(20.2) + l(11.8) + l(12.2) + l(17.3)) / 7)'
++ bc -l
+ SUM='0 + 17.04546268 + 16.38502831'
+ for n in '$RUNS'
+ partial=0
+ count=0
++ sed -n -e '/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p' CFP2006.037.ref.txt CINT2006.037.ref.txt
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8)'
+ count=1
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6)'
+ count=2
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.0)'
+ count=3
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.0) + l(19.5)'
+ count=4
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.0) + l(19.5) + l(12.1)'
+ count=5
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.0) + l(19.5) + l(12.1) + l(11.9)'
+ count=6
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.0) + l(19.5) + l(12.1) + l(11.9) + l(18.8)'
+ count=7
+ [[ 0 + l(14.8) + l(23.6) + l(17.0) + l(19.5) + l(12.1) + l(11.9) + l(18.8) != 0 ]]
++ echo 'scale=8; e((0 + l(14.8) + l(23.6) + l(17.0) + l(19.5) + l(12.1) + l(11.9) + l(18.8)) / 7)'
++ bc -l
+ SUM='0 + 17.04546268 + 16.38502831 + 16.35819360'
+ for n in '$RUNS'
+ partial=0
+ count=0
++ sed -n -e '/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p' CFP2006.038.ref.txt CINT2006.038.ref.txt
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8)'
+ count=1
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6)'
+ count=2
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.4)'
+ count=3
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.4) + l(19.6)'
+ count=4
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.4) + l(19.6) + l(11.6)'
+ count=5
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.4) + l(19.6) + l(11.6) + l(11.8)'
+ count=6
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.6) + l(17.4) + l(19.6) + l(11.6) + l(11.8) + l(17.9)'
+ count=7
+ [[ 0 + l(14.8) + l(23.6) + l(17.4) + l(19.6) + l(11.6) + l(11.8) + l(17.9) != 0 ]]
++ echo 'scale=8; e((0 + l(14.8) + l(23.6) + l(17.4) + l(19.6) + l(11.6) + l(11.8) + l(17.9)) / 7)'
++ bc -l
+ SUM='0 + 17.04546268 + 16.38502831 + 16.35819360 + 16.19236524'
+ for n in '$RUNS'
+ partial=0
+ count=0
++ sed -n -e '/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p' CFP2006.039.ref.txt CINT2006.039.ref.txt
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8)'
+ count=1
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2)'
+ count=2
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(18.3)'
+ count=3
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(18.3) + l(19.9)'
+ count=4
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(18.3) + l(19.9) + l(11.8)'
+ count=5
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(18.3) + l(19.9) + l(11.8) + l(11.9)'
+ count=6
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(18.3) + l(19.9) + l(11.8) + l(11.9) + l(18.2)'
+ count=7
+ [[ 0 + l(14.8) + l(23.2) + l(18.3) + l(19.9) + l(11.8) + l(11.9) + l(18.2) != 0 ]]
++ echo 'scale=8; e((0 + l(14.8) + l(23.2) + l(18.3) + l(19.9) + l(11.8) + l(11.9) + l(18.2)) / 7)'
++ bc -l
+ SUM='0 + 17.04546268 + 16.38502831 + 16.35819360 + 16.19236524 + 16.40349218'
+ for n in '$RUNS'
+ partial=0
+ count=0
++ sed -n -e '/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p' CFP2006.040.ref.txt CINT2006.040.ref.txt
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8)'
+ count=1
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.5)'
+ count=2
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.5) + l(17.3)'
+ count=3
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.5) + l(17.3) + l(19.5)'
+ count=4
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.5) + l(17.3) + l(19.5) + l(11.9)'
+ count=5
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.5) + l(17.3) + l(19.5) + l(11.9) + l(11.9)'
+ count=6
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.5) + l(17.3) + l(19.5) + l(11.9) + l(11.9) + l(18.3)'
+ count=7
+ [[ 0 + l(14.8) + l(23.5) + l(17.3) + l(19.5) + l(11.9) + l(11.9) + l(18.3) != 0 ]]
++ echo 'scale=8; e((0 + l(14.8) + l(23.5) + l(17.3) + l(19.5) + l(11.9) + l(11.9) + l(18.3)) / 7)'
++ bc -l
+ SUM='0 + 17.04546268 + 16.38502831 + 16.35819360 + 16.19236524 + 16.40349218 + 16.28736216'
+ for n in '$RUNS'
+ partial=0
+ count=0
++ sed -n -e '/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p' CFP2006.041.ref.txt CINT2006.041.ref.txt
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8)'
+ count=1
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.3)'
+ count=2
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.3) + l(17.5)'
+ count=3
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.3) + l(17.5) + l(19.4)'
+ count=4
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.3) + l(17.5) + l(19.4) + l(11.8)'
+ count=5
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.3) + l(17.5) + l(19.4) + l(11.8) + l(12.1)'
+ count=6
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.3) + l(17.5) + l(19.4) + l(11.8) + l(12.1) + l(19.9)'
+ count=7
+ [[ 0 + l(14.8) + l(23.3) + l(17.5) + l(19.4) + l(11.8) + l(12.1) + l(19.9) != 0 ]]
++ echo 'scale=8; e((0 + l(14.8) + l(23.3) + l(17.5) + l(19.4) + l(11.8) + l(12.1) + l(19.9)) / 7)'
++ bc -l
+ SUM='0 + 17.04546268 + 16.38502831 + 16.35819360 + 16.19236524 + 16.40349218 + 16.28736216 + 16.49777609'
+ for n in '$RUNS'
+ partial=0
+ count=0
++ sed -n -e '/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p' CFP2006.042.ref.txt CINT2006.042.ref.txt
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8)'
+ count=1
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2)'
+ count=2
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(17.4)'
+ count=3
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(17.4) + l(19.6)'
+ count=4
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(17.4) + l(19.6) + l(12.0)'
+ count=5
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(17.4) + l(19.6) + l(12.0) + l(11.8)'
+ count=6
+ for b in '`sed -n -e '\''/^=\+$/,/^ Est. SPEC/!d; s/[0-9]\{3\}\.\w\+\s\+[0-9]\+\s\+[0-9]\+\s\+\([0-9.]\+\)\s\+\*/\1/p'\'' *.$n.*txt 2>/dev/null`'
+ partial='0 + l(14.8) + l(23.2) + l(17.4) + l(19.6) + l(12.0) + l(11.8) + l(17.8)'
+ count=7
+ [[ 0 + l(14.8) + l(23.2) + l(17.4) + l(19.6) + l(12.0) + l(11.8) + l(17.8) != 0 ]]
++ echo 'scale=8; e((0 + l(14.8) + l(23.2) + l(17.4) + l(19.6) + l(12.0) + l(11.8) + l(17.8)) / 7)'
++ bc -l
+ SUM='0 + 17.04546268 + 16.38502831 + 16.35819360 + 16.19236524 + 16.40349218 + 16.28736216 + 16.49777609 + 16.21830495'
++ echo 'scale=2; (0 + 17.04546268 + 16.38502831 + 16.35819360 + 16.19236524 + 16.40349218 + 16.28736216 + 16.49777609 + 16.21830495)/1'
++ bc
+ SUM=131.38
+ echo 'Final result: 131.38'
Final result: 131.38