Tags:
tag this topic
create new tag
view all tags
<!-- keep this as a security measure: * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.LCGAdminGroup * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.LCGAdminGroup #uncomment this if you want the page only be viewable by the internal people #* Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.LCGAdminGroup --> ---+ Lustre Oss hardware ---++Using 3 Virident cards, 2 as metadataOnly, 1 as dataAndMetadata - 256K blocksize <verbatim> Sep 14 11:19 [root@wn120:tmpgpfs]# dd if=/dev/zero of=test bs=4096k count=10000 10000+0 records in 10000+0 records out 41943040000 bytes (42 GB) copied, 37.7524 seconds, 1.1 GB/s Sep 14 11:20 [root@wn120:tmpgpfs]# time tar xf linux-2.6.36.1.tar real 0m13.559s user 0m0.267s sys 0m3.727s make real 8m12.997s user 6m56.126s sys 0m58.973s </verbatim> ---++Using 3 virident cards as metadata + 3 raids, 2 on oss11, 1 on oss12 - 256K blocksize <verbatim> Sep 14 12:01 [root@wn120:tmpgpfs]# dd if=/dev/zero of=test bs=4096k count=10000 10000+0 records in 10000+0 records out 41943040000 bytes (42 GB) copied, 232.677 seconds, 180 MB/s Sep 14 12:27 [root@wn120:tmpgpfs]# time tar xf linux-2.6.36.1.tar real 0m37.709s user 0m0.051s sys 0m0.773s time make real 8m26.329s user 6m56.563s sys 0m58.123s dd test on three hosts Sep 14 12:42 [root@xen11:~]# dsh -w wn[120-122] /tmpgpfs/testgpfs.sh wn120: 10000+0 records in wn120: 10000+0 records out wn120: 41943040000 bytes (42 GB) copied, 666.36 seconds, 62.9 MB/s wn120: wn120: real 11m6.362s wn120: user 0m0.025s wn120: sys 0m20.165s wn121: 10000+0 records in wn121: 10000+0 records out wn121: 41943040000 bytes (42 GB) copied, 666.798 seconds, 62.9 MB/s wn121: wn121: real 11m6.800s wn121: user 0m0.011s wn121: sys 0m18.103s wn122: 10000+0 records in wn122: 10000+0 records out wn122: 41943040000 bytes (42 GB) copied, 671.016 seconds, 62.5 MB/s wn122: wn122: real 11m11.017s wn122: user 0m0.011s wn122: sys 0m17.614s </verbatim> on ten hosts, it's about 10 MB/s ---+ Xen16 machine ---++ dd <verbatim> [root@xen16 ~]# dd if=/dev/zero of=/virident/thing bs=8k count=1024k 1048576+0 records in 1048576+0 records out 8589934592 bytes (8.6 GB) copied, 13.6994 s, 627 MB/s [...] do some stuff [root@xen16 ~]# dd if=/virident/thing of=/dev/null ^[[6~16777216+0 records in 16777216+0 records out 8589934592 bytes (8.6 GB) copied, 18.8337 s, 456 MB/s </verbatim> ---++ Bonnie <verbatim> [root@xen16 ~]# bonnie++ -u root -s 60000 -r 0 -b -d /virident/ Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP xen16.lcg.cs 60000M 733 99 530557 94 298348 78 3522 99 722689 88 1078 112 Latency 11048us 14500us 194ms 2686us 4304us 5686us Version 1.96 ------Sequential Create------ --------Random Create-------- xen16.lcg.cscs.ch -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 1599 6 +++++ +++ 2253 7 1777 8 +++++ +++ 2248 6 Latency 557ms 375us 4239us 6312us 17us 5872us </verbatim> ---++ Kernel compile <verbatim> [root@xen16 virident]# time tar -xf linux-2.6.36.1.tar real 0m1.828s user 0m0.123s sys 0m1.706s [root@xen16 linux-2.6.36.1]# time make real 7m53.340s user 6m46.717s sys 1m18.671s </verbatim> ---+ KVM over local Virident card (xen16) ---++ dd <verbatim> [root@xen16 ~]# dd if=/dev/zero of=/root/thing bs=4k count=1024k 1048576+0 records in 1048576+0 records out 4294967296 bytes (4.3 GB) copied, 25.7644 seconds, 167 MB/s [...] do some stuff [root@xen16 ~]# dd if=/root/thing of=/dev/null 8388608+0 records in 8388608+0 records out 4294967296 bytes (4.3 GB) copied, 12.177 seconds, 353 MB/s </verbatim> ---++ Bonnie <verbatim> [root@xen16 ~]# bonnie++ -u root -s 6000 -r 0 -b -d /root/ Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP ppwn04.lcg.cs 6000M 870 99 139143 47 93778 37 3428 97 395061 38 3353 238 Latency 10000us 1418ms 828ms 3999us 10999us 73999us Version 1.96 ------Sequential Create------ --------Random Create-------- ppwn04.lcg.cscs.ch -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 905 3 +++++ +++ 890 2 888 4 +++++ +++ 892 2 Latency 30999us 1000us 21000us 32999us 1000us 16000us </verbatim> ---++ Kernel compile <verbatim> [root@xen16 virident]# time tar -xf linux-2.6.36.1.tar real 0m3.740s user 0m0.174s sys 0m2.973s [root@xen16 linux-2.6.36.1]# time make </verbatim> ---+ KVM over remote Virident card mounted on NFS through 1 gigabit interface on each machine (xen16-xen04) ---++ Bonnie <verbatim> Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP ppwn03.lcg.c 11000M 753 98 43095 11 28484 11 2881 97 130806 25 1053 88 Latency 13000us 2539ms 1990ms 8000us 295ms 299ms Version 1.96 ------Sequential Create------ --------Random Create-------- ppwn03.lcg.cscs.ch -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 333 1 +++++ +++ 318 0 346 1 +++++ +++ 321 0 Latency 273ms 1000us 213ms 28000us 1000us 408ms </verbatim> -- Main.PabloFernandez - 2011-09-05
E
dit
|
A
ttach
|
Watch
|
P
rint version
|
H
istory
: r6
<
r5
<
r4
<
r3
<
r2
|
B
acklinks
|
V
iew topic
|
Ra
w
edit
|
M
ore topic actions
Topic revision: r6 - 2011-09-14
-
JasonTemple
LCGTier2
Log In
(Topic)
LCGTier2 Web
Create New Topic
Index
Search
Changes
Notifications
Statistics
Preferences
Users
Entry point / Contact
RoadMap
ATLAS Pages
CMS Pages
CMS User Howto
CHIPP CB
Outreach
Technical
Cluster details
Services
Hardware and OS
Tools & Tips
Monitoring
Logs
Maintenances
Meetings
Tests
Issues
Blog
Home
Site map
CmsTier3 web
LCGTier2 web
PhaseC web
Main web
Sandbox web
TWiki web
LCGTier2 Web
Users
Groups
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
P
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Warning: Can't find topic "".""
Account
Log In
E
dit
A
ttach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback