HardwareDS3700 < LCGTier2

Tags: view all tags

<!-- keep this as a security measure:
   * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.LCGAdminGroup
   * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.LCGAdminGroup
   #uncomment this if you want the page only be viewable by the internal people
   #* Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.LCGAdminGroup
-->

---+ Hardware Card for IBM DCS3700

Short description about the hardware.

%TOC%

---++ Specifications

What's inside the hardware, probably depending on what we're talking about.
   * Disk controller model. Linux driver needed? *Drives connected over FC with multipath.*
   * Network card model<strong> 2x 8Gb FC Dual HBA <br /></strong>
   * Space that takes in the rack (U's): *4U*
   * Max CPUs, Max Disks, Max Mem slots. *currently 60x 3TB HDD*
   * CDROM? *No*
   * Dual power supply? *Yes*
   * Front / Back picture (maybe available online?)
   * Any other?
Controller IBM System x3650 M4

   * Disk controller model. Linux driver needed? *LSI 2208 SAS controller, installs fine with standard SL*
   * Network card model<strong> 2x Intel I350 gigabit adapter and 2x QLogic QLE2562 - QLogic 8Gb FC Dual-port HBA for System x</strong>
   * Space that takes in the rack (U's): *2U*
   * Max CPUs, Max Disks, Max Mem slots. *2x Xeon E5, currently 2x Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz. Up to 384 GB DDR-3 LRDIMM memory via 12 DIMM slots, currently 64GB (8x 8GB DIMMS).*
   * CDROM? *No*
   * Dual power supply? *Yes*
   * Front / Back picture (maybe available online?)
   * Any other?

---+++ Power consumption (measured before and during the CPU/Disk tests)
   * Minimum
   * Maximum

---+++ Ambiental details
   * External working temperature
   * Normal CPU/memory internal temperature
   * Air flow (cubic meters per hour)
   * Noise (dB)

---++ Operations

Interesting information like how to handle it, anything interesting which is not trivial.
---+++ How to get into the ILOM
---+++ Power up/down procedures
---+++ Commands to issue in an internal console
---+++ Firmware updates
---+++ Replacement of internal components

---++ Installation notes

Instructions on how to set up a new machines that arrives, with things like:

<span style="text-decoration: underline;"> *Setting up multipath* </span>

Copy a multipath.conf from another machine

=scp se11:/etc/multipath.conf /etc/=

Ensure mulipathd is runninig.

=service multipathd start=

=chkconfig multipathd on=

Run a "multipath -ll". Ensure that you see the WWIDs that corrospond to the attached storage, this information can be found within SMclient.

=multipath -ll=

=phaseG_storage_8_array4 (360080e50002e3fea0000039d507668fc) dm-2 IBM,1818 FAStT=

=size=22T features='0' hwhandler='1 rdac' wp=rw<br />|-+- policy='round-robin 0' prio=6 status=active<br />| `- 2:0:0:4 sdq 65:0 active ready running<br />`-+- policy='round-robin 0' prio=1 status=enabled<br /> `- 1:0:0:4 sdk 8:160 active ghost running=

=...=

Update the multipath.conf with the correct WWIDs and aliases.Be sure to copy this back into cfengine under the appropriate hosts.

Check that dev mapper has detected the storage.

=Feb 14 13:06 [root@se12:~]# ll /dev/mapper/=

Proceed to create LVM volumes and format as normal but ensure that the LVM.conf only allows for acitvation of one set of stoarge per se server. E.G. on se11 =volume_list = [ "vg_root", "phaseG_storage_7" ]= whereas on se12 =volume_list = [ "vg_root", "phaseG_storage_8" ]=
---+++ BIOS settings
---+++ RAID configuration

Disks array grouped into RAID6 arrarys consisting of 10 disks. There are 6 arrays within each DS3700 enclosure.
---+++ Drivers required / kernel compatibility
---+++ Check firmware homogeneity with other machines in the cluster

---++ Benchmarks

Information about benchmarks performed in the machine.
---+++ Disk benchmarks (i.e. Bonnie++)

Example: 
%EDITTABLE{}%
| Version 1.96 | | *Sequential Output* |||||| *Sequential Input* |||| *Random* ||
| Concurrency | 1 | *Per Chr* || *Block* || *Rewrite* || *Per Chr* || *Block* || *Seeks* ||
| *Machine* | *Size* | *K/sec* | *%CP* | *K/sec* | *%CP* | *K/sec* | *%CP* | *K/sec* | *%CP* | *K/sec* | *%CP* | */sec* | *%CP* |
| xfs | 100000M | 1673 | 98 | 258522 | 32 | 164738 | 23 | 3401 | 99 | 455836 | 29 | 518.8 | 13 |
| *Latency* || 4954us | | 314ms | | 231ms | | 7143us | | 177ms | | 74101us | |
| ext4 | 100000M | 927 | 98 | 256154 | 46 | 159501 | 19 | 3983 | 99 | 530509 | 35 | 400.6 | 34 |
| *Latency* || 9114us | | 206ms | | 599ms | | 2388us | | 150ms | | 627ms | |

---+++ CPU benchmarks (i.e. HepSpec06)
---+++ Network benchmarks (i.e. Iperf)

---++ Monitoring

Instructions about monitoring the hardware
---+++ Power Consumption
---+++ Raid Sanity
---+++ Other?

---++ Manuals

External links to manuals

---++ Issues

Information about issues found with this hardware, and how to deal with them
---+++ Issue1
---+++ Issue2

   * [[%ATTACHURL%/PhaseG_DCS3700_connections.pdf][PhaseG_DCS3700_connections.pdf]]: PhaseG DCS3700 connections to IO servers

   * [[%ATTACHURL%/PhaseG_DCS3700_connections.vsd][PhaseG_DCS3700_connections.vsd]]: PhaseG DCS3700 connections to IO servers

HardwareCardForm
Model name	DCS3700
Manufacturer	IBM
Used for	dCache
Number in production	6
You install OS inside	No
First purchase date	12.2012
CPU performance in HS06
Disk performance in MB/s
Power consumption in Watts

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
pdf	PhaseG_DCS3700_connections.pdf	r1	manage	168.3 K	2012-12-07 - 13:47	MiguelGila	PhaseG DCS3700 connections to IO servers
vsd	PhaseG_DCS3700_connections.vsd	r1	manage	91.5 K	2012-12-07 - 13:47	MiguelGila	PhaseG DCS3700 connections to IO servers