<!-- keep this as a security measure: * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.LCGAdminGroup * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.LCGAdminGroup #uncomment this if you want the page only be viewable by the internal people #* Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.LCGAdminGroup --> ---+ Phoenix Hardware and OS related services description Here are described the infrastructure services run on, together with procedures related to them, drivers, incompatibilities, monitoring, testing and other useful information. We will have two sections: Hardware Cards and OS Features. ---++ Hardware Cards Here is a list of the Hardware Cards we have filled in so far. *You can add new ones in HardwareCards*. %SEARCH{ "META\:TOPICPARENT.*\"HardwareCards\"" regex="on" casesensitive="on" order="created" reverse="on" nosearch="on" nototal="on" header="| *Page* | *Name* | *Used for* | *N. in Production* | *OS Inside* | *First puchase* | *CPU (HS06)* | *Disk (MB/s)* | *Power (Watts)* | *Last modified* |" format="| [[$topic]] | $formfield(Manufacturer) $formfield(Model name) | $formfield(Used for) | $formfield(Number in production) | $formfield(You install OS inside) | $formfield(First purchase date) | $formfield(CPU performance in HS06) | $formfield(Disk performance in MB/s) | $formfield(Power consumption in Watts) | $date |" }% ---++ Other Hardware or OS related documents ---+++ Hardware control. The ILOM network is in a separate network (192.168.*). You can access it through the Xen hosts (i.e. Xen12) and perform ON/OFF commands with =ion=, =ioff= or =ireset= ---+++ Infiniband + VNIC manual setup To set the VNIC on any machine, we need to install all the mlx4 packages available from the MLNX ISO and configure a few files: 1. If the kernel and/or the SL version, download the right ISO file from the Mellanox website (http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=26&menu_section=34#tab-three). 1. Do the following steps:<verbatim> mount -o loop ./MLNX_OFED_LINUX-1.5.3-3.0.0-rhel6-x86_64.iso /mnt cp -r /mnt/ ./mlnx # MAKE SURE .mlnx is also copied umount /mnt cd mlnx/docs vim mlnx_add_kernel_support.sh # add this so the script detects SL as redhat: # sl-release-*6.0*| ./mlnx_add_kernel_support.sh --iso /root/MLNX_OFED_LINUX-1.5.3-3.0.0-rhel6-x86_64.iso # Once it is finished, the new image is created in /tmp, replace the one we downloaded with it: mv /tmp/MLNX_OFED_LINUX-1.5.3-3.0.0-rhel6-x86_64.iso /root/ mount -o loop ./MLNX_OFED_LINUX-1.5.3-3.0.0-rhel6-x86_64.iso /mnt /mnt/mlnxofedinstall # Done! </verbatim> 1. Once these steps are done, *reboot* the machine. 1. If the package =infiniband-diags= is not installed, install it from the iso image just created. This package provides utilities that we can use to test the status of the infiniband fabric. In particular, =ibstat=, =ibstatus=, =ibhosts=, =ibswitches= and =ibping= are really useful. 1. At this point, the infiniband card should be up and running if the =ifcfg-ib0= file is correct and the fabric is properly connected. 1. To install the VNIC, modify =/etc/infiniband/openib.conf= <verbatim>sed -i -e 's/MLX4_VNIC_LOAD=no/MLX4_VNIC_LOAD=yes/' /etc/infiniband/openib.conf</verbatim> And create =/etc/infiniband/mlx4_vnic.conf= as follows (in this example, eth0 has the 10.10 IP): <verbatim>VMAC1=`ifconfig |grep eth0|awk '{print $5}'|awk --field-separator ":" '{print $1}'` VMAC2=`ifconfig |grep eth0|awk '{print $5}'|awk --field-separator ":" '{print $2}'` VMAC3=`ifconfig |grep eth0|awk '{print $5}'|awk --field-separator ":" '{print $3}'` VMAC4=`ifconfig |grep eth0|awk '{print $5}'|awk --field-separator ":" '{print $4}'` VMAC5=`ifconfig |grep eth0|awk '{print $5}'|awk --field-separator ":" '{print $5}'` VMAC6=`ifconfig |grep eth0|awk '{print $5}'|awk --field-separator ":" '{print $6}'` IP=`/sbin/ifconfig eth0 | grep 'inet addr:' | awk ' { print $2 } ' | cut -d. -f4-` SN=`expr $IP + 100` echo "name=eth2 mac=00:00:${VMAC3}:${VMAC4}:${VMAC5}:${VMAC6} ib_port=mlx4_0:1 vnic_id=${SN} bx=00:02:C9:03:00:11:11:77 eport=A10" > /etc/infiniband/mlx4_vnic.conf</verbatim> 1. Then, create the proper file =ifcfg-eth2= and the VNIC should be working well. ---++ This page's children (to organize) %SEARCH{ "META\:TOPICPARENT.*\"%TOPIC%\"" regex="on" casesensitive="on" order="created" reverse="on" nosearch="on" nototal="on" header="| *Page* | *Title* | *Date created* | *Date last modified* |" format="| [[$topic]] | [[$topic][ $pattern(.*---\+!*\s+([^\n\r]+).*) ]] | $createdate | $date |" }% -- Main.PabloFernandez - 2011-01-20
This topic: LCGTier2
>
WebHome
>
HardwareInformation
Topic revision: r13 - 2013-02-22 - PabloFernandez
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback