WNsSuperMicro < CmsTier3

<!-- keep this as a security measure:
   #uncomment if the subject should only be modifiable by the listed groups 
   * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup
   * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup
   #uncomment this if you want the page only be viewable by the listed groups
   # * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup
-->


---+!! Node Type: %CALC{"$SUBSTITUTE(%TOPIC%,NodeType,)"}%
---++!! Firewall requirements

| *local port* | *open to* | *reason* |

<!-- Example line
 #| 22/tcp      | * |  Example entry for ssh |
-->

---

%TOC{title="Table of contents"}%

---+ Regular Maintenance work
---++ Nagios
[[https://t3nagios.psi.ch/nagios/cgi-bin/status.cgi?hostgroup=T3-WNs-SL6&style=detail&limit=0][t3nagios]]

---++ SuperMicro IPMI tools
---+++ In-Band and Out-Band tools
https://www.supermicro.com/products/nfo/SMS_IPMI.cfm
---+++ Out-Band tools on =t3admin=
<pre>
total 187968
lrwxrwxrwx 1 root root       27 Nov 21 14:41 %BLUE%IPMICFG%ENDCOLOR% -> IPMICFG_1.25.0_build.160715
drwxrwx--- 5 root root     4096 Nov 21 14:41 IPMICFG_1.25.0_build.160715
-rw-r----- 1 root root  1571697 Nov 21 12:00 IPMICFG_1.25.0_build.160715.zip
drwxr-x--- 4 root root     4096 Nov 18 22:33 IPMI_FIRMWARE
lrwxrwxrwx 1 root root       48 Nov 21 14:53 %RED%IPMIView%ENDCOLOR% -> IPMIView_2.12.0_build.160804_bundleJRE_Linux_x64
drwxr-x--- 4 root root     4096 Nov 21 15:00 IPMIView_2.12.0_build.160804_bundleJRE_Linux_x64
-rw-r----- 1 root root 91078080 Nov 21 14:43 IPMIView_2.12.0_build.160804_bundleJRE_Linux_x64.tar.gz
drwxr-x--- 2 root root     4096 Nov 18 22:34 IPMIView_2.9.23_jar_20130709
-rw-r----- 1 root root 27155569 Jun 18  2014 IPMIView_2.9.23_jar_20130709.zip
lrwxrwxrwx 1 root root       51 Nov 21 15:06 %RED%SMCIPMITool%ENDCOLOR% -> SMCIPMITool_2.16.0_build.160816_bundleJRE_Linux_x64
drwxr-x--- 4 root root     4096 Nov 21 14:43 SMCIPMITool_2.16.0_build.160816_bundleJRE_Linux_x64
-rw-r----- 1 root root 72440975 Nov 21 14:36 SMCIPMITool_2.16.0_build.160816_bundleJRE_Linux_x64.tar.gz
lrwxrwxrwx 1 root root        7 Nov 21 14:41 %BLUE%to_be_used_locally_on_the_t3wn_servers%ENDCOLOR% -> %BLUE%IPMICFG%ENDCOLOR%
</pre>
---+++ In-Band tool on =t3wn[41-50]=
   * =/opt/SUPERMICRO/IPMICFG= can be used to totally reset the BMC if that's got stuck during an IPMI FW upgrade process ( seldom occurred though )
   * <pre>
[root@t3wn41 64bit]# ./IPMICFG-Linux.x86_64 -fde     
Reset to the factory default completed.
</pre>
   * <pre>
IPMICFG

IPMICFG is an in-band utility for configuring IPMI devices. It is a command line tool providing standard 
IPMI and Supermicro® proprietary OEM commands. This CLI-based utility can be executed on DOS, 
Windows, and Linux OS and does not require any installation procedures. 
This utility is used for BMC/FRU configuration. IPMICFG Key Features:
  - Set up IPMI IP Address
  - Set up IPMI Configuration
  - Configure IPMI User Management
  - Configure IPMI FRU
  - Manage System Event Log (SEL)
  - Manage IPMI by node management (NM) protocol 
</pre>
   * [[%ATTACHURL%/IPMICFG_UserGuide.pdf][IPMICFG_UserGuide.pdf]] 
---+++ Updating the SuperMicro IPMI FW itself ( seldom needed )
   * https://www.thomas-krenn.com/en/wiki/IPMI_Firmware_Update_for_Supermicro_Motherboards_with_ATEN_IPMI_Software
   * http://supermicro.com/support/bios/firmware0.aspx
   * <pre>
# obviously the FW will be newer in your case ; look the screen command
[root@t3wn43 ~]# cd /opt/SUPERMICRO/IPMI_FIRMWARE/SMT_322/Linux\ 1.15/
[root@t3wn43 Linux 1.15]# screen ./lUpdate.sh -f /opt/SUPERMICRO/IPMI_FIRMWARE/SMT_322/SMT_322.bin -i kcs -r n
</pre>

---+ Emergency Measures

<!--
 #List any measures that must be taken in case of some major incident, e.g. whether a mailing
 #list must be contacted or whether other services need to be shut down, etc. 
--> 

The WNs servers run on the top of a =mdadm raid10=, so they can survive to 1 broken disk ; to repair *online* the broken disk simply adapt this procedure NodeTypeWNsIntelS2600JF#Degraded_RAID1_or_Failed_RAID0 to these servers. Generally speaking all the =mdadm= operations apply.

---+ Installation

<!--
 #Comment here on any peculiarities of the installation, e.g. on special packages needed, special setup
 #procedures which are not obvious
--> 
Because of the =mdadm raid10= we should never get to the point where a reinstallation is needed ; anyhow the Puppet recipes are in =/afs/psi.ch/service/linux/puppet/var/puppet/environments/DerekDevelopment/manifests= :
   * =SL6_wn.pp=
   * =SL6.pp=
   * =tier3-baseclasses.pp=
   * and the partition/raid10 layouts are =/afs/psi.ch/software/linux/dist/scientific/66/x86_64/custom/tier3/t3wn/ks-partition.cfg=

Fabio uses these alias :
%TWISTY{ mode="div" }%<pre>
alias ROOT='. /afs/cern.ch/sw/lcg/external/gcc/4.8/x86_64-slc6/setup.sh && . /afs/cern.ch/sw/lcg/app/releases/ROOT/5.34.26/x86_64-slc6-gcc48-opt/root/bin/thisroot.sh'
alias cscsela='ssh -AX fmartine@ela.cscs.ch'
alias cscslogin='ssh -AX fmartine@login.lcg.cscs.ch'
alias cscspub='ssh -AX fmartinelli@pub.lcg.cscs.ch'
alias dcache='ssh -2 -l admin -p 22224 t3dcachedb.psi.ch'
alias dcache04='ssh -2 -l admin -p 22224 t3dcachedb04.psi.ch'
alias gempty='git commit --allow-empty-message -m '\'''\'''
alias kscustom54='cd /afs/psi.ch/software/linux/dist/scientific/54/custom'
alias kscustom57='cd /afs/psi.ch/software/linux/dist/scientific/57/custom'
alias kscustom60='cd /afs/psi.ch/software/linux/dist/scientific/60/custom'
alias kscustom64='cd /afs/psi.ch/software/linux/dist/scientific/64/custom'
alias kscustom66='cd /afs/psi.ch/software/linux/dist/scientific/66/x86_64/custom'
alias ksdir='cd /afs/psi.ch/software/linux/kickstart/configs'
alias ksprepostdir='cd /afs/psi.ch/software/linux/dist/scientific/60/kickstart/bin'
alias l.='ls -d .* --color=auto'
alias ll='ls -l --color=auto'
alias ls='ls --color=tty'
alias mc='. /usr/libexec/mc/mc-wrapper.sh'
alias pdir='cd /afs/psi.ch/service/linux/puppet/var/puppet/environments/DerekDevelopment/'
alias pdirf='cd /afs/psi.ch/service/linux/puppet/var/puppet/environments/FabioDevelopment/'
alias pdirmanifests='cd /afs/psi.ch/service/linux/puppet/var/puppet/environments/DerekDevelopment/manifests/'
alias pdirredhat='cd /afs/psi.ch/service/linux/puppet/var/puppet/environments/DerekDevelopment/modules/Tier3/files/RedHat'
alias pdirsolaris='cd /afs/psi.ch/service/linux/puppet/var/puppet/environments/DerekDevelopment/modules/Tier3/files/Solaris/5.10'
alias vi='vim'
alias which='alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde'
alias yumdir5='cd /afs/psi.ch/software/linux/dist/scientific/57/scripts'
alias yumdir6='cd /afs/psi.ch/software/linux/dist/scientific/6/scripts'
alias yumdir7='cd /afs/psi.ch/software/linux/dist/scientificlinux/7x/x86_64/Tier3/all'
alias yumdir7old='cd /afs/psi.ch/software/linux/dist/scientific/70.PLEASE_DO_NOT_USE_AND_DO_NOT_RENAME/scripts'
</pre>%ENDTWISTY%

---+ Services
Basically only SSHd.
---++ =netstat -tupl=
%TWISTY{mode="div"}%<pre>
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 *:7937                      *:*                         LISTEN      7720/nsrexecd       
tcp        0      0 *:7938                      *:*                         LISTEN      7720/nsrexecd       
tcp        0      0 *:5666                      *:*                         LISTEN      7652/nrpe           
tcp        0      0 *:40291                     *:*                         LISTEN      -                   
tcp        0      0 *:secure-ts                 *:*                         LISTEN      7720/nsrexecd       
tcp        0      0 localhost:smux              *:*                         LISTEN      7188/snmpd          
tcp        0      0 *:37480                     *:*                         LISTEN      -                   
tcp        0      0 *:sge_execd                 *:*                         LISTEN      7170/sge_execd      
tcp        0      0 *:sunrpc                    *:*                         LISTEN      2730/rpcbind        
tcp        0      0 *:8146                      *:*                         LISTEN      7720/nsrexecd       
tcp        0      0 *:ssh                       *:*                         LISTEN      7204/sshd           
tcp        0      0 *:46838                     *:*                         LISTEN      2750/rpc.statd      
tcp        0      0 localhost:x11-ssh-offset    *:*                         LISTEN      10728/sshd          
udp        0      0 *:mdns                      *:*                                     6882/avahi-daemon   
udp        0      0 *:sunrpc                    *:*                                     2730/rpcbind        
udp        0      0 *:46194                     *:*                                     6882/avahi-daemon   
udp        0      0 *:ipp                       *:*                                     2597/portreserve    
udp        0      0 t3wn41.psi.ch:ntp           *:*                                     7214/ntpd           
udp        0      0 localhost:ntp               *:*                                     7214/ntpd           
udp        0      0 *:ntp                       *:*                                     7214/ntpd           
udp        0      0 *:7938                      *:*                                     7720/nsrexecd       
udp        0      0 localhost:syslog            *:*                                     2618/syslog-ng      
udp        0      0 *:781                       *:*                                     2730/rpcbind        
udp        0      0 *:snmp                      *:*                                     7188/snmpd          
udp        0      0 *:54693                     *:*                                     2750/rpc.statd      
udp        0      0 localhost:806               *:*                                     2750/rpc.statd      
udp        0      0 *:48426                     *:*                                     -                   
udp        0      0 *:bootpc                    *:*                                     2540/dhclient       
udp        0      0 *:afs3-callback             *:*                                     -  
</pre>%ENDTWISTY%
<!--
 #List all the important services, their installation, configuration and how to start and stop them
-->

---+ Backups
Not needed.
NodeTypeForm
Hostnames	t3wn[41-50]
Services	Sun Grid Engine 6.2u5 execution hosts
Hardware	SuperMicro 1u got from CSCS
Install Profile	t3wn
Guarantee/maintenance until	out of warranty
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
pdf	IPMICFG_UserGuide.pdf	r1	manage	677.3 K	2016-11-21 - 11:18	FabioMartinelli	SuperMicro In-Band management on Linux
This topic: CmsTier3 > WebHome > AdminArea > WNsSuperMicro
Topic revision: r5 - 2016-12-13 - FabioMartinelli