Tags:
tag this topic
create new tag
view all tags
<!-- keep this as a security measure: * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.LCGAdminGroup * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.LCGAdminGroup #uncomment this if you want the page only be viewable by the internal people #* Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.LCGAdminGroup --> Keywords: vmware, esx, esxi, console, ssh, infiniband ---+ Working with VMWare ESXi 4 %TOC% ---++ Setting up a VMWare ESXi 4 host * Installation with the ILOM * in the ILOM, redirect the installation cdrom (/extras/VMware-VMvisor-Installer-4.1.0-260247.x86_64.iso) and reboot the machine =icdrom xx, ireset xx= * Follow the installation, it's just continue-continue. * When you see the welcome screen press F2 to customize it. * The first password is empty. Set it in the menu. * Check that the management network is correct (it should be in the 10.10 network, taken from DHCP) * Enter also Troubleshooting Options and enable Remote Tech Support (SSH) * You can exit the ILOM, the basic setup is done. * Keep in mind the *security* if you assign a public IP to the admin interface. There is more than just the ssh interface. * Copy the needed files into the ESXi host (drivers and script wrapper, check attachments) from *xen02*<verbatim>scp /extra/MEL-OFED-1.4.1-375-offline_bundle.zip 10.10.66.90:/scratch/ scp /extra/vm 10.10.66.90:/scratch/</verbatim> * Log in the ESXi Host to make some changes <verbatim>ssh 10.10.66.90</verbatim> * Enable IPoIB Infiniband support drivers <verbatim>ssxupdate --bundle /scratch/MEL-OFED-1.4.1-375-offline_bundle.zip update</verbatim> * Activate the license with this (see VmwareEsxiLicense)<verbatim>vim-cmd vimsvc/license --set XXXXX-XXXXX-XXXXX-XXXXX-XXXXX</verbatim> * Disallow passord login <verbatim>sed 's/shell -i/shell -i -s/' -i /etc/inetd.conf</verbatim> * Put some private keys to be able to ssh into the machine <verbatim>echo '# Enable pubkey login mkdir /.ssh chmod 700 /.ssh echo "ssh-dss XXX " > /.ssh/authorized_keys ' > /etc/rc.local vi /etc/rc.local</verbatim> * Enable auto-start and soft-shutdown of guests by default <verbatim>vim-cmd hostsvc/autostartmanager/enable_autostart 1 vim-cmd hostsvc/autostartmanager/update_defaults 120 120 GuestShutdown 0</verbatim> * Backup the actual config to make it work in the next bootup. Do it twice! (does not copy everything in the first run) <verbatim>/sbin/auto-backup.sh /sbin/auto-backup.sh</verbatim> * Check =rc.local= and =inetd.conf= were backed up. Then Reboot the machine, and log in again <verbatim>reboot</verbatim> ---+++ Configure infiniband networking * Check the actual network configuration in the host <verbatim>esxcfg-nics -l</verbatim> * Enable IPoIB in both infiniband ports<verbatim>esxcfg-module -s "port_types=24,1,1,24,1,1 port_type_default=1" mlx4_en reboot</verbatim> * Verify that both ports are visible <verbatim>esxcfg-nics -l</verbatim> * Create a virtual Switch with one of the ports, for all the virtual machines to use it (involves creating a vSwitch, linking a device to it, and create a PortGroup with it)<verbatim>esxcfg-vswitch -a vSwitch1 esxcfg-vswitch -L vmnic18.p1 vSwitch1 esxcfg-vswitch -A ib0 vSwitch1 esxcfg-vswitch -l</verbatim> At the end of this process, there will be a PortGroup called *ib0* where we can plug in our virtual machines. Everything should be ready now, you can proceed creating guests! ---++ Operations with VM Guests ---+++ Create * In the Host, go to <verbatim>cd /vmfs/volumes/datastore1</verbatim> * There you should see all VM Guest filesystems, check also the available size<verbatim>ls df -h| egrep "vmfs3|Used"</verbatim> * Create your own directory<verbatim>VM=testmachine mkdir $VM cd $VM</verbatim> * Create a virtual disk<verbatim>vmkfstools -c 12G -a lsilogic $VM.vmdk</verbatim> * Create the VM definition file with =vi $VM.vmx= and put inside<verbatim> config.version = "8" virtualHW.version= "7" guestOS = "rhel5-64" memsize = "1024" displayname = "@VM@" scsi0.present = "TRUE" scsi0.virtualDev = "lsilogic" scsi0:0.fileName = "@VM@.vmdk" scsi0:0.present = "TRUE" ethernet0.address = "@MAC@" ethernet0.addressType = "static" ethernet0.networkName = "VM Network" ethernet0.present = "TRUE" ethernet0.virtualDev = "e1000" ethernet1.addressType = "generated" ethernet1.networkName = "ib0" ethernet1.present = "TRUE" ethernet1.virtualDev = "vmxnet3"</verbatim> * Replace the values inside<verbatim>sed "s/@VM@/$VM/" -i $VM.vmx sed "s/@MAC@/00:50:56:68:02:02/" -i $VM.vmx chmod 744 $VM.vmx</verbatim> * Activate the VM (make it visible for ESXi) and vereify it<verbatim>vim-cmd solo/registervm /vmfs/volumes/datastore1/$VM/$VM.vmx $VM vim-cmd vmsvc/getallvms</verbatim> * Later on you should install the vmware tools from inside the guest =# /root/vmware-install.pl --default= ---+++ Power control * Check state<verbatim>vim-cmd vmsvc/power.getstate 16</verbatim> * Power on<verbatim>vim-cmd vmsvc/power.on 16</verbatim> * Shutdown<verbatim>vim-cmd vmsvc/power.shutdown 16</verbatim> * Hard stop<verbatim>vim-cmd vmsvc/power.off 16</verbatim> * Soft reboot<verbatim>vim-cmd vmsvc/power.reboot 16</verbatim> * Reset<verbatim>vim-cmd vmsvc/power.reset 16</verbatim> ---+++ Other operations * List available guests<verbatim>vim-cmd vmsvc/getallvms</verbatim> * Get extra online information from VM<verbatim>vim-cmd vmsvc/get.summary 16</verbatim> * Install VMWare Tools (don't use it here, we do it in another way)<verbatim>vim-cmd vmsvc/tools.install 16</verbatim> * Reload changes done in the =.vmx= file<verbatim>vim-cmd vmsvc/reload 16</verbatim> * Unregister a machine (run before removing the files!) <verbatim>vim-cmd vmsvc/unregister 16</verbatim> * Check running machines CPU usage<verbatim>esxtop</verbatim> * See the system console (what you see from the ILOM)<verbatim>dcui</verbatim> <verbatim></verbatim> <verbatim></verbatim> <verbatim></verbatim> ---++ Working with the GUI Some times you would like to work with the User Interface (it's called vSphere Client) * Identify the ID of the windows VM guest on xen02 (with xm list. In this example, id=11) * Open a tunnel from your workstation to route VNC traffic =ssh root@xen02 -L 12345:localhost:5911= (change the 11 for the ID from before) * Run in your desktop =vncviewer localhost:12345= * Start the VMWare vSphere Client, and open the session to the 10.10 management interface of the host. ---++ Working with the console You need to keep in mind that the root filesystem is overwritten every time you reboot. To make changes permanent you need to edit an already existing file in /etc (for example) and run =/sbin/auto-backup.sh=. If you want to add actions to the boot procedure, edit the =/etc/rc.local= and do it permanent. Also, you can use =/sbin/services.sh restart= to restart all enabled services, specified in =/etc/chkconfig.db=. This does not affect the virtual machine guests. ---+++ External links There are some pages on how to manage ESXi from the command line: * http://www.vm-help.com/esx40i/manage_without_VI_client_1.php (main page links you to some more tips: http://www.vm-help.com/) * http://robertbchase.blogspot.com/2008/12/vmware-esxi-ssh-cli-commands.html * http://myitstuff.tumblr.com/post/350186035/how-to-control-esxi-virtual-machine-from-a-command-line * http://benincosa.org/blog/?p=266 * http://www-01.ibm.com/support/docview.wss?uid=isg3T1012138 * http://www.virtuallyghetto.com/ And maybe, at some point, you want to use this, but probably not: http://www.vmware.com/support/developer/vima/ -- Main.PabloFernandez - 2010-12-02
Attachments
Attachments
Topic attachments
I
Attachment
History
Action
Size
Date
Who
Comment
zip
MEL-OFED-1.4.1-375-offline_bundle.zip
r1
manage
443.6 K
2010-12-10 - 15:06
PabloFernandez
E
dit
|
A
ttach
|
Watch
|
P
rint version
|
H
istory
: r9
<
r8
<
r7
<
r6
<
r5
|
B
acklinks
|
V
iew topic
|
Ra
w
edit
|
M
ore topic actions
Topic revision: r9 - 2011-01-24
-
PabloFernandez
LCGTier2
Log In
(Topic)
LCGTier2 Web
Create New Topic
Index
Search
Changes
Notifications
Statistics
Preferences
Users
Entry point / Contact
RoadMap
ATLAS Pages
CMS Pages
CMS User Howto
CHIPP CB
Outreach
Technical
Cluster details
Services
Hardware and OS
Tools & Tips
Monitoring
Logs
Maintenances
Meetings
Tests
Issues
Blog
Home
Site map
CmsTier3 web
LCGTier2 web
PhaseC web
Main web
Sandbox web
TWiki web
LCGTier2 Web
Users
Groups
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
P
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Warning: Can't find topic "".""
Account
Log In
E
dit
A
ttach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback