Keywords: vmware, esx, esxi, console, ssh, infiniband
Working with VMWare ESXi 4
Setting up a VMWare ESXi 4 host
- Installation with the ILOM
- in the ILOM, redirect the installation cdrom (/extras/VMware-VMvisor-Installer-4.1.0-260247.x86_64.iso) and reboot the machine
icdrom xx, ireset xx
- Follow the installation, it's just continue-continue.
- When you see the welcome screen press F2 to customize it.
- The first password is empty. Set it in the menu.
- Check that the management network is correct (it should be in the 10.10 network, taken from DHCP)
- Enter also Troubleshooting Options and enable Remote Tech Support (SSH)
- You can exit the ILOM, the basic setup is done.
- Keep in mind the security if you assign a public IP to the admin interface. There is more than just the ssh interface.
- Copy the needed files into the ESXi host (drivers and script wrapper, check attachments) from *xen02*
scp /extra/MEL-OFED-1.4.1-375-offline_bundle.zip 10.10.66.90:/scratch/
scp /extra/vm 10.10.66.90:/scratch/
- Log in the ESXi Host to make some changes
ssh 10.10.66.90
- Enable IPoIB Infiniband support drivers
ssxupdate --bundle /scratch/MEL-OFED-1.4.1-375-offline_bundle.zip update
- Activate the license with this (see VmwareEsxiLicense)
vim-cmd vimsvc/license --set XXXXX-XXXXX-XXXXX-XXXXX-XXXXX
- Disallow passord login
sed 's/shell -i/shell -i -s/' -i /etc/inetd.conf
- Put some private keys to be able to ssh into the machine
echo '# Enable pubkey login
mkdir /.ssh
chmod 700 /.ssh
echo "ssh-dss XXX
" > /.ssh/authorized_keys
' > /etc/rc.local
vi /etc/rc.local
- Enable auto-start and soft-shutdown of guests by default
vim-cmd hostsvc/autostartmanager/enable_autostart 1
vim-cmd hostsvc/autostartmanager/update_defaults 120 120 GuestShutdown 0
- Backup the actual config to make it work in the next bootup. Do it twice! (does not copy everything in the first run)
/sbin/auto-backup.sh
/sbin/auto-backup.sh
- Check
rc.local
and inetd.conf
were backed up. Then Reboot the machine, and log in again reboot
Configure infiniband networking
- Check the actual network configuration in the host
esxcfg-nics -l
- Enable IPoIB in both infiniband ports
esxcfg-module -s "port_types=24,1,1,24,1,1 port_type_default=1" mlx4_en
reboot
- Verify that both ports are visible
esxcfg-nics -l
- Create a virtual Switch with one of the ports, for all the virtual machines to use it (involves creating a vSwitch, linking a device to it, and create a PortGroup with it)
esxcfg-vswitch -a vSwitch1
esxcfg-vswitch -L vmnic18.p1 vSwitch1
esxcfg-vswitch -A ib0 vSwitch1
esxcfg-vswitch -l
At the end of this process, there will be a
PortGroup called
ib0 where we can plug in our virtual machines. Everything should be ready now, you can proceed creating guests!
Operations with VM Guests
Create
- In the Host, go to
cd /vmfs/volumes/datastore1
- There you should see all VM Guest filesystems, check also the available size
ls
df -h| egrep "vmfs3|Used"
- Create your own directory
VM=testmachine
mkdir $VM
cd $VM
- Create a virtual disk
vmkfstools -c 12G -a lsilogic $VM.vmdk
- Create the VM definition file with
vi $VM.vmx
and put inside
config.version = "8"
virtualHW.version= "7"
guestOS = "rhel5-64"
memsize = "1024"
displayname = "@VM@"
scsi0.present = "TRUE"
scsi0.virtualDev = "lsilogic"
scsi0:0.fileName = "@VM@.vmdk"
scsi0:0.present = "TRUE"
ethernet0.address = "@MAC@"
ethernet0.addressType = "static"
ethernet0.networkName = "VM Network"
ethernet0.present = "TRUE"
ethernet0.virtualDev = "e1000"
ethernet1.addressType = "generated"
ethernet1.networkName = "ib0"
ethernet1.present = "TRUE"
ethernet1.virtualDev = "vmxnet3"
- Replace the values inside
sed "s/@VM@/$VM/" -i $VM.vmx
sed "s/@MAC@/00:50:56:68:02:02/" -i $VM.vmx
chmod 744 $VM.vmx
- Activate the VM (make it visible for ESXi) and vereify it
vim-cmd solo/registervm /vmfs/volumes/datastore1/$VM/$VM.vmx $VM
vim-cmd vmsvc/getallvms
- Later on you should install the vmware tools from inside the guest
# /root/vmware-install.pl --default
Power control
- Check state
vim-cmd vmsvc/power.getstate 16
- Power on
vim-cmd vmsvc/power.on 16
- Shutdown
vim-cmd vmsvc/power.shutdown 16
- Hard stop
vim-cmd vmsvc/power.off 16
- Soft reboot
vim-cmd vmsvc/power.reboot 16
- Reset
vim-cmd vmsvc/power.reset 16
Other operations
- List available guests
vim-cmd vmsvc/getallvms
- Get extra online information from VM
vim-cmd vmsvc/get.summary 16
- Install VMWare Tools (don't use it here, we do it in another way)
vim-cmd vmsvc/tools.install 16
- Reload changes done in the
.vmx
filevim-cmd vmsvc/reload 16
- Unregister a machine (run before removing the files!)
vim-cmd vmsvc/unregister 16
- Check running machines CPU usage
esxtop
- See the system console (what you see from the ILOM)
dcui
Working with the GUI
Some times you would like to work with the User Interface (it's called vSphere Client)
- Identify the ID of the windows VM guest on xen02 (with xm list. In this example, id=11)
- Open a tunnel from your workstation to route VNC traffic
ssh root@xen02 -L 12345:localhost:5911
(change the 11 for the ID from before)
- Run in your desktop
vncviewer localhost:12345
- Start the VMWare vSphere Client, and open the session to the 10.10 management interface of the host.
Working with the console
You need to keep in mind that the root filesystem is overwritten every time you reboot. To make changes permanent you need to edit an already existing file in /etc (for example) and run
/sbin/auto-backup.sh
. If you want to add actions to the boot procedure, edit the
/etc/rc.local
and do it permanent.
Also, you can use
/sbin/services.sh restart
to restart all enabled services, specified in
/etc/chkconfig.db
. This does not affect the virtual machine guests.
External links
There are some pages on how to manage ESXi from the command line:
And maybe, at some point, you want to use this, but probably not:
http://www.vmware.com/support/developer/vima/
--
PabloFernandez - 2010-12-02