Thursday, June 23, 2011

HowTo - Selectively enable service notifications in Check_mk / OMD

Check_mk installed via Open Monitoring Distribution are an extremely powerful combination for monitoring devices on a network.

Notifications from Nagios for the services discovered by Check_mk can overwhelm your inbox / mobile phone if notification for all services is enabled (default).

The following configures Nagios via Check_mk in an 'opt-in' method for service notifications using extra_service_conf in main.mk

The following section in main.mk will:
* Enable notifications for "IPMI Sensor Summary" to get temperature alerts
* Enable notifications for "fs_*" to get alerts for all file system disk usage
* Enable notifications for "Memory Used" for a specific server, server1
* Disabling all other service notifications


extra_service_conf["notifications_enabled"] = [
  ( "1", ALL_HOSTS, ["IPMI Sensor Summary","ambient_temp"]),
  ( "1", ALL_HOSTS, ["IPMI Sensor Summary","fs_*"]),
  ( "1", ["server1"], ["Memory Used"]),
  ( "0", ALL_HOSTS, ALL_SERVICES),
]

Wednesday, June 1, 2011

Upgrading VMware ESXi 4.0 to ESXi 4.1 Update 1

The following are notes I took while upgrading several ESXi 4.0 servers (Dell PowerEdge M600) to Update 4.1.0 Update 1.

I decided to post these notes since I encountered an error that wasn't specifically identified in a VMware KB, although there is one for the issue as you'll see, the error is slightly different.

Here's the VMware KB article that discusses the upgrade options to 4.1:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1022140

The following is the specific video that I used as reference (uses esxupdate from the SSH consol):
http://www.youtube.com/watch?v=F0wSHPSvmpk&feature=player_embedded#at=57

  1. I updated the ESXi 4.0 servers to the latest patches prior to performing the upgrade to 4.1u1. I'm not sure if this is required or not.
  2. Copy the upgrade-from-esxi4.0-to-4.1-update01-348481.zip to the local datastore (I put it in a new directory /vmfs/volumes/datastore1/esxi-upgrade). The upgrade package may be different if your version isn't currently ESXi version 4.0. Also, if you are upgrading from ESX 4.0 make sure to watch the video and follow the initial preupgrade step!
  3. SSH into the ESXi server
  4. Put the ESXi server into maintenance mode via either the vSphere Client GUI or from the command line
    
    # vim-cmd /hostsvc/maintenance_mode_enter
    # vim-cmd /hostsvc/runtimeinfo | grep inMaintenanceMode
       inMaintenanceMode = true, 
    
    
  5. Try to run the update, mine ran for several minutes and eventually errored:
    
    # esxupdate --bundle=/vmfs/volumes/datastore1/esxi-upgrade/upgrade-from-esxi4.0-to-4.1-update01-348481.zip update
    
    The following problems were encountered trying to resolve dependencies:
       Requested VIB deb_vmware-esx-firmware_4.1.0-1.4.348481 conflicts with the
       host
    
    I found a KB article that covered a similar error and followed it's advice to remove an unneeded Cisco Nexus package:
    http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1026752
    • Check if the cisco package is installed
      
      # esxupdate query --vib-view | grep cross_cisco | grep installed
      
      cross_cisco-vem-v100-esx_4.0.4.1.1.28-0.5.2                        installed     2009-07-17T17:53:15.448003+00:00 
      
      
    • Remove the package
      
      # esxupdate -b cross_cisco-vem-v100-esx_4.0.4.1.1.28-0.5.2 remove
      
  6. Run the update again, this time it succeeds (ran for 10 minutes or so)
    
    # esxupdate --bundle=/vmfs/volumes/datastore1/esxi-upgrade/upgrade-from-esxi4.0-to-4.1-update01-348481.zip update
    
    Unpacking vmware-esx-tools-light-4.1.0-1.4.348481.i386.vib          ################################################################################################ [100%]
    Unpacking vmware-esx-firmware-4.1.0-1.4.348481.i386.vib             ################################################################################################ [100%]
    Unpacking cross_oem-vmware-esx-drivers-net-vxge_400.2.0.28.21239-.. ################################################################################################ [100%]
    Unpacking vmware-esx-esxupdate-esxi-4.1.0-0.0.260247.i386.vib       ################################################################################################ [100%]
    Removing packages :vmware-esx-tools-light vmware-esx-viclient       ################################################################################################ [100%]
    Installing packages :deb_vmware-esx-esxupdate-esxi_4.1.0-0.0.260247 ################################################################################################ [100%]
    Installing packages :deb_vmware-esx-firmware_4.1.0-1.4.348481       ################################################################################################ [100%]
    Installing packages :cross_oem-vmware-esx-drivers-net-vxge_400.2... ################################################################################################ [100%]
    Installing packages :deb_vmware-esx-tools-light_4.1.0-1.4.348481    ################################################################################################ [100%]
    
    Running [/usr/sbin/vmkmod-install.sh]...
    ok.
    The update completed successfully, but the system needs to be rebooted for the
    changes to be effective.
    
  7. Exit maintenance mode and reboot
    
    # vim-cmd /hostsvc/maintenance_mode_exit
    # vim-cmd /hostsvc/runtimeinfo | grep inMaintenanceMode
       inMaintenanceMode = false, 
    
    # reboot