Duncan Epping recently described an issue with virtual machines (VMs) which have moved via Storage vMotion (SvMotion) and are connected to a vNetwork Distributed Switch (VDS), if you are using a configuration where VMs are connected to a VDS and could potentially move via SvMotion then please make sure you read his article here.

William recently showed how we could check for this issue using Perl, on this post you will see a similar script which uses PowerCLI to look for the issue and also resolve the issue fixing the VMs which could potentially have an issue.

In this script I use the VMware VDS Fling which adds VDS cmdlets to PowerCLI, more information and lots of examples on this fling can be found here.  Please make sure you have it installed before using this script and are using a 32 bit PowerShell or PowerCLI console.

Using the script

To check the VMs we can easily pipe a list of VMs into our function which can be seen below.  This can be all VMs in a Cluster, all VMs on a particular host or any other list of VMs you can think of, for my examples below I have shown all VMs attached to a vCenter

As you can see from the above screenshot, all VMs are fine apart from VM12 which currently has the problem described in Duncan’s article,  now to fix the issue.

We can use the same script with a –Fix parameter which allows us to fix the issue, when fixing the issue the script will move each of the VMs network connections to a new port on the same portgroup and then move it back again to its original port.   If no free ports are available the script will expand your portgroup temporarily and then decrease the ports when finished.

As you can see from the above screenshot, the issue has now been resolved for this VM by using the function with the –Fix parameter and further running of the script in test mode will show all are now fine.

UPDATE: The script has now been updated to support remediation for VMs connected to both a VMware VDS as well as Cisco N1KV. The solution, thanks to one of our internal engineers was to “move” the VM’s dvport from one to another, all while staying within the existing dvPortgroup which will also force the creation of the .dvsdb port file. Once the dvport move has successfully completed, we will move it back to it’s original dvport that it initially resided on. We no longer have to rely on creating a temporally dvPortgroup and best of all, we can now remediate both VDS and N1KV.

Disclaimer: This script is not officially supported by VMware, please test this in a development environment before using on production systems.

The Script

If (-Not (Get-PSSnapin VMware.VimAutomation.VdsComponent -WarningAction SilentlyContinue) ) {
	Add-PSSnapin VMware.VimAutomation.VdsComponent
}

Function Test-VDSVMIssue {
	Param (
		[parameter(Mandatory=$true,ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]
        [PSObject[]]$VM,
		[switch]$Fix
	)
	Process {
		Foreach ($VMachine in $VM){
			Foreach ($NA in ($VMachine | Get-NetworkAdapter)) {
				$VMName = $VMachine.Name
				If (($NA.ExtensionData.Backing.GetType()).Name -eq "VirtualEthernetCardDistributedVirtualPortBackingInfo") {
					$PortKey = $NA.ExtensionData.Backing.Port.PortKey
					$vSwitchID = $NA.ExtensionData.Backing.Port.SwitchUUID
					$Datastore = (($VMachine.ExtensionData.Config.Files.VmPathName).split("]")[0]).Replace("[","")
					$filename = "$($datastore):\.dvsData\$vSwitchID\$PortKey"
					if (-not (Get-PSDrive $datastore -ErrorAction SilentlyContinue)) {
						$NewDrive = New-PSDrive -Name $Datastore -Location (Get-Datastore $Datastore) -PSProvider VimDatastore -Root '\'
					}
					$filecheck = Get-ChildItem -Path $filename -ErrorAction SilentlyContinue
					if ($filecheck) {
						Write-Host -ForegroundColor Green "$VMName $($NA.Name) is OK"
					} Else {
						Write-Host -ForegroundColor Red "Problem found with $VMName $($NA.Name)"
						If ($Fix) {
							Write-Host -ForegroundColor Yellow "Fixing issue..."
							$VDSPG = Get-VdsDistributedPortgroup $NA.NetworkName
							$DVPort = $null
							Write-Host -ForegroundColor Yellow "..Finding free port on $($NA.NetworkName)"
							$DVPort = Get-VdsDVPort -DVPortgroup $VDSPG -Active:$false | Select -last 1
							$Move = $True
							if (-not $DVPort) {
								Write-Host -ForegroundColor Yellow "..No free ports found on $($VDSPG.Name), adding an additional port"
								If (($VDSPG.PortBinding -eq "Ephemeral") -or ($VDSPG.PortBinding -eq "Dynamic")) {
									Write "Unable to add a port to $($NA.NetworkName) since dvportgroup is configured as $($VDSPG.PortBinding)"
									Write-Host -ForegroundColor Red "Problem still exists with $VMName please resolve manually"
									$Move = $false
								} Else {
									$CurrentPorts = $VDSPG.NumPorts 
									$NewTotalPorts = $VDSPG.NumPorts + 1
									Set-VdsDistributedPortgroup -NumPorts $NewTotalPorts -DVPortgroup $VDSPG | Out-Null
									$PGAdded = $true
									$DVPort = Get-VdsDVPort -DVPortgroup $VDSPG -Active:$false | Select -last 1
								}
							}
							If ($Move){
								Write-Host -ForegroundColor Yellow "..Moving $($NA.Name) to another free port on $($VDSPG.Name)"
								$NA | Set-NetworkAdapter -PortKey $DVPort.Key -DistributedSwitch $VDSPG.VirtualSwitch -Confirm:$false | Out-Null
								Write-Host -ForegroundColor Yellow "..Moving $($NA.Name) back to port $PortKey"
								$NA | Set-NetworkAdapter -PortKey $PortKey -DistributedSwitch $VDSPG.VirtualSwitch -Confirm:$false | Out-Null
								Write-Host -ForegroundColor Yellow "..Checking changes were completed"
								$filecheck = Get-ChildItem -Path $filename -ErrorAction SilentlyContinue
								if ($filecheck) {
									Write-Host -ForegroundColor Green "$VMName $($NA.Name) is now fixed and OK"
								} Else {
									Write-Host -ForegroundColor Red "Problem still exists with $VMName please resolve manually"
								}
								If ($PGAdded) {
									Write-Host -ForegroundColor Yellow "..Removing the added port on $($VDSPG.Name)"
									Set-VdsDistributedPortgroup -NumPorts $CurrentPorts -DVPortgroup $VDSPG | Out-Null
									$PGAdded = $false
								}
							}
						}
					}	
				} Else {
					Write-Host -ForegroundColor Green "$VMName is not connected to a dvSwitch so this issue is not relevant."
				}
			}
		}
		Get-PSDrive | Where { ($_.Provider -like "*VimDatastore") -and ( $_.Name -notlike "*vmstore*")} | Foreach {
			Remove-PSDrive $_ | Out-Null
		}
	}
}

Thanks to all the people who sent me a plugin for vCheck 6.0, these have now been published as part of vCheck 6.10 and zipped into a single download file which can be downloaded from below.  I also love the enhancements which have been sent to me for the core vCheck script, these include a script which times and reports on how long the plugins take – great for troubleshooting !

[wpdm_file id=17]

As always you can find more information about vCheck and how to use this by visiting this page.

Change log

There are now over 70 plugins, some fantastic stuff, below is a list of items which have changed or been fixed since 6.0:

# v 6.10 – Fixed multiple spelling mistakes and small plugin issues
# v 6.9 – Fixed VMKernel logs but had to remove date/Time parser due to inconsistent VMKernel Log entries
# v 6.8 – Added Creator of snapshots back in due to popular demand
# v 6.7 – Added Multiple plugins from contributors – Thanks!
# V 6.6 – Tech Support Mode Plugin fixed to work with 5.0 hosts
# V 6.5 – HW Version plugin fixed due to string output
# V 6.4 – Added a 00 plugin and VeryLastPlugin for vCenter connection info to separate the report entirely from VMware if needed.
# V 6.3 – Changed the format of each Plugin so you can include a count for each header and altered plugin layout for each plugin.
# V 6.2 – Added Time to Run section based on TimeToBuild by Frederic Martin
# V 6.1 – Bug fixes, filter for ps1 files only in the plugins folder so other files can be kept in the plugins folder.

Reminder

If a plugin is not needed in your environment remove it from the plugins folder as it will speed up the execution of your script.

One of the most popular scripts on my site with 26,966 downloads of v5 has been the vCheck script, for those of you who haven’t seen this yet here is a quick summary of this script:

Summary

vCheck is a vCenter checking script, the script is designed to run as a scheduled task before you get into the office to present you with key information via an email directly to your inbox in a nice easily readable format.

This script picks on the key known issues and potential issues of the virtual infrastructure and reports it all in one place so all you do in the morning is check your email.

One of they key things about this report is if there is no issue in a particular place you will not receive that section in the email, for example if there are no datastores with less than 5% free space (configurable) then the disk space section will not show in the email, this ensures that you have only the information you need in front of you when you get into the office.

This script is not to be confused with an Audit script, I don’t want to remind you that you have 5 hosts and what there names are and how many CPU’s they have each and every day as you don’t want to read that kind of information unless you need it, this script will only tell you about problem areas with your infrastructure.

Continue reading →