Updated Script to remediate VDS/SvMotion issue

Following my recent post where I showed how we could search for and fix any VMs which had an issue after being Storage vMotioned and attached to a VDS I received a couple of emails from people with a few issues.

Firstly the script was written using the VDS PowerShell fling (Check it out, its cool), some people were not able to use this fling in production systems as it is not officially supported by VMware.

Secondly the fling only supported VDS and not the Nexus 1kv, some customers had this issue with the N1KV so needed a resolution.

The good news…

I have re-written the script to no longer use the fling but instead use the raw API’s, this means all you now need is PowerCLI 5.0.1 and you can run the below script in the same way as before to check and remediate for issues.

Thanks goes to Luc who came up with an easy way to check for free ports on the VDS here.

Remember this does not fix the issue completely, only a patch from VMware will do that but it does however tell you if you have the issue and plug the gap temporarily.

Just as a reminder, here is how the script is used:

Using the script

To check the VMs we can easily pipe a list of VMs into our function which can be seen below.  This can be all VMs in a Cluster, all VMs on a particular host or any other list of VMs you can think of, for my examples below I have shown all VMs attached to a vCenter

image

As you can see from the above screenshot, all VMs are fine apart from VM12 which currently has the problem described in Duncan’s article,  now to fix the issue.

We can use the same script with a –Fix parameter which allows us to fix the issue, when fixing the issue the script will move each of the VMs network connections to a new port on the same portgroup and then move it back again to its original port.   If no free ports are available the script will expand your portgroup temporarily and then decrease the ports when finished.

image

As you can see from the above screenshot, the issue has now been resolved for this VM by using the function with the –Fix parameter and further running of the script in test mode will show all are now fine.

The Script

Function Get-FreeVDSPort ($VDSPG) {
	$nicTypes = "VirtualE1000","VirtualE1000e","VirtualPCNet32","VirtualVmxnet","VirtualVmxnet2","VirtualVmxnet3"
	$ports = @{}

	# Get all the portkeys on the portgroup
	$VDSPG.ExtensionData.PortKeys | Foreach {
		$ports.Add($_,$VDSPG.Name)
	}

	# Remove the portkeys in use  Get-View
	$VDSPG.ExtensionData.Vm | Foreach {
	    $VMView = Get-View $_
		$nic = $VMView.Config.Hardware.Device | where {$nicTypes -contains $_.GetType().Name -and $_.Backing.GetType().Name -match "Distributed"}
	    $nic | where {$_.Backing.Port.PortKey} | Foreach {$ports.Remove($_.Backing.Port.PortKey)}
	}

	# Assign the first free portkey
	if ($ports.Count -eq 0) {
		$null
	} Else {
		$ports.Keys | Select -First 1
	}
}

Function Set-VDSPGNumPorts ($VDSPG, $NumPorts) {
	$spec = New-Object VMware.Vim.DVPortgroupConfigSpec
    $spec.numPorts = $NumPorts
	$spec.ConfigVersion = $VDSPG.ExtensionData.Config.Configversion
    $VDSPG.ExtensionData.ReconfigureDVPortgroup($spec)
}

Function Test-VDSVMIssue {
	Param (
		[parameter(Mandatory=$true,ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]
        [PSObject[]]$VM,
		[switch]$Fix
	)
	Process {
		Foreach ($VMachine in $VM){
			Foreach ($NA in ($VMachine | Get-NetworkAdapter)) {
				$VMName = $VMachine.Name
				If (($NA.ExtensionData.Backing.GetType()).Name -eq "VirtualEthernetCardDistributedVirtualPortBackingInfo") {
					$PortKey = $NA.ExtensionData.Backing.Port.PortKey
					$vSwitchID = $NA.ExtensionData.Backing.Port.SwitchUUID
					$Datastore = (($VMachine.ExtensionData.Config.Files.VmPathName).split("]")[0]).Replace("[","")
					$filename = "$($datastore):\.dvsData\$vSwitchID\$PortKey"
					if (-not (Get-PSDrive $datastore -ErrorAction SilentlyContinue)) {
						$NewDrive = New-PSDrive -Name $Datastore -Location (Get-Datastore $Datastore) -PSProvider VimDatastore -Root '\'
					}
					$filecheck = Get-ChildItem -Path $filename -ErrorAction SilentlyContinue
					if ($filecheck) {
						Write-Host -ForegroundColor Green "$VMName $($NA.Name) is OK"
					} Else {
						Write-Host -ForegroundColor Red "Problem found with $VMName $($NA.Name)"
						If ($Fix) {
							Write-Host -ForegroundColor Yellow "Fixing issue..."
							$VDSPG = Get-VirtualPortGroup -Distributed -Name $NA.NetworkName
							$DVPort = $null
							Write-Host -ForegroundColor Yellow "..Finding free port on $($NA.NetworkName)"
							$DVPort = Get-FreeVDSPort $VDSPG
							$Move = $True
							if (-not $DVPort) {
								Write-Host -ForegroundColor Yellow "..No free ports found on $($VDSPG.Name), adding an additional port"
								If (($VDSPG.ExtensionData.Config.Type -ne "lateBinding") -and ($VDSPG.ExtensionData.Config.Type -ne "earlyBinding")) {
									Write "Unable to add a port to $($NA.NetworkName) since dvportgroup is configured as $($VDSPG.PortBinding)"
									Write-Host -ForegroundColor Red "Problem still exists with $VMName please resolve manually"
									$Move = $false
								} Else {
									$CurrentPorts = $VDSPG.NumPorts
									$NewTotalPorts = $VDSPG.NumPorts + 1
									Set-VDSPGNumPorts -VDSPG $VDSPG -NumPorts $NewTotalPorts
									$PGAdded = $true
									$VDSPG = Get-VirtualPortGroup -Distributed -Name $NA.NetworkName
									$DVPort = Get-FreeVDSPort $VDSPG
								}
							}
							If ($Move){
								Write-Host -ForegroundColor Yellow "..Moving $($NA.Name) to another free port on $($VDSPG.Name)"
								$NA | Set-NetworkAdapter -PortKey $DVPort -DistributedSwitch $VDSPG.VirtualSwitch -Confirm:$false | Out-Null
								Write-Host -ForegroundColor Yellow "..Moving $($NA.Name) back to port $PortKey"
								$NA | Set-NetworkAdapter -PortKey $PortKey -DistributedSwitch $VDSPG.VirtualSwitch -Confirm:$false | Out-Null
								Write-Host -ForegroundColor Yellow "..Checking changes were completed"
								$filecheck = Get-ChildItem -Path $filename -ErrorAction SilentlyContinue
								if ($filecheck) {
									Write-Host -ForegroundColor Green "$VMName $($NA.Name) is now fixed and OK"
								} Else {
									Write-Host -ForegroundColor Red "Problem still exists with $VMName please resolve manually"
								}
								If ($PGAdded) {
									Write-Host -ForegroundColor Yellow "..Removing the added port on $($VDSPG.Name)"
									Set-VDSPGNumPorts -VDSPG $VDSPG -NumPorts $CurrentPorts
									$PGAdded = $false
								}
							}
						}
					}
				} Else {
					Write-Host -ForegroundColor Green "$VMName is not connected to a dvSwitch so this issue is not relevant."
				}
			}
		}
		Get-PSDrive | Where { ($_.Provider -like "*VimDatastore") -and ( $_.Name -notlike "*vmstore*")} | Foreach {
			Remove-PSDrive $_ | Out-Null
		}
	}
}

# Example code to check all VMs attached to vCenter for the issue:
# Get-VM | Test-VDSVMIssue

# Example code to fix all VMs attached to vCenter:
# Get-VM | Test-VDSVMIssue -Fix

# Example code to fix all VMs in Cluster01 for the issue:
# Get-Cluster01 | Get-VM | Test-VDSVMIssue

# Example code to fix all VMs in Cluster01:
# Get-Cluster01 | Get-VM | Test-VDSVMIssue -Fix

3 thoughts on “Updated Script to remediate VDS/SvMotion issue”

  1. Alan – Do you have a script to fix ephemeral port groups (since they do not have extra ports to move to)? Because vCloud uses ephemeral port groups…

  2. hi, i have two questions regarding the script :

    – is it possible to lauch– fix only on somes VMs ?

    – When fixing, is there any network failure on the VMs ?

  3. Awesome stuff alan, luckily from what I have heard, there will be a patch by the end of the month to fix this “oversight/issue”

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.