Duncan Epping recently described an issue with virtual machines (VMs) which have moved via Storage vMotion (SvMotion) and are connected to a vNetwork Distributed Switch (VDS), if you are using a configuration where VMs are connected to a VDS and could potentially move via SvMotion then please make sure you read his article here.
William recently showed how we could check for this issue using Perl, on this post you will see a similar script which uses PowerCLI to look for the issue and also resolve the issue fixing the VMs which could potentially have an issue.
In this script I use the VMware VDS Fling which adds VDS cmdlets to PowerCLI, more information and lots of examples on this fling can be found here. Please make sure you have it installed before using this script and are using a 32 bit PowerShell or PowerCLI console.
Using the script
To check the VMs we can easily pipe a list of VMs into our function which can be seen below. This can be all VMs in a Cluster, all VMs on a particular host or any other list of VMs you can think of, for my examples below I have shown all VMs attached to a vCenter
As you can see from the above screenshot, all VMs are fine apart from VM12 which currently has the problem described in Duncan’s article, now to fix the issue.
We can use the same script with a –Fix parameter which allows us to fix the issue, when fixing the issue the script will move each of the VMs network connections to a new port on the same portgroup and then move it back again to its original port. If no free ports are available the script will expand your portgroup temporarily and then decrease the ports when finished.
As you can see from the above screenshot, the issue has now been resolved for this VM by using the function with the –Fix parameter and further running of the script in test mode will show all are now fine.
UPDATE: The script has now been updated to support remediation for VMs connected to both a VMware VDS as well as Cisco N1KV. The solution, thanks to one of our internal engineers was to “move” the VM’s dvport from one to another, all while staying within the existing dvPortgroup which will also force the creation of the .dvsdb port file. Once the dvport move has successfully completed, we will move it back to it’s original dvport that it initially resided on. We no longer have to rely on creating a temporally dvPortgroup and best of all, we can now remediate both VDS and N1KV.
Disclaimer: This script is not officially supported by VMware, please test this in a development environment before using on production systems.
The Script
If (-Not (Get-PSSnapin VMware.VimAutomation.VdsComponent -WarningAction SilentlyContinue) ) { Add-PSSnapin VMware.VimAutomation.VdsComponent } Function Test-VDSVMIssue { Param ( [parameter(Mandatory=$true,ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)] [PSObject[]]$VM, [switch]$Fix ) Process { Foreach ($VMachine in $VM){ Foreach ($NA in ($VMachine | Get-NetworkAdapter)) { $VMName = $VMachine.Name If (($NA.ExtensionData.Backing.GetType()).Name -eq "VirtualEthernetCardDistributedVirtualPortBackingInfo") { $PortKey = $NA.ExtensionData.Backing.Port.PortKey $vSwitchID = $NA.ExtensionData.Backing.Port.SwitchUUID $Datastore = (($VMachine.ExtensionData.Config.Files.VmPathName).split("]")[0]).Replace("[","") $filename = "$($datastore):\.dvsData\$vSwitchID\$PortKey" if (-not (Get-PSDrive $datastore -ErrorAction SilentlyContinue)) { $NewDrive = New-PSDrive -Name $Datastore -Location (Get-Datastore $Datastore) -PSProvider VimDatastore -Root '\' } $filecheck = Get-ChildItem -Path $filename -ErrorAction SilentlyContinue if ($filecheck) { Write-Host -ForegroundColor Green "$VMName $($NA.Name) is OK" } Else { Write-Host -ForegroundColor Red "Problem found with $VMName $($NA.Name)" If ($Fix) { Write-Host -ForegroundColor Yellow "Fixing issue..." $VDSPG = Get-VdsDistributedPortgroup $NA.NetworkName $DVPort = $null Write-Host -ForegroundColor Yellow "..Finding free port on $($NA.NetworkName)" $DVPort = Get-VdsDVPort -DVPortgroup $VDSPG -Active:$false | Select -last 1 $Move = $True if (-not $DVPort) { Write-Host -ForegroundColor Yellow "..No free ports found on $($VDSPG.Name), adding an additional port" If (($VDSPG.PortBinding -eq "Ephemeral") -or ($VDSPG.PortBinding -eq "Dynamic")) { Write "Unable to add a port to $($NA.NetworkName) since dvportgroup is configured as $($VDSPG.PortBinding)" Write-Host -ForegroundColor Red "Problem still exists with $VMName please resolve manually" $Move = $false } Else { $CurrentPorts = $VDSPG.NumPorts $NewTotalPorts = $VDSPG.NumPorts + 1 Set-VdsDistributedPortgroup -NumPorts $NewTotalPorts -DVPortgroup $VDSPG | Out-Null $PGAdded = $true $DVPort = Get-VdsDVPort -DVPortgroup $VDSPG -Active:$false | Select -last 1 } } If ($Move){ Write-Host -ForegroundColor Yellow "..Moving $($NA.Name) to another free port on $($VDSPG.Name)" $NA | Set-NetworkAdapter -PortKey $DVPort.Key -DistributedSwitch $VDSPG.VirtualSwitch -Confirm:$false | Out-Null Write-Host -ForegroundColor Yellow "..Moving $($NA.Name) back to port $PortKey" $NA | Set-NetworkAdapter -PortKey $PortKey -DistributedSwitch $VDSPG.VirtualSwitch -Confirm:$false | Out-Null Write-Host -ForegroundColor Yellow "..Checking changes were completed" $filecheck = Get-ChildItem -Path $filename -ErrorAction SilentlyContinue if ($filecheck) { Write-Host -ForegroundColor Green "$VMName $($NA.Name) is now fixed and OK" } Else { Write-Host -ForegroundColor Red "Problem still exists with $VMName please resolve manually" } If ($PGAdded) { Write-Host -ForegroundColor Yellow "..Removing the added port on $($VDSPG.Name)" Set-VdsDistributedPortgroup -NumPorts $CurrentPorts -DVPortgroup $VDSPG | Out-Null $PGAdded = $false } } } } } Else { Write-Host -ForegroundColor Green "$VMName is not connected to a dvSwitch so this issue is not relevant." } } } Get-PSDrive | Where { ($_.Provider -like "*VimDatastore") -and ( $_.Name -notlike "*vmstore*")} | Foreach { Remove-PSDrive $_ | Out-Null } } }
Pingback: Fix Vds Basic Provider Error 1 Vmware Windows XP, Vista, 7, 8 [Solved]
I’m having some trouble with this script. I run the command “Get-VM | Test-VDSVMIssue” and after a few minutes it’s done, but no output… so I’m thinking it didn’t work.
When I run just the command “Get-VM”, it lists all the VMs.
When I run just the command “Test-VDSVMIssue” I get this:
cmdlet Test-VDSVMIssue at command pipeline position 1
Supply values for the following parameters:
VM[0]:
If I try to enter a VM manually at that line I get this error:
Get-NetworkAdapter : Cannot process argument transformation on parameter ‘VM’.
Strings as pipeline input are not supported.
Any ideas?
Pingback: Storage vMotion / dvSwitch patch released (but manual fix required) | Virten.net
Pingback: VMware vCenter Server 5.0 Update 1a released « Aravind Sivaraman
Pingback: July 2012 ESXi and vCenter updates « alpacapowered
Pingback: HA / Distributed vSwitch problems after Storage vMotion scripts available for KB2013639 « blocksandbytes
Pingback: Storage vMotion and dvSwitch / HA problem explained | Virten.net
Pingback: Updated Script to fix VDS/SvMotion issue | | Virtu-Al.NetVirtu-Al.Net
Pingback: 143: Kritik bir HA-dvS problemi « Orcunus.cOm
Pingback: Storage vMotion and dvSwitch / HA problem explained | Virten.net
Hi Alan,
The script correctly identifies affected VMs in my environment but it failes when I add -Fix command.
———————–
Fixing issue…
Get-VdsDistributedPortgroup : 03/05/2012 14:46:37 Get-VdsDistributedPortgrou
p Unable to cast object of type ‘VMware.Vim.DVPortSetting’ to type ‘VMwa
re.Vim.VMwareDVSPortSetting’.
At line:26 char:65
+ $VDSPG = Get-VdsDistributedPortgroup <<<< $NA.Ne
tworkName
+ CategoryInfo : NotSpecified: (:) [Get-VdsDistributedPortgroup],
VimException
+ FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomatio
n.VdsComponent.Commands.Cmdlets.GetDVPortgroup
———————–
In our environment all Port Groups have "_" – underscore in the Port Group name. Can it be the problem?
Thank you.
More weird issues with powershell scripts started occurring (not just with this one!) so i ran teh script on another machine and it worked correctly! Sorry for bothering you before thinking of trying that!
Pingback: Scripts release for Storage vMotion / HA problem « vResource.net
I cannot get the script to work. There is an error with the $VDSPG.VirtualSwitch being null.
..Moving Network adapter 1 to another free port on
Set-NetworkAdapter : Cannot validate argument on parameter ‘DistributedSwitch’. The argument is null. Supply a non-null
argument and try the command again.
At C:\Users\friesem1\scripts\Test-VDSVMIssue.ps1:58 char:73
+ $NA | Set-NetworkAdapter -PortKey $DVPort.Key -DistributedSwitch <<<< $VDSPG.VdsId –
Confirm:$false | Out-Null
+ CategoryInfo : InvalidData: (:) [Set-NetworkAdapter], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.VirtualDev
ice.SetNetworkAdapter
I am using the following PowerCLI versions:
VMware VDS vSphere PowerCLI Component 4.1.1 build 000001
VMWare vSphere PowerCLI 4.1 U1 build 332441
Any ideas regarding the problem? I noticed that I cannot get anything but a null value returned when calling $VDSPG.VirtualSwitch, but can get a value when doing $VDSPG | Select -property VirtualSwitch
Thanks!
So one minor issue I had with the script was in our VDI environment. Many of the ‘non-active’ ports were simply shutdown VMs so when doing the Get-vdsdvport function, it would return an inactive port, but a shutdown VM is still connected so it fails. The simple fix is to also add the -Connected:$false to this line in the script:
$DVPort = Get-VdsDVPort -DVPortgroup $VDSPG -Active:$false -Connected:$false | Select -last 1
Updated the script so that it can report the vm’s affected to the pipe
If (-Not (Get-PSSnapin VMware.VimAutomation.VdsComponent -WarningAction SilentlyContinue) ) {
Add-PSSnapin VMware.VimAutomation.VdsComponent
}
Function Test-VDSVMIssue {
Param (
[parameter(Mandatory=$true,ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]
[PSObject[]]$VM,
[switch]$Fix,
[Switch]$ReportVM
)
Process {
Foreach ($VMachine in $VM){
$Vmreport = $Null
Foreach ($NA in ($VMachine | Get-NetworkAdapter)) {
$VMName = $VMachine.Name
If (($NA.ExtensionData.Backing.GetType()).Name -eq “VirtualEthernetCardDistributedVirtualPortBackingInfo”) {
$PortKey = $NA.ExtensionData.Backing.Port.PortKey
$vSwitchID = $NA.ExtensionData.Backing.Port.SwitchUUID
$Datastore = (($VMachine.ExtensionData.Config.Files.VmPathName).split(“]”)[0]).Replace(“[“,””)
$filename = “$($datastore):\.dvsData\$vSwitchID\$PortKey”
if (-not (Get-PSDrive $datastore -ErrorAction SilentlyContinue)) {
$NewDrive = New-PSDrive -Name $Datastore -Location (Get-Datastore $Datastore) -PSProvider VimDatastore -Root ‘\’ -erroraction silentlycontinue
}
$filecheck = Get-ChildItem -Path $filename -ErrorAction SilentlyContinue
if ($filecheck) {
Write-Verbose “$VMName $($NA.Name) is OK”
} Else {
Write-Verbose “Problem found with $VMName $($NA.Name)”
$Vmreport = $VM
If ($Fix) {
Write-Verbose “Fixing issue…”
$VDSPG = Get-VdsDistributedPortgroup $NA.NetworkName
$DVPort = $null
Write-Verbose “..Finding free port on $($NA.NetworkName)”
$DVPort = Get-VdsDVPort -DVPortgroup $VDSPG -Active:$false | Select -last 1
$Move = $True
if (-not $DVPort) {
Write-Verbose “..No free ports found on $($VDSPG.Name), adding an additional port”
If (($VDSPG.PortBinding -eq “Ephemeral”) -or ($VDSPG.PortBinding -eq “Dynamic”)) {
Write “Unable to add a port to $($NA.NetworkName) since dvportgroup is configured as $($VDSPG.PortBinding)”
Write-Verbose “Problem still exists with $VMName please resolve manually”
$Move = $false
} Else {
$CurrentPorts = $VDSPG.NumPorts
$NewTotalPorts = $VDSPG.NumPorts + 1
Set-VdsDistributedPortgroup -NumPorts $NewTotalPorts -DVPortgroup $VDSPG | Out-Null
$PGAdded = $true
$DVPort = Get-VdsDVPort -DVPortgroup $VDSPG -Active:$false | Select -last 1
}
}
If ($Move){
Write-Verbose “..Moving $($NA.Name) to another free port on $($VDSPG.Name)”
$NA | Set-NetworkAdapter -PortKey $DVPort.Key -DistributedSwitch $VDSPG.VirtualSwitch -Confirm:$false | Out-Null
Write-Verbose “..Moving $($NA.Name) back to port $PortKey”
$NA | Set-NetworkAdapter -PortKey $PortKey -DistributedSwitch $VDSPG.VirtualSwitch -Confirm:$false | Out-Null
Write-Verbose “..Checking changes were completed”
$filecheck = Get-ChildItem -Path $filename -ErrorAction SilentlyContinue
if ($filecheck) {
Write-Verbose “$VMName $($NA.Name) is now fixed and OK”
} Else {
Write-Verbose “Problem still exists with $VMName please resolve manually”
}
If ($PGAdded) {
Write-Verbose “..Removing the added port on $($VDSPG.Name)”
Set-VdsDistributedPortgroup -NumPorts $CurrentPorts -DVPortgroup $VDSPG | Out-Null
$PGAdded = $false
}
}
}
}
} Else {
Write-Verbose “$VMName is not connected to a dvSwitch so this issue is not relevant.”
}
}
}
Get-PSDrive | Where { ($_.Provider -like “*VimDatastore”) -and ( $_.Name -notlike “*vmstore*”)} | Foreach {
Remove-PSDrive $_ | Out-Null
}
If (($ReportVM) -and ($VMreport -ne $null)) {
$VMReport
}
}
}
Erik, great to hear it worked out well for you.
Hiya Alan,
Thanks so much for your Test-VDSVMIssue function. It works.
Is there anyway to output to a log file the description of the VM that are affected with the missing dvPort setting ?
I’m running the script against a Cluster with 600 VMs and I have about 20% of the VMs that are affected due to SDRS being in Automatic mode.
Thanks,
Erik
My problem seems to be related to the fact that we have / (forward slash) in the naming convention of distributed portgroups. I’m working on a fix, but the original script is working fine for portgroups without / in their names 🙂
Hi!
I’ve been testing this script but am unable to get it working.
The first part, checking which virtual servers are affected is working fine but the second part where I ass -Fix doesn’t work.
I’m getting the following error:
New-VdsDistributedPortgroup : Cannot bind parameter 'Vds'. Cannot convert the "
" value of type "System.Management.Automation.PSCustomObject" to type "VMware.V
imAutomation.VdsComponent.Types.V1.VdsVSphereDistributedSwitch".
At C:\SCRIPTS\vmware_skript\vds_svmotion_check.ps1:34 char:56
+ New-VdsDistributedPortgroup -Name $NewName -V
ds <<<< (Get-vds | Where {$_.Key -eq $NA.ExtensionData.Backing.Port.SwitchUUID
}) -ReferenceDVPortgroup $VDSPG | Out-Null
+ CategoryInfo : InvalidArgument: (:) [New-VdsDistributedPortgrou
p], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgumentNoMessage,VMware.VimAutomat
ion.VdsComponent.Commands.Cmdlets.NewDVPortgroup
This results is the new dv-portgroup not being created and as such, the rest of the script also throws out errors.
Any tips?
Thanks Alpaca, I updated the post to link to the VDS fling – well spotted. Thanks for the further info too !
Pingback: Problem with vSphere 5 HA and Storage vMotion when VM on vDS « Long White Virtual Clouds
Jon:
You first need to install the VDS Powercli Fling:
http://labs.vmware.com/flings/vdspowercli
Seems like Alan actually forgot to link it with his “can be found here”.
– Install the fling
– you must also use the 32bit Powercli, since the VDS fling doesn’t work with the 64bit environment
– Connect to your vCenter via “connect-viserver”
– To get the function Alan wrote into your powershell session, execute “Import-Module D:\querysvmotionissue.ps1” or you might as well paste the whole code just into the shell, quick and dirty
– Execute the Test-VDSVMIssue function against your VMs like “Get-VM | Test-VDSVMIssue”
– Enjoy
Hi Apologie for the basic question I am about to ask.
How do i install/runt he script. I see you are calling a function from the screenshot, how am i able to add the script so I can call the function.
Again sorry for the basic question, pretty new to PS and CLI
I have tried the script and it seams it does not fix the issues. it keeps returning i have to fix it manually.
Pingback: Scripts release for Storage vMotion / HA problem - Yellow Bricks
Pingback: HA fails to initiate restart when a VM is SvMotioned and on a VDS! - Yellow Bricks
Pingback: - Cliff Davies