PowerCLI is a great tool to manage your vSphere environment. No discussion about that!

The package contains cmdlets for, let’s say 80% of your vSphere day to day tasks, and for the missing 20% you can always fall back on the vSphere APIs (or ask in the PowerCLI community).

The only problem that pops up from time to time is the performance of some of the PowerCLI cmdlets.The Get-Vm cmdlet is one of these infamous cmdlets which, when executed in a decently sized (number of guests) vSphere environment, tends to be quite slow.

But as with the 80%-20% rule above, you can fall back on the vSphere APIs to speed things up a bit.
Add to that some medium to advanced PowerShell features and the time gains you can reach are impressive.

The problem at hand was that we needed a script to update a number of custom attributes on our guests.

A quick method of doing this would be something along these lines:

Get-VM | Set-CustomField -Name “MyField” -Value “new-value”

Quick and easy, but you’re in for a surprise when you run this in your 1000+ guest environment. You could have considerably more than 1 coffee while this is running.

The not-so-spectacular speed of the Get-VM cmdlet is common knowledge in the PowerCLI community and in fact confirmed by VMware in several PowerCLI community threads. See for example here and here.

A faster alternative is to use the Get-View cmdlet with the -ViewType VirtualMachine parameter. This cmdlet gets the objects that represent guests directly from the vSphere environment without converting them to PowerCLI objects.

The only problem we have now is that the Get-View cmdlet returns a VirtualMachine (vSPhere) object instead of a VirtualMachineImpl (PowerCLI) object, like the Get-Vm cmdlet does.

Luckily there is a PowerCLI cmdlet that allows us to go from a VirtualMachine object to a VirtualMachineImpl object; the Get-VIObjByVIView cmdlet.

Our script becomes

Get-View -ViewType VirtualMachine | Get-VIObjectByVIView |` Set-CustomField -Name “MyField” -Value “new-value”

This will run a bit faster, but not that fast that it would bring down your coffee addiction.

As always, when at a dead-end with PowerCLI look in the APIs.

We find a method called setCustomValue on the ExtensibleManagedObject object.The ExtensibleManagedObject object is extended by the ManagedEntity object and that in it’s turn is extended by the VirtualMachine object.

image

And that’s the kind of object we had from our Get-View -ViewType VirtualMachine cmdlet. Looks promising.

But wait a minute, the setCustomValue method requires 2 parameters, key and value.The value parameter is easy, that is a string that contains the new content of the custom attribute.But how do we get at the key parameter ?

setCustomValue

For most vSphere objects there is a manager available that provides additional methods and properties for the specific types of objects. The same goes for the custom fields.

From the ServiceInstance we can get at the customFieldsManager. This manager contains one property, called field, which is an array of CustomFieldDef objects.

image

Each CustomFieldDef object defines the name and key of a custom attribute. So this looks the place where we find the link between the custom attribute name and the key.

customfielddef

    Let’s store this information in a kind of dictionary (in PS lingo called a hash table).

    We construct two hash tables

  • one where we can look up a key when we know the name of the custom attribute
  • one where we can find the name when we know the key of the custom attribute
    • # Create Custom Field & Name lookup table
    $customFieldMgr = Get-View (Get-View ServiceInstance).Content.CustomFieldsManager
    $customKeyLookup = @{}
    $customNameLookup = @{}
    $customFieldMgr.Field | where {$_.ManagedObjectType -eq “VirtualMachine”} | % {
    $customKeyLookup.Add($_.Key, $_.Name)
    $customNameLookup.Add($_.Name, $_.Key)
    }

Now we need to get our VirtualMachine objects. The fastest way of getting these is with the Get-View -ViewType VirtualMachine cmdlet.

Now we need to consider how we are going to store these objects. For that we better first list what we actually want to do with these objects. In our case we came up with the following requirements:

  • store the VirtualMachine object to be able to use the setCustomValue method
  • store the actual contents of the custom attributes for each guest
  • create a report that lists all guests and their custom attributes
  • import/export the custom attributes from/to a CSV file
  • find all guests that have a specific value in a custom attribute
  • update a specific custom attribute for specific guests

After some trial and error I came up with this structure

image

The outer structure is a hash list with the guest’s name as the key.

The value part of this outer structure is a custom object with two properties:

  • Custom: is another hash table which contains all custom attributes for that guest
  • Object: is the actual VirtualMachine object

    • Based on this structure we can now script all the requirements.

    1) Fill the hash table with all the guests and all their custom attributes for which there is a value.
# Get all guests as VirtualMachine objects
$vms = Get-View -ViewType VirtualMachine
# Create dictionary with all guests and their custom fields
$vmArray = @{}
$vms | % {
$VmCustom = @{}
$_.CustomValue | % {
$VmCustom.Add($_.Key, $_.Value)
}
$row = New-Object psobject
$row | Add-Member -MemberType noteproperty -Name Object -Value $_
$row | Add-Member -MemberType noteproperty -Name Custom -Value $VmCustom
$vmArray.Add($_.Name, $row)
}
    2) Report all guests and their custom fields
# Report all guests and their custom fields
Write-Host “Report all guest with custom fields” -fore red
$vmArray.GetEnumerator() | %{
Write-Host $_.Name
$_.Value.Custom.getenumerator() | %{
Write-Host “`t” $customKeyLookup[$_.Name] $_.Value
}
}
    3) Export all custom attributes to a CSV file
# Export all custom attributes to a CSV file (backup)
$list = @()
$vmArray.GetEnumerator() | %{
$vmName = $_.Name
$_.Value.Custom.getenumerator() | %{
$row = New-Object psobject
$row | Add-Member -memberType noteproperty -name VmName -value $vmName
$row | Add-Member -memberType noteproperty -name CAName -value $customKeyLookup[$_.Name]
$row | Add-Member -memberType noteproperty -name CAValue -value $_.Value
$list += $row
}
}
$list | Export-Csv “C:\CA-backup.csv” -noTypeInformation
    4) Import all custom attributes from a CSV file
# Import all custom attributes from a CSV file (restore)
Import-Csv “C:\CA-backup.csv” | %{
$vmArray[$_.VmName].Object.setCustomValue($_.CAName,$_.CAValue)
}
    5) Add/update a custom field
# Add/update a custom field, named MyField# with a new value for all guests whose name starts with “PC”
$field = “MyField”
$newValue = “1234″
if (-not $customNameLookup.ContainsKey($field)){
$customFieldMgr.AddCustomFieldDef($field, “VirtualMachine”, $null, $null)
}
$vmArray.GetEnumerator() | where {$_.Name -like “PC*”} | % {
$_.Value.object.setCustomValue(“MyField”,$newValue)
}
    6) Find all guests that have a specific custom attribute set to a specific value
# Find all guests that have MyField=”yes” (case sensitive)
Write-Host “Find all with MyField=yes” -fore red
$tgtKey = $customNameLookup["MyField"]
$vmArray.GetEnumerator() | where {$_.Value.Custom[$tgtKey] -ceq “yes”} | %{
$_.Name
}
    7) Update a specific custom field
# Update custom field, named VCBDRM. Change all “yes” to “Yes”
Write-Host “Update all VCBDRM=Yes to VCBDRM=yes” -fore red
$tgtKey = $customNameLookup["VCBDRM"]
$vmArray.GetEnumerator() | where {$_.Value.Custom[$tgtKey] -ceq “Yes”} | %{
$_.Value.object.setCustomValue(“VCBDRM”,“yes”)
}

Now that we have all the functionality we need it’s time to see if we were able to lower the execution time.In my limited test environment I was able to execute the custom attribute updates in 1/40th of the original time.

For a test in a bigger environment I got the help from Arnim van Lieshout (thanks Arnim).

He was able to bring the execution time of one custom attribute update on all the guests in his VI environment down from 70 seconds to less than 1 second.

Arnim’s environment contained 842 guests.

It would mean that he could do his planned updates of the custom attributes on all the guests in less than 1 hour, where he previously would have needed more than 65 hours.

In the mean time I received confirmation from Carter Shanklin that there is indeed a bug in the current release of the Set-CustomField cmdlet that could explain part of this behaviour.

My conclusion, it is sometimes useful to dive into the vSphere APIs and to learn the more advanced PowerShell techniques, like hash tables.