Creating re-usable PowerCLI functions with “Onyx for the Web Client”

Onyx is a tool I have been using for years, it’s a tool that sat between the vSphere C# Client or Windows Client and the vCenter server and intercepted the SOAP calls, it would inspect these SOAP calls and convert them into PowerCLI code.  With the recent move away from the C# Client there is an obvious reason why the original Onyx Fling would not work anymore, its not like you can put it in between the web client and the vCenter service is it ?!

This tool was always a life saver for me as I’m not one to read API documentation, it was always a way to not only work out the code for me to alter and reuse but it was a great way for me to work out how VMware would do something, there is often multiple ways you can code something and if you can see what the C# client was doing to perform an action then that had to be a great place to start.

So up until now it has always been for the C# client, well today all that changes, just released is “Onyx for the Web Client” and it is awesome!

Firstly I want to thank the engineer who worked on it, Atanas Atanasov is a member of the PowerCLI team and was fantastic throughout the project taking feedback from myself and the internal testers at VMware (You know who you are and thanks!) and turning it around very fast to produce the end result which works fantastically.

So how does it work?

There are already some great blog posts out there on the official PowerCLI Blog, on Luc’s Site, Jonathans Site and on Roberts site which go into some detail so I thought I would do a quick video to show you how I like to use it.

In the video below I show how I do not know how to automate an advanced setting of a VM, I show how easy it is to record the action of changing that setting, take the code that the fling produces and then use it to firstly change a single VM to a different setting and then how easy it is to turn this into a function that can be used on all VMs in the environment.  This shows how we can create repeatable code that can be used (and shared) to achieve the task we need all with little knowledge of the API.

Download it now!

Make sure you download and install the fling in your test environment from here.

Watch how to use the fling

PowerCLI 6.0 and vCloud Air

One of the great enhancements in the recently released PowerCLI 6.0 is the ability to connect to vCloud Air and vCloud Air Networks, Massimo has already started doing a great write-up on these over at the vCloud Air Blog here.

So why is this a great enhancement?  Well any new area that supports PowerCLI or PowerShell is a great enhancement to me, I mean being able to use PowerCLI to take your automation one step further is always a bonus!  I love the fact that we can now from one shell manage our vSphere environments and our vCloud Air environments for troubleshooting, reporting, provisioning and general automation.  This also goes along with VMware’s key focus of being able to use the same tools to seamlessly extend your datacenter into the cloud by allowing management with the same tools you use to manage your vSphere environment.  On top of this, the ability to manipulate data in both areas and integrate scripts between your on-premises datacenter and your vCloud Air datacenter opens up some great possibilities, imagine being able to automatically spin up new VMs in the cloud when a script works out you need more resources and automatically adding and removing these to a load balancer etc. True hybrid computing!

 

Anyway, I could come up with use cases for this all day but lets get down to business and explain how to get started, firstly, read Massimo’s great posts here, secondly lets talk about connecting, there are a couple of tricks I have been using for a while now in vSphere that are transferable skills and also work in the vCloud Air environment that make it more convenient to use.

Connecting to vCloud Air with Username and Password

Firstly its easy to connect to vCloud Air and specify your username and password, this would look like the following:

image

The first thing you will notice is that we didn’t specify a server to connect to, with the Connect-PIServer cmdlet if no server is specified it uses the default of vchs.vmware.com, one less parameter to specify but what id I don’t want to give my username and password as part of the script?  What if I am giving a demo and I don’t want people to see my password?

Connecting to vCloud Air with interactive prompt

If we want to specify a username and password interactively we can just use Connect-PIServer, this is assuming you have not stored credentials in the VICredentialStore, which we will cover next.

image

Storing your credentials

This is the method I use the most, with this method I can have a highly secure random password and not need to remember it to connect to vCloud Air, this uses the same method of saving credentials that we use for vSphere environments and even the same secure file.  There are a couple of ways we can do this, the first is with the –SaveCredntials parameter as below:

image

Or use the New-VICredentialStoreItem cmdlet to create a credential item for vchs.vmware.com

image

Once you have the credentials stored you can manage them with the Set and Remove –VICredentialStoreItem cmdlets.

Now this has been done if we do not specify a credential it will check the VICredentialStore for credentials and use these so connecting to vCloud Air now becomes a single cmdlet:

image

Listing the Datacenters and Connecting

Now we have made the initial connection to vCloud Air we can easily list the datacenters we have access to and connect to them, firstly lets list the vCloud Air Datacenters with Get-PIDatacenter and then connect to them with Connect-PIDatacenter (which is actually an alias to the Connect-CIServer cmdlet).

image

Now you have some tips and can see how easy it is to connect to vCloud Air, in future posts I will make use of this connection to start automating vCloud Air and providing PowerCLI enabled resources to use with vCloud Air.

Horizon Mirage joins the PowerCLI club

VMware Horizon Mirage is a product in VMware’s EUC range of products that manage images across physical desktops and point of service devices. It uses something called layers to simplify OS and application rollouts. With Mirage, devices are separated into logical layers that can be controlled by the admin or end users. Admins can streamline OS and application layer updates, and end users can keep their personalized settings. Horizon Mirage enables quick OS updates and migrations, application upgrades, and hardware upgrades. Admins can also quickly enroll new PCs with bare metal provisioning.

This has historically been managed via a MMC style console and recently a web interface but new to this product from VMware Horizon Mirage 5.1 is its integration into PowerCLI and the ability of PowerCLI Cmdlets.  The Cmdlets are installed where you would install the Mirage Management Console, normally this is on the Administrators workstation.  Once installed you can connect to the Mirage Server and use the cmdlets to work with Mirage.

Download Mirage and Mirage PowerCLI here: https://my.vmware.com/web/vmware/details?productId=407&downloadGroup=MIRAGE-510

Cmdlets

Name Synopsis
Apply-MirageOsMigration This cmdlet applies download only migrations.
Archive-MirageCvd This cmdlet archives the CVD.
Connect-MirageServer This cmdlet configures a connection to the Mirage API server.
Disconnect-MirageServer Disconnects from the Mirage server.
Get-MirageAppLayer This cmdlet returns the app layers from the Mirage system.
Get-MirageBaseLayer This cmdlet returns the base layers from Mirage system.
Get-MirageCvd This cmdlet returns the CVDs from Mirage system.
Get-MirageCvdCollection This cmdlet retrieves the collections from the Mirage system.
Get-MirageOsMigration This cmdlet retrieves the download only migrations from the Mirage system.
Get-MiragePendingDevice This cmdlet retrieves the pending devices from Mirage system.
Get-MiragePolicy This cmdlet retrieves the policies from the Mirage system.
Get-MirageVolume This cmdlet retrieves the volumes from the Mirage system.
New-MirageCvd This cmdlet creates a new CVD with the specified policy and volume in the Mirage sytstem.
New-MirageOsMigration This cmdlet migrates CVD with the specified base layer, app layer and related info in the Mirage system.
Remove-MirageCvd This cmdlet removes the CVD.
Set-MirageCvd This cmdlet updates the CVD with the specified policy.
Sync-MirageCvd This cmdlet synchronizes the CVD’s device information.

Screenshots

image

image

Retrieving VM Metrics from vCloud Air

Recently I was working with vCloud air and PowerCLI and was asked if there was a way to retrieve the statics of a VM and export these to get a better idea of the utilization of that VM.

Working with vCloud Air and the API through the PowerCLI cmdlets I was able to use the GetMetricsCurrent() and GetMetricsHistoric() to retrieve both the current VM Metrics and also the historic metrics of a VM.  The type of metrics and their values is described here.

VM Current Metrics

Current metrics are instantaneous values collected at the time the request is processed. The GetMetricsCurrent() method retrieves a list of current metric values, one for each category. I put this into a handy function which can be found at the end of this post, with this function we can pass vCloud AIR VMs into it and retrieve the current metrics.

image

VM Historic Metrics

Historic metrics are collected and stored for 14 days. A call to the GetMetricsCurrent() method retrieves the past 24 hours of metric history.  again I created a function that allows us to select a VM (or multiple) and get the historic data for this VM.

image

You will see that this time the method returns a collection of results in the Sample entry, we can then expand this to return the results for the metric over the last 24 hours.

image

Creating a VM Statistical report

Of course with this being PowerShell we can add value to these results, I mean who wants to look at a long list of numbers, wouldn’t you rather see a pretty graph?  We can easily use Luc’s great Export-XLSX function to export the results for each metric straight into Excel and even create us a nice graph for each statistic:

image

image

The Functions

Make sure you download Lucs Export-Xlsx function from here, and below are the functions I used to retrieve this information from vCloud Air.

Function Get-CIVMMetric {
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelinebyPropertyName=$True)]
		$CIVM
	)
	Process {
		foreach ($VM in $CIVM) {
			If ($VM.Status -ne "PoweredOn") {
				Write-Host "$VM must be Powered On to retrieve metrics"
				exit
			} Else {
				$Metrics = $CIVM.ExtensionData.GetMetricsCurrent()
				$metrics.Metric
			}
		}
	}
}

Function Get-CIVMHistoricalMetric {
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelinebyPropertyName=$True)]
		$CIVM
	)
	Process {
		foreach ($VM in $CIVM) {
			If ($VM.Status -ne "PoweredOn") {
				Write-Host "$VM must be Powered On to retrieve metrics"
				exit
			} Else {
				$Metrics = $CIVM.ExtensionData.GetMetricsHistoric()
				$metrics.MetricSeries
			}
		}
	}
}

# Get the Current Metrics for the VM
Get-CIVM sql-w-01a | Get-CIVMMetric

# List the Available Historical Metrics for the VM
Get-CIVM sql-w-01a | Get-CIVMHistoricalMetric

# Get the CPU Historical Metrics
Get-CIVM sql-w-01a | Get-CIVMHistoricalMetric | Where {$_.Name -eq "cpu.usage.average"} | Select -ExpandProperty Sample

$VMName = "sql-w-01a"
Foreach ($stat in (Get-CIVM $VMName | Get-CIVMHistoricalMetric)){
    Foreach ($Entry in $Stat) {
        $Data = $Entry.Sample | Select Timestamp, @{Name=($Entry.Name);Expression={$_.Value}}
        Export-Xlsx $Data C:\Temp\$VMName.xlsx -WorksheetName ($Entry.Name) -ChartType "xlXYScatterSmooth" -AppendWorksheet
    }
}

Receiving alerts on if a VM has over a given number of snapshots

Last week I was asked if there was an easy way to get an alert once a VM had over a certain number of snapshots, this is a frequent request as it would seam some backup products like to keep snapshoting VMs and note removing them.

There are several ways this could be kicked off, including setting a vCenter Alarm on snapshots and getting it to run a script but in this case it was simple, they wanted to run a scheduled task once a day and get an email listing the VMs that had over x amount of snapshots and the details about the last one created, of course it was a 5 minute script but I thought I would blog it anyway for others to use if they needed it.  Enjoy

 

BTW, its also very easy to get this to work with gmail if you use that as your email provider, just replace the Send-EmailMessage line with the below:

$SMTPClient = New-Object Net.Mail.SmtpClient($SmtpServer, 587)
$SMTPClient.EnableSsl = $true
$SMTPClient.Credentials = New-Object System.Net.NetworkCredential(“my.email@gmail.com”, “mYpa$$w0rd”);
$SMTPClient.Send($EmailFrom, $EmailTo, $Subject, $Body)

 

Example email

image

Script

Automated deployment of vCloud Networking and Security 5.5

A while ago I posted a PowerCLI method for deploying vShield 5.0 which can be found here, as mentioned in the comments of that post this did not work for vCNS 5.5, this is because the process of deploying it changed and actually became a lot neater than what it was before.  Before we had to reach inside the guest OS and mess around with configuration files.  In the 5.5 release the vCNS team made some changes to make this easier for people to automatically deploy,

 

I actually wrote this a while ago and forgot about it until recently when I was asked if it could be done, this was a great reminder for me to post the code for others to use.

 

You will see in the code that we need to specify some variables at the start of the script, you will see in the code that you need to specify a cluster to deploy vCNS to, as part of this script it automatically picks a random host and a shared datastore which has enough room to deploy the appliance to, this is just an example and can be changed to your requirements as needed – I was just having fun with a kind of automated placement Winking smile

You will need PowerCLI 5.8 R1 for this and don’t forget, once vShield has been deployed you can easily take the configuration further by using the vShield module I created here.   For other examples of deployment scripts for VMware appliances check out the repo here and contribute!

Example output

image

Code

Automating Tags and Tag Category creation and assignment with PowerCLI

Fimageor a couple of releases now PowerCLI has been able to work with vSphere Tags, A tag is a label that you can apply to objects in the vSphere inventory.

After creating a tag, you can assign that tag to a category. Categories allow you to group related tags together. When you define a category, you can also specify the type of objects to which its tags can be applied to and whether more than one tag in the category can be applied to an object.

For example, if you want to tag your virtual machines by the owner, you can create a category called “Owner” and specify that it applies to virtual machines only and that only a single tag can be applied to a virtual machine at any time. The tags in this category could be Alan, John or Fred etc.

I have had a few people ask me how they can use PowerCLI to work with external systems, CMDBs, databases or even just a CSV file.

One example of this is where a company could have various information about hosts or datastores or virtual machines, like the project that purchased these, a cost code or an owner.  This data is generally stored somewhere else but it would be great to see this information straight in the vSphere Web Client where you manage the objects so that you can instantly contact the owner or work out which project the object is being used for etc.

The below video shows how we can use PowerCLI and this generic script I created to import the data, create the tags and tag categories and assign them to the machines, it uses a csv as input but this could obviously be changed to anything which can be read in PowerShell, like a API, database, application etc etc.

Automating tags and tag categories video

Example Script

This script is the script I created as an example which relates each of the items in the Name column to an object in the inventory then for each of the other column headers it will create a category and then the tags that are under the categories, once this has been done it will apply the tags to the objects in the Name column.

Connect-viserver myvc.corp.local -user administrator@vsphere.local -pass Pa$$w0rd
$CMDBInfo = Import-CSV C:\Software\cmdbinfo.csv

# Get the header names to use as tag category names
$TagCatNames = $cmdbinfo | Get-Member | Where {$_.MemberType -eq "NoteProperty"} | Select -Expand Name

# Create the Tag Category if it doesnt exist
Foreach ($Name in ($TagCatNames | Where {$_ -ne "Name"})) {
    if (-Not (Get-TagCategory $Name -ea SilentlyContinue)) {
        Write-Host "Creating Tag Category $Name"
        New-TagCategory -Name $Name -Description "$Name from CMDB" | Out-Null
        
        # Create Tags under the Tag Categories
        $UniqueTags = $cmdbinfo | Select -expand $name | Get-Unique
        Foreach ($Tag in $UniqueTags) {
            if (-Not (Get-Tag $Tag -ea SilentlyContinue)) {
                Write-Host "..Creating Tag under $Name of $Tag"
                New-Tag -Name $Tag -Category $name -Description "$Tag from CMDB" | Out-Null
            }
            # Assign the Tags to the VMs/Hosts
            $cmdbinfo | Where {$_.($Name) -eq $Tag} | Foreach {
                Write-Host ".... Assigning $Tag in Category of $Name to $($_.Name)"
                $TagAssignment = Get-Tag -Category $Name -name $Tag
                New-TagAssignment -entity $($_.Name) -Tag $Tagassignment | Out-Null
            }
        }          
    }
}

PowerShell User Group–Campbell–CA–Nov 5th

Are you interested in PowerShell? Do you want free Pizza? If the answer to either of these questions is YES then join us!

Are you in the Campbell, California area on 5th Nov 2014?

I will be presenting about PowerCLI and covering the following areas:

  • Introduction to managing VMware products with PowerShell
  • Reporting
  • Deployment
  • Automation
  • Integration
  • Community
  • Where are VMware headed?
  • Question time!

If you are not interested in managing VMware products with PowerShell then attend anyway, we will no doubt reference lots of PowerShell general tips and tricks and definitely have some great conversations about PowerShell in general.

When and Where?

Wednesday, November 5, 2014

6:00 PM to 9:00 PM

 

Groupware Corporate Office

550 Division Street, Campbell, CA (map)

RSVP

Make sure you sign up here.

PowerCLI in the vSphere Web Client–Announcing PowerActions

You don’t know how excited I am to write this!  Around a year ago I presented something we were working on internal as a tech preview for my VMworld session, the response was phenomenal, if you were there you would remember people standing up and clapping and asking when this awesomeness would be available, its taken a while but its here and its worth the wait.  So what is this that I am so excited about?

 

PowerActions is a new fling from VMware which can be downloaded here, it adds the automation power of PowerCLI into the web client for you to use your scripts and automation power back inside the client, have you ever wanted to right click an object in the web client and run a custom automation action, maybe return the results and then work with them further all from the convenience of the web client…. Now you can!

This works in 2 ways….

Console

PowerShell console in the Web Client

Firstly you can access a PowerCLI console straight in the web interface, even in safari, this fling allows for a dedicated host to be used as a PowerShell host and this machine will be responsible for running the PowerCLI actions, once its setup you will access the console from within the web client and all commands will run remotely on the PowerShell host, it even uses your current logged on credentials to launch the scripts meaning you don’t have to connect the PowerCLI session.

 

You can use tab completion on your cmdlets and even use other PowerShell snapins and modules to control any PowerShell enabled infrastructure to extend your automation needs within the vSphere Web Client.

 

MenuRight Click your objects

Secondly you can now right click an object in the Web Client and create a dedicated script which will work against this object, have the ability to extend your web client and take the object as an input to use inside your script.

This comes with 2 options, Create a script and also execute a script.

 

My Scripts and Shared Scripts

Not only can you create your own scripts to run against objects in the web client but advanced admins can create scripts and share them with all users of the web client by storing them in the Shared Scripts section of this fling, read the documentation to find out more about how to do this.  This gives the great ability to have not only shared scripts but actually a golden set of scripts which all users of the web client can use while you keep your items in a separate area “My Scripts”, enabling each user to have their own custom actions.

 

Download and read more

Download the fling from the VMware Labs site here, also make sure you grab the document from the same site and also check out the great post on the PowerCLI Blog for more information here.

 

Check out the video for a quick introduction

To help with the details I shot a quick install and usage video that covers the basics, make sure you read the PDF that comes with the fling and make sure you are active, if you like this then let is know, if you want more then let us know…. basically give us feedback!

Deploying Log Insight with PowerCLI 5.8 R1 new features

PowerCLI 5.8 R1 was recently released and of the great new features included was the ability to now set the OVF/OVA properties of a VM or vAPP stored in an Open Virtualization format.

The new cmdlet allows us to query the file for the properties and interactively find out which entries we need to fill out, we can then fill these entries out and send them as part of the deployment of the VM when using the Import-vApp cmdlet which has been a part of PowerCLI for a while now.  For more information on how we do this check out the recent post on the PowerCLI blog here.

The bit I like about this cmdlet is that we can now dynamically work out which host, network and storage to put the VM on based on our own metrics, like say for instance I wanted to choose the host with the most amount of memory in a cluster or a datastore on that host that had the most room left:

$VMHost = Get-Cluster "Cluster Site A" | Get-VMHost | Sort MemoryGB | Select -first 1
$Datastore = $VMHost | Get-datastore | Sort FreeSpaceGB -Descending | Select -first 1

Another method to see the entries and what needs filling out as part of the OVF/OVA Deployment is to use the cmdlet to store the information inside a variable as shown in the video I created below, this can be used to browse through the properties and fill them out as needed or create a script for later deployment.  Another option for this is to load the settings into a variable and then use one of the advanced PowerShell editors like PowerGUI Script Editor to click through the variable in the variable pane and explore the options and their values as can be seen below:

image

The code

I want to automate the entire build out of my virtual infrastructure like this, I hear you say…. well why not help out, get involved, I started a new Github project where we can all contribute to make sure we are not reinventing the wheel, why not check out the github project here and add your own deployment scripts, so far there are scripts to deploy both Log Insight 2.0 (the one shown in the video) and also vShield Manager 5.5.x.

 

Video: Deploying Log Insight 2.0

Virtually everything is POSHable