Category Archives: Automation

PowerCLI 6.0 and vCloud Air

1 Reply

One of the great enhancements in the recently released PowerCLI 6.0 is the ability to connect to vCloud Air and vCloud Air Networks, Massimo has already started doing a great write-up on these over at the vCloud Air Blog here.

So why is this a great enhancement?  Well any new area that supports PowerCLI or PowerShell is a great enhancement to me, I mean being able to use PowerCLI to take your automation one step further is always a bonus!  I love the fact that we can now from one shell manage our vSphere environments and our vCloud Air environments for troubleshooting, reporting, provisioning and general automation.  This also goes along with VMware’s key focus of being able to use the same tools to seamlessly extend your datacenter into the cloud by allowing management with the same tools you use to manage your vSphere environment.  On top of this, the ability to manipulate data in both areas and integrate scripts between your on-premises datacenter and your vCloud Air datacenter opens up some great possibilities, imagine being able to automatically spin up new VMs in the cloud when a script works out you need more resources and automatically adding and removing these to a load balancer etc. True hybrid computing!

 

Anyway, I could come up with use cases for this all day but lets get down to business and explain how to get started, firstly, read Massimo’s great posts here, secondly lets talk about connecting, there are a couple of tricks I have been using for a while now in vSphere that are transferable skills and also work in the vCloud Air environment that make it more convenient to use.

Connecting to vCloud Air with Username and Password

Firstly its easy to connect to vCloud Air and specify your username and password, this would look like the following:

image

The first thing you will notice is that we didn’t specify a server to connect to, with the Connect-PIServer cmdlet if no server is specified it uses the default of vchs.vmware.com, one less parameter to specify but what id I don’t want to give my username and password as part of the script?  What if I am giving a demo and I don’t want people to see my password?

Connecting to vCloud Air with interactive prompt

If we want to specify a username and password interactively we can just use Connect-PIServer, this is assuming you have not stored credentials in the VICredentialStore, which we will cover next.

image

Storing your credentials

This is the method I use the most, with this method I can have a highly secure random password and not need to remember it to connect to vCloud Air, this uses the same method of saving credentials that we use for vSphere environments and even the same secure file.  There are a couple of ways we can do this, the first is with the –SaveCredntials parameter as below:

image

Or use the New-VICredentialStoreItem cmdlet to create a credential item for vchs.vmware.com

image

Once you have the credentials stored you can manage them with the Set and Remove –VICredentialStoreItem cmdlets.

Now this has been done if we do not specify a credential it will check the VICredentialStore for credentials and use these so connecting to vCloud Air now becomes a single cmdlet:

image

Listing the Datacenters and Connecting

Now we have made the initial connection to vCloud Air we can easily list the datacenters we have access to and connect to them, firstly lets list the vCloud Air Datacenters with Get-PIDatacenter and then connect to them with Connect-PIDatacenter (which is actually an alias to the Connect-CIServer cmdlet).

image

Now you have some tips and can see how easy it is to connect to vCloud Air, in future posts I will make use of this connection to start automating vCloud Air and providing PowerCLI enabled resources to use with vCloud Air.

Retrieving VM Metrics from vCloud Air

7 Replies

Recently I was working with vCloud air and PowerCLI and was asked if there was a way to retrieve the statics of a VM and export these to get a better idea of the utilization of that VM.

Working with vCloud Air and the API through the PowerCLI cmdlets I was able to use the GetMetricsCurrent() and GetMetricsHistoric() to retrieve both the current VM Metrics and also the historic metrics of a VM.  The type of metrics and their values is described here.

VM Current Metrics

Current metrics are instantaneous values collected at the time the request is processed. The GetMetricsCurrent() method retrieves a list of current metric values, one for each category. I put this into a handy function which can be found at the end of this post, with this function we can pass vCloud AIR VMs into it and retrieve the current metrics.

image

VM Historic Metrics

Historic metrics are collected and stored for 14 days. A call to the GetMetricsCurrent() method retrieves the past 24 hours of metric history.  again I created a function that allows us to select a VM (or multiple) and get the historic data for this VM.

image

You will see that this time the method returns a collection of results in the Sample entry, we can then expand this to return the results for the metric over the last 24 hours.

image

Creating a VM Statistical report

Of course with this being PowerShell we can add value to these results, I mean who wants to look at a long list of numbers, wouldn’t you rather see a pretty graph?  We can easily use Luc’s great Export-XLSX function to export the results for each metric straight into Excel and even create us a nice graph for each statistic:

image

image

The Functions

Make sure you download Lucs Export-Xlsx function from here, and below are the functions I used to retrieve this information from vCloud Air.

Function Get-CIVMMetric {
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelinebyPropertyName=$True)]
		$CIVM
	)
	Process {
		foreach ($VM in $CIVM) {
			If ($VM.Status -ne "PoweredOn") {
				Write-Host "$VM must be Powered On to retrieve metrics"
				exit
			} Else {
				$Metrics = $CIVM.ExtensionData.GetMetricsCurrent()
				$metrics.Metric
			}
		}
	}
}

Function Get-CIVMHistoricalMetric {
	Param (
		[Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelinebyPropertyName=$True)]
		$CIVM
	)
	Process {
		foreach ($VM in $CIVM) {
			If ($VM.Status -ne "PoweredOn") {
				Write-Host "$VM must be Powered On to retrieve metrics"
				exit
			} Else {
				$Metrics = $CIVM.ExtensionData.GetMetricsHistoric()
				$metrics.MetricSeries
			}
		}
	}
}

# Get the Current Metrics for the VM
Get-CIVM sql-w-01a | Get-CIVMMetric

# List the Available Historical Metrics for the VM
Get-CIVM sql-w-01a | Get-CIVMHistoricalMetric

# Get the CPU Historical Metrics
Get-CIVM sql-w-01a | Get-CIVMHistoricalMetric | Where {$_.Name -eq "cpu.usage.average"} | Select -ExpandProperty Sample

$VMName = "sql-w-01a"
Foreach ($stat in (Get-CIVM $VMName | Get-CIVMHistoricalMetric)){
    Foreach ($Entry in $Stat) {
        $Data = $Entry.Sample | Select Timestamp, @{Name=($Entry.Name);Expression={$_.Value}}
        Export-Xlsx $Data C:\Temp\$VMName.xlsx -WorksheetName ($Entry.Name) -ChartType "xlXYScatterSmooth" -AppendWorksheet
    }
}

PowerCLI in the vSphere Web Client–Announcing PowerActions

9 Replies

You don’t know how excited I am to write this!  Around a year ago I presented something we were working on internal as a tech preview for my VMworld session, the response was phenomenal, if you were there you would remember people standing up and clapping and asking when this awesomeness would be available, its taken a while but its here and its worth the wait.  So what is this that I am so excited about?

 

PowerActions is a new fling from VMware which can be downloaded here, it adds the automation power of PowerCLI into the web client for you to use your scripts and automation power back inside the client, have you ever wanted to right click an object in the web client and run a custom automation action, maybe return the results and then work with them further all from the convenience of the web client…. Now you can!

This works in 2 ways….

Console

PowerShell console in the Web Client

Firstly you can access a PowerCLI console straight in the web interface, even in safari, this fling allows for a dedicated host to be used as a PowerShell host and this machine will be responsible for running the PowerCLI actions, once its setup you will access the console from within the web client and all commands will run remotely on the PowerShell host, it even uses your current logged on credentials to launch the scripts meaning you don’t have to connect the PowerCLI session.

 

You can use tab completion on your cmdlets and even use other PowerShell snapins and modules to control any PowerShell enabled infrastructure to extend your automation needs within the vSphere Web Client.

 

MenuRight Click your objects

Secondly you can now right click an object in the Web Client and create a dedicated script which will work against this object, have the ability to extend your web client and take the object as an input to use inside your script.

This comes with 2 options, Create a script and also execute a script.

 

My Scripts and Shared Scripts

Not only can you create your own scripts to run against objects in the web client but advanced admins can create scripts and share them with all users of the web client by storing them in the Shared Scripts section of this fling, read the documentation to find out more about how to do this.  This gives the great ability to have not only shared scripts but actually a golden set of scripts which all users of the web client can use while you keep your items in a separate area “My Scripts”, enabling each user to have their own custom actions.

 

Download and read more

Download the fling from the VMware Labs site here, also make sure you grab the document from the same site and also check out the great post on the PowerCLI Blog for more information here.

 

Check out the video for a quick introduction

To help with the details I shot a quick install and usage video that covers the basics, make sure you read the PDF that comes with the fling and make sure you are active, if you like this then let is know, if you want more then let us know…. basically give us feedback!

VMworld 2014 Orchestration, Automation & Optimization Panel

Leave a reply

One of the highlights for VMworld 2014 in San Francisco this year for me was on the first day, I was asked by the vBrownbag team to sit on the “Opening Acts 2014 Orchestration, Automation, & Optimization Panel”, we had some great conversations and a few laughs along the way as well.  I have posted the video here for you to watch, also make sure you check out the complete list of Opening Act Panels from the vBrownbag Opening Acts feed here.

 

PowerCLI 5.5 R2 Released

5 Replies

imageJust in case you don’t read the vSphere PowerCLI Official Blog I thought I would let you know that the latest version of PowerCLI greatness is out and its pretty awesome!

In this release there are a lot of new features, not to mention the first support for Site Recovery Manager, cmdlets for creating Tags and Tag Categories, Cmdlets for working with EVC and much much more, see the full list below for the details:

  • Manage vCenter Site Recovery Manager
  • Create and remove tags and tag categories
  • Retrieve information and configure Enhanced vMotion Compatibility (EVC) mode on clusters.
  • Manage security policies for vSphere standard switches and port groups.
  • Support for Windows PowerShell 4.0.
  • Support for vSphere servers configured with IPv6.
  • Specify the priority of a VM migration
  • Provide a Hard Disk object to the RelatedObject of Get-Datastore
  • Get-Datastore cmdlet to allow filtering by cluster.
  • Get-Stat and Get-StatType now works with all types
  • Added support for e1000e network adapter type.
  • Specify all values for DiskStorageFormat when cloning a virtual machine
  • 64-Bit Support for New-OSCustomizationSpec and Set-OSCustomizationSpec
  • ToolsVersion property to VMGuest that shows the version as a string.
  • Provide a virtual portgroup to the RelatedObject of the Get-VirtualSwitch and Get-DVSwitch
  • Retrieve virtual machines by virtual switches.
  • Other bug fixes and general performance enhancements have been made to various PowerCLI cmdlets.

Download it and read more

Download and Install PowerCLI 5.5 Release 2 today from here.

For a more comprehensive list including all parameter and functional improvements, security enhancements, and deprecated features, see the vSphere PowerCLI Release Notes and the vSphere PowerCLI Change Log.

For more information on specific product features, see the VMware vSphere PowerCLI 5.5 Release 2 User’s Guide.

For more information on specific cmdlets, see the VMware vSphere PowerCLI 5.5 Release 2 Cmdlet Reference.

Check out the details

On the day of the release I presented on the vBrownBag Community Podcast, this was great for showing people exactly how to use the SRM Cmdlets and how to understand the sample scripts we have, I went a little long in the presentation but grab a coffee (or 3) and check it out here:

Automation Tip 7–Testing

Leave a reply

image

One of the great things about automation is the ability to achieve consistent results, same thing every time, but go careful. What happens for example if your automation task is incorrect, if it doesn’t achieve what you want it to do.

Guess what?

You will get the undesired result every time.  Or as I like to say, you will retrieve mass produced crap!

This is exactly why testing in automation is key, both functional and scale testing, what happens when you start working at a larger scale, do things still react in the same way, do you still get the desired results?

Sometimes testing will show areas where throttling mechanisms may need to be used, take for example the ability to create a virtual machine, when we automate this we may produce 10 virtual machines in the same timescale, maybe 100, 1000 or even more.  Does your hardware allow you to do this? How does the storage or the network or the IP management system manage working at this rate? Throttling may cause a slight delay in the process but it will certainly ensure everything is available for future workflows.

In summary, testing will help you be aware of your bottlenecks and stop mass produced crap!

More Automation tips

More tips from this series created by myself and Thomas Corfmat for a session at VMworld 2013 can be found below (updated as published), if you have a login for VMworld.com you can also watch the full session here.

Automation Tip 1 – Measure It!

Automation Tip 2 – Find your standards

Automation Tip 3 – Its all about people

Automation Tip 4 – Select the right mix of people

Automation Tip 5 – Simplify, Don’t automate a broken process

Automation Tip 6 – Automate incrementally

Automation Tip 7 – Testing

Automation Tip 8 – Documentation

Automation Tip 9 – End to end automation

Automation Tip 10 – Follow the “Automation Triangle”

Automation Tip 6–Automate Incrementally

1 Reply

image

I think the correct phase for this one is “Don’t try and boil the Ocean”, if you try and automate everything at once you will end up with a broken system and will ultimately fail.

Identify the areas that cause you the most pain at the moment, the tasks that take the most time or need the most people to currently work.  Once you have identified these areas take a look at the tasks that make up this process.   What part of the process is the biggest pain today, the one that is loosing you the most time and effecting service delivery? That’s a great place to start.

Once you have automated this area and showed the results that have been achieved by automation, only then move on to the next item that causes you pain. Following this iterative approach and over time you fill find that you have automated a good amount of your datacenter operations and over time the cost of automation will exceed the value.

More Automation tips

More tips from this series created by myself and Thomas Corfmat for a session at VMworld 2013 can be found below (updated as published), if you have a login for VMworld.com you can also watch the full session here.

 

Automation Tip 1 – Measure It!

Automation Tip 2 – Find your standards

Automation Tip 3 – Its all about people

Automation Tip 4 – Select the right mix of people

Automation Tip 5 – Simplify, Don’t automate a broken process

Automation Tip 6 – Automate incrementally

Automation Tip 7 – Testing

Automation Tip 8 – Documentation

Automation Tip 9 – End to end automation

Automation Tip 10 – Follow the “Automation Triangle”

Automation Tip 5 – Simplify, Don’t automate a broken process

1 Reply

image

The most important item in automation is the process, if the process is broken so will the automated process be, automation is not a magic wand that will suddenly fix any issues with processes you have at the moment.

Automating a task does however give you a good chance to review a process and think about if this is still the best way to achieve things, often processes are organically grown and are not reviewed, take this opportunity to draw out the process with the people involved, make it as simple as possible, do you still need 17 approvals?

Remember, some steps just can not be automated, make sure you identify these and look for a way in the future to resolve these, its not ideal but sometimes the only way to succeed is by automating most of the work and then processing the manual tasks at the end or in worst case half way through, in the past I have seen prompts to an operator or an email telling them what to do before clicking on a link to perform the rest of the automation task.

Sometimes there are tasks that simply can not be automated, these are far and few between but don’t worry, don’t try and jam them in to an automated process, move on and return later, think about how to mitigate this when looking at new software or SDDC solutions.

Simplify, it will save you time and it will make you automation tasks much easier to implement and debug later!

More Automation tips

More tips from this series created by myself and Thomas Corfmat for a session at VMworld 2013 can be found below (updated as published), if you have a login for VMworld.com you can also watch the full session here.

 

Automation Tip 1 – Measure It!

Automation Tip 2 – Find your standards

Automation Tip 3 – Its all about people

Automation Tip 4 – Select the right mix of people

Automation Tip 5 – Simplify, Don’t automate a broken process

Automation Tip 6 – Automate incrementally

Automation Tip 7 – Testing

Automation Tip 8 – Documentation

Automation Tip 9 – End to end automation

Automation Tip 10 – Follow the “Automation Triangle”

Automation Tip 4–Select the right mix of people

2 Replies

image

When automating, make sure you think about automating the solution end-to-end, as part of this you will need to talk to domain experts, the people with tribal knowledge and normally admin rights to the systems.  The people who know not only how the systems work in your environment but also have the specific process knowledge for that area.  These people will help you automate the task so much quicker and easier.  This will also go a long way to gaining their buy in for the task and the areas that will be handed off and covered by this automation process.

It helps if these people already know what you are trying to achieve and even more if they have previous knowledge on automation, scripting or development.  Creating a cross departmental or technology automation task force* will help with building relationships and pave the path for future automation projects.

*YMCA dance not necessary when forming automation task force!

 

More Automation tips

More tips from this series created by myself and Thomas Corfmat for a session at VMworld 2013 can be found below (updated as published), if you have a login for VMworld.com you can also watch the full session here.

 

Automation Tip 1 – Measure It!

Automation Tip 2 – Find your standards

Automation Tip 3 – Its all about people

Automation Tip 4 – Select the right mix of people

Automation Tip 5 – Simplify, Don’t automate a broken process

Automation Tip 6 – Automate incrementally

Automation Tip 7 – Testing

Automation Tip 8 – Documentation

Automation Tip 9 – End to end automation

Automation Tip 10 – Follow the “Automation Triangle”

Automation Tip 3–Its all about people

3 Replies

image

 

Remember, when automating its all about the people, you will be crossing boundaries, sometimes annoying people by accessing their systems and potentially removing some of the work they need to do in a manual process.  This is why its key to get a corporate sponsor or champion that fully backs what you are trying to do and agrees with your vision.  You will need their backup and help!

 

Tell people about what you want to do, educate the people who are involved in the current manual process, tell them about what will be changing, how it will be working and how they can help, often people can see the benefits of Automation and are keen to help achieve the goal, you will need their help.

 

You will be talking to many different teams in complex processes as you will want to automate tasks from end to end, this includes gaining access to systems you may not have access to at the moment, breaking down the silo-ed mentality that some teams have.  Storage admins will not let you touch their baby, equally networking staff who have named and cared for each of their switches will not want anyone having access but them!

 

This can be a challenge which requires help from the sponsor and also enthusiasm through education of the end goal.

 

 

More Automation tips

More tips from this series created by myself and Thomas Corfmat for a session at VMworld 2013 can be found below (updated as published), if you have a login for VMworld.com you can also watch the full session here.

 

Automation Tip 1 – Measure It!

Automation Tip 2 – Find your standards

Automation Tip 3 – Its all about people

Automation Tip 4 – Select the right mix of people

Automation Tip 5 – Simplify, Don’t automate a broken process

Automation Tip 6 – Automate incrementally

Automation Tip 7 – Testing

Automation Tip 8 – Documentation

Automation Tip 9 – End to end automation

Automation Tip 10 – Follow the “Automation Triangle”