Category Archives: PowerShell

Using Hugging Face Code Autocomplete in VSCode to help with PowerShell and PowerCLI Scripts

Whilst at VMware Explore this year I was introduced to Hugging Face Code Autocomplete plugin for Visual Studio Code as part of the SafeCoder announcement.

In the keynote demo Chris Wolf showed how SafeCoder could be used to safely complete some python code using a machine learning model which had been trained on the internal VMware code base, it gave great results and showed how using code completion can be safely used by providing an on-premises solution, SafeCoder made sure your company’s core IP or code was not sent to a public service and shared with the world whilst giving you the benefit of code assisted development.

I was intrigued by the demo and wondered how easy it would be and what kind of job the Hugging Face Code Autocomplete plugin for Visual Studio Code would do with PowerShell and specifically PowerCLI Code. Imagine having a LucD sat over your shoulder as you type, suggesting what your code should look like… well its not quite as amazing as Luc, I mean we all strive for that!

I was however impressed, below you can see a quick gif of me using it to create some comments about what I want to do it and it filling out the code as I do so, this is an awesome head start to creating your own scripts and learning.

Note: The current implementation uses the Hugging Face API and model which was trained on the internet so the data wont be 100% accurate, it will do a great job of giving you the fundamentals but rounding the edges and ensuring the script does what you need it to do will be your responsibility!

Hugging Face Code Autocomplete plugin for Visual Studio Code

Installing Hugging Face Code Autocomplete in VSCode

  1. Install VS Code if you don’t have it already.
  2. Install the VS Code plugin from here.
  3. Sign up for a Hugging Face account (free), you will need this for the plugin to access their API.
  4. Setup an API token which you can use for VS Code to access the Hugging Face API.
  5. In VS Code press Cmd/Ctrl+Shift+P to open VSCode command palette.
  6. Type: Hugging Face Code: Set API token.
  7. Paste your API token into the plugin.
  8. You are done! I normally type #PowerShell Code as my first comment to help it know that’s what i expect and not some other language.

Monitoring processes inside a vm with PowerCLI

As part of a large scale data analysis project I was working on recently I used Horizon View and Instant clones to allow me to deploy hundreds of VMs based on my original templated VM which had my app I wanted to use to take in the data, transform it and then return a result. Its important to note that this app was never written to work in a batch mode, it literally took one input and gave an output based on a number of factors.

Whilst this post is not about the benefits of Instant Clones, let me tell you, once I had the original template VM working correctly and optimized for performance it worked like a charm, there is something to be said for the simplicity and efficiency of instant clones and the memory sharing techniques it uses to be able to run hundreds of essentially the same VM.

Once I had these VMs deployed via Horizon I could easily send a job to each VM and tell it to run the job, as these jobs however took an indeterminate amount of time to crunch the data, I needed a way to monitor them and let me know when all the jobs had finished on all the VMs so I could pull the data to analyze it.

First I went down the route of using Invoke-VMScript to hook inside the VM’s and see if the process was running, this however took a long time to complete on 100’s of virtual machines and the monitoring job often took over 20 minutes to monitor the VMs and tell me if the job was completed… far too long for what I needed. So after some googling I learned from my good friend William Lam that there was a new API available that through VMtools would update the GuestInfo with the processes running inside the VM on a configurable timely basis (see his post here), this allowed me to essentially push the information externally to the VM Guest Operating system and grab the info when I needed it to see if my process was running.

William had also written a handy function which I adjusted to work with an array of VMs and tell me if the process was running.

Using these new found skills I was easily able to write a function that allowed me to pull the running processes from the VMs and remove them from the list as that process finished inside the VM, for the icing on the cake I even got it to update me in slack on how it was doing 😉

So i thought I would share this script and encourage you to think about using this way which is much easier and a more performant way to pull the results of the running processes..

Monitoring Script

Function Send-SlackMessage ($Channel = "#MyProjectChannel", $Message) {
        
    $payload = @{
        "channel"    = $Channel
        "icon_emoji" = ":datacenter:"
        "text"       = $Message
        "username"   = "DC Script"
    }

    Invoke-WebRequest `
        -UseBasicParsing `
        -Body (ConvertTo-Json -Compress -InputObject $payload) `
        -Method Post `
        -Uri "https://myslackhookurl" | Out-Null
}

Function Get-VMApplicationInfo {
<#
    .DESCRIPTION Retrieves discovered applications running inside of a VM
    .NOTES  Author:  William Lam
    .NOTES  Site:    www.virtuallyghetto.com
    .NOTES  Reference: http://www.virtuallyghetto.com/2019/12/application-discovery-in-vsphere-with-vmware-tools-11.html
    .PARAMETER VM
        VM Object
    .PARAMETER Output
        CSV or JSON output file
    .EXAMPLE
        Get-VMApplicationInfo -VM (Get-VM "DC-01")
    .EXAMPLE
        Get-VMApplicationInfo -VM (Get-VM "DC-01") -UniqueOnly
    .EXAMPLE
        Get-VMApplicationInfo -VM (Get-VM "DC-01") -Output CSV
    .EXAMPLE
        Get-VMApplicationInfo -VM (Get-VM "DC-01") -Output JSON
#>
    param(
        [Parameter(Mandatory=$true)]$VM,
        [Parameter(Mandatory=$false)][ValidateSet("CSV","JSON")][String]$Output,
        [Parameter(Mandatory=$false)][Switch]$UniqueOnly
    )

    $appInfoValue = (Get-AdvancedSetting -Entity $VM -Name "guestinfo.appInfo").Value

    if($appInfoValue -eq $null) {
        Write-Host "Application Discovery has not been enabled for this VM"
    } else {
        $appInfo = $appInfoValue | ConvertFrom-Json
        $appUpdateVersion = $appInfo.updateCounter

        if($UniqueOnly) {
            $results = $appInfo.applications | Sort-Object -Property a -Unique| Select-Object @{Name="Application";e={$_.a}},@{Name="Version";e={$_.v}}
        } else {
            $results = $appInfo.applications | Sort-Object -Property a | Select-Object @{Name="Application";e={$_.a}},@{Name="Version";e={$_.v}}
        }

        Write-verbose "Application Discovery Time: $($appInfo.publishTime)"
        if($Output -eq "CSV") {
            $fileOutputName = "$($VM.name)-version-$($appUpdateVersion)-apps.csv"

            Write-Host "`tSaving output to $fileOutputName"
            ($appInfo.applications) | ConvertTo-Csv | Out-File -FilePath "$fileOutputName"
        } elseif ($Output -eq "JSON") {
            $fileOutputName = "$($VM.name)-version-$($appUpdateVersion)-apps.json"

            Write-Host "`tSaving output to $fileOutputName"
            ($appInfo.applications) | ConvertTo-Json | Out-File -FilePath "$fileOutputName"
        } else {
            $results
        }
    }
}


Set-PowerCLIConfiguration -InvalidCertificateAction Ignore -ErrorAction SilentlyContinue -Confirm:$false -DisplayDeprecationWarnings $false | out-null
$ProgressPreference = 'SilentlyContinue'


Connect-viserver vcsa-01a.myenv.local -username Administrator@vsphere.local -password VMware1! | Out-Null
$vms = get-vm InstantCloneVM* | Where-Object {$_.PowerState -eq "PoweredOn"} | Sort-Object Name
Send-Slackmessage -message "Monitor Job Started" 
do {
    foreach ($vm in $vms) {
        Write-Host "Checking $VM status..."
        $result = Get-VMApplicationInfo -VM $VM| Where { $_.Application -eq "MyAppProcess.exe" }
        if (!$result){
            Write-Host "Removing $vm as it has completed"
            $vms = $vms | where { $_.Name -ne $vm.name}
        } else {
            Write-Host "$vm still running"
        }
        if (($vms.count -lt 10) -and (! $sentmail)){
            Send-Slackmessage -message "Less than 10 Sigma VMs left"
            $sentmail = $true
        }
    }
    if (! ($oldnumvms -eq ($vms.count))){
        Send-Slackmessage -message "$($vms.count) VMs still running"
    }
    start-sleep -s 30
    $oldnumvms = $vms.count
}
while ($vms)
Send-Slackmessage -message  "All Jobs Completed"

New Year, New Look vCheck

Its been a while since I blogged about vCheck but that doesn’t mean there has been lots of work ongoing with the project, in fact there has been multiple releases and hundreds of pull requests with great new plugins checking for even more issues with your vCenter and lots of bug fixes, in fact this project and the community updating and using it is awesome, here are some the current stats:

  • 954 Commits
  • 64 Contributors (Thanks to all here)
  • 66 Open Issues and 269 Closed Issues

Whats more, one of the most recent releases 6.25 here includes a great new look, the new look is based upon VMwares Clarity framework and personally I think it looks fantastic as you can see below:

vCheck Clarity

Also available (currently in the dev branch) is the brand new dark version of the theme as seen below:

vCheck Clarity Dark

Download for free now!

Download the latest version here:  https://github.com/alanrenouf/vCheck-vSphere/archive/master.zip

And the dev branch version here: https://github.com/alanrenouf/vCheck-vSphere/archive/dev.zip

 

Checking you are up to date with PowerCLI

Now that PowerCLI is a module and in the PowerShell Gallery there have been a lot of releases and bug fixes, you would be forgiven for not having the latest version installed or even knowing what the latest version is.

With this in mind and with the latest 6.5.3 version triggering this in my mind, I created a function that checks your installed PowerCLI version against the one thats in the PowerShell Gallery online and lets you know if there is a new version.

Now of course this can be run manually and it will return the results letting you know which modules are out of date:

And once completed of course its easy to update PowerCLI to the latest version:

Even better why not add it to your profile. It does take a couple of seconds to run so maybe you will want to run it on a certain day past a certain time in your profile so it doesn’t slow down every launch of PowerShell you have, here is an example of what I have in my profile where I check every Wednesday after 2PM.

if ( ((Get-Date).tostring('%H') -ge "14" ) -and ( (Get-Date).DayofWeek -eq "Wednesday" ) ) {
        Check-PowerCLIUpdate
}

Check-PowerCLIUpdate Script

Here is the function that allows you to check for updates:

Function Check-PowerCLIUpdate {
    #Based on great module by Jeff Hicks here: http://jdhitsolutions.com/blog/powershell/5441/check-for-module-updates/
    [cmdletbinding()]
    Param()

    # Getting installed modules
    $modules = Get-Module -ListAvailable VMware* | Sort Version -Descending | Select-object -Unique

    #Filter to modules from the PSGallery
    $gallery = $modules.where({$_.repositorysourcelocation})

    # Comparing to online versions
    $AllUpdatedModules = @()
    foreach ($module in $gallery) {

         #find the current version in the gallery
         Try {
            $online = Find-Module -Name $module.name -Repository PSGallery -ErrorAction Stop
         }
         Catch {
            Write-Warning "Module $($module.name) was not found in the PSGallery and therefore not checked for an update"
         }

         #compare versions
         if ($online.version -gt $module.version) {
            $AllUpdatedModules += new-object PSObject -Property @{
                Name = $module.name
                InstalledVersion = $module.version
                OnlineVersion = $online.version
                Update = $True
                Path = $module.modulebase
             } 
         }
    }
    $AllUpdatedModules | Format-Table
    #Check completed

}

Is your PowerShell Core up to date?

I have been using PowerShell Core for a while now, its great not to have to spin up a windows VM every time I want to work with PowerShell on Mac!

Let’s face it though, not everything is there at the moment, so every time there is an update I want to know.  The Microsoft team are doing a great job of updating PowerShell Core and it seems like every time I go check there is a new version which fixes bug or introduces new cmdlets.

This does however mean I need to constantly go check the releases page or keep an eye on twitter to see when new releases come out.. until now!

The following function is a 5 minute function I wrote to keep on top of the updates, I have placed it in my PowerShell profile and now every time I launch PowerShell from my mac it checks and tells me if there is a new version as below:

Code

Just use the following code and paste it into the file located at $profile and every time you launch PowerShell it will go check for you!

Function Get-PowerShellRelease {
    #Using this to get rid of the nasty output Invoke-WebRequest gives you in PowerShell on the Mac
    $progress = $ProgressPreference
    $ProgressPreference = "SilentlyContinue"
    $JSON = Invoke-WebRequest "https://api.github.com/repos/powershell/powershell/releases/latest"| ConvertFrom-Json
    If ($psversiontable.GitCommitId) {
        If ($JSON.tag_name -ne $psversiontable.GitCommitId) {
            Write-Output "New version of PowerShell available!"
            $JSON.body
        } Else {
            "PowerShell is currently up to date!"
        }
    }
    $ProgressPreference = $progress
}

Get-PowerShellRelease

Check your Cisco UCS environment for issues with vCheck

Do you use Cisco UCS? If you do then this will interest you.  vCheck is basically a FREE HTML reporting framework that was written to originally check a VMware environment but as it uses PowerShell it is easily adjustable to check anything that is PowerShell enabled, like for instance…. UCS environments.

One of the objectives of me putting vCheck into github was the hope that people would be able to easily pick up the framework and create their own projects to check other PowerShell enabled products, this has already been done with a number of different products as listed below:

Download Link Github Project Example Output Page
vCheck for vSphere Click here Click here
vCheck for Exchange 2010 Click here Click here
vCheck for vCD Click here Coming soon
vCheck for SCVMM Click here Click here
vCD Audit script Click here Click here
vCloud Air Audit Click here Click here

And now you can add Cisco UCS to the list, Joshua Barton has done a great job of using the framework to create a new vCheck edition that will enable UCS Admins to run a check on their systems and bring back any known issues that may be a problem.  Don’t forget, this as with all the other vCheck editions is a community project so you can add your own checks very easily in the plugins folder, if you need help then check out this early video which was recorded of me showing how to easily do that.

So what does it check currently?

  • General Details
  • Recent Faults
  • Unassociated Profiles
  • High Pool Utilization
  • Fault Retention Policy
  • Default Adapter Behavior
  • Non-Functioning Enabled Ports
  • Switching Mode
  • Inactive Servers
  • Uplink Flow Control
  • LACP Policies
  • UDLP Policies
  • Maintenance Policies
  • Default Pool Schema
  • Chassis Discovery Policy
  • Power Policy
  • SEL Policy

What does it look like?

As you can see, the html file that is created, which can also be scheduled to be sent to you as an email looks pretty awesome:

UCS-vCheck

 

The full example can be seen here: http://www.foobartn.com/assets/examples/example-report.html

How do I download this and get involved?

Visit the Github Rep here: https://github.com/FooBartn/vCheck-UCS/tree/dev

Download it directly from here: https://github.com/FooBartn/vCheck-UCS/archive/dev.zip

And make sure you thanks Joshua on twitter via @FooBartn

Adding a vGPU For a vSphere 6.0 VM via PowerCLI

I had a conversation with a VMware customer the other day and they were asking if there was a way to automate the addition and removal of a vGPU to a VM dynamically on a requested or scheduled basis, VMGuru has a great post here on what exactly a vGPU is, I highly recommend reading it.

They asked for a PowerCLI command or function to configure the VM graphics card, specifically to assign a vGPU resources to the VM, a desktop user’s VM is configured by default with vSGA (to not tie up  vGPU resources) then the user schedules the vGPU resource for a specific time frame using their end user portal, before the scheduled vGPU reservation time frame the user is logged out and the script would kick in. The script would reconfigure the VM settings and assigns a vGPU profile to it, the equivalent of what is being done in the UI as follows:

image

Continue reading

PowerCLI 6.0 R2 – The most advanced version VMware has ever made

imageThis week VMware released a new version of PowerCLI and this release is the most advanced version ever made.  As the Product Manager for PowerCLI I am particularly proud of this release as we not only add core functionality to the vSphere cmdlets allowing access to report, manage and automate even more of the core infrastructure but yet again we introduce a number of new features allowing PowerCLI to reach even further into the SDDC and automate and integrate more products.

I always like to ask people what they want to see from PowerCLI next and for a long time now people have been talking to me about vROPs and telling me their use cases of being able to use the rich dataset and analytics provided by vROPs to make decisions and take further action with PowerCLI, something as simple as monitoring a web server to know when it is under duress and provision many more based on these statics all the way to being able to take action on storage systems by monitoring the workload over time.  The use cases are endless and now achievable by the latest version of PowerCLI.  More on this to come in a future post but lets just say I am excited to see what the awesome PowerCLI community does here.

There is so much to talk about in this release, I wanted to give you an overview in this post and then dive deeper into some of the key features in the future.

What’s New?

More Module Enhancements

The community made it clear that we had to move to modules, and whilst we are still getting there, in this release we made even more module enhancements, both to the core distribution model of PowerCLI cmdlets and also converting more of our code base from snapins to modules. In this version the License snap-in has been converted to a PowerShell module. In the new release the PowerCLI Modules have also been moved to the System PSModulePath allowing all users of a machine to access them once installed.

vROPS Support

As briefly mentioned above, a new module and cmdlets have been added to this release to allow access to vRealize Operations, access to the entire public API is available from the $global:DefaultOMServers variable using the ExtensionData property and full cmdlets have been included for the most used features, this gives us a good mix of fully functional easy to use cmdlets and yet the access for advanced users to be able to access the entire API through the ExtensionData property to create their own functions to expose anything to automate vROPs.  The following cmdlets were added for using with vROPs:

  • Connection: Connect-OMServer , Disconnect-OMServer
  • Alerts: Get-OMAlert, Get-OMAlertDefinition, Get-OMAlertSubType, Get-OMAlertType, Set-OMAlert
  • Recommendations: Get-OMRecommendation
  • Resources: Get-OMResource
  • Statistics: Get-OMStat, Get-OMStatKey
  • User Management: Get-OMUser

Update Manager

PowerCLI for Update Manager has always been available as a separate downloadable installer but it was hard to work with, you needed to work out which version of Update Manager you had installed on the server and install the exact version of PowerCLI cmdlets to work with it, this was obviously a pain when using multiple versions of vCenter or when trying to manage them from one machine.  In this version PowerCLI for vSphere Update Manager is no longer a separate downloadable component or installer and is now included in the core PowerCLI installer, it is selected by default during the PowerCLI install wizard which allows for simpler and quicker deployment and management of VMware products through PowerCLI. Enhancements have also been made to ensure a better backwards compatibility experience of this module and now supports versions of vSphere Update Manager all they back to 5.5.

vCloud Air

In the previous release we introduced PowerCLI for vCloud Air, allowing you to connect to your dedicated resources you have purchased and transfer your automation skillset into the cloud.  In this release you can now also connect to and manage vCloud Air On-Demand (vCA) instances with the –VCA parameter added to the Connect-PIServer and the new Get-PIComputeInstance cmdlet to list all available compute instances. Existing cmdlets for managing vCloud Director and vCloud Air can be used to work with vCloud Air On-Demand where applicable. Additional to these enhancements a new cmdlet has been added to make it easier and enhance the ability to work with OrgVDC Networks called Get-OrgVdcNetwork.

ESXi Host Hardware

This feature was asked for in the PowerCLI Communities, this is just one of the places we monitor to work out what our future releases look like.  New cmdlets have been added to work with ESXi host hardware, these cmdlets give the ability to interrogate your ESXi hosts and provide core system and hardware information.

Two new cmdlets have been added to work with this area:

  • Get-VMHostHardware lists host information
  • Get-VMHostPCIDevice lists all PCI Devices

Storage

Over the last number of releases we keep making enhancements to the storage cmdlets, with VSAN, IO Filters and now a number of enhancements have been made to the storage module introducing new cmdlets for working with VMware vSphere® API for Storage Awareness (VASA), NFS 4.1 and updated vSphere API for IO Filtering (VAIO) cmdlets.

The following new additional cmdlets are now available:

  • New VASA Cmdlets : New-VasaProvider, Remove-VasaProvider, Get-VasaProvider and Get-VasaStorageArray
  • New NFS Cmdlets: New-NfsUser, Remove-NfsUser, Get-NfsUser and Set-NfsUser
  • Added VAIO filters Cmdlet: Set-VAIOFilter

A pet peeve of mine and others

In previous versions of PowerCLI when you started the interactive shell it would always start PowerCLI in the directory where we installed the components, as I watched people use PowerCLI I would see them run a CD\ to get back to the root of the C: or they would use it from the install directory and often their commands would line wrap making it slightly harder to read, a small but hugely beneficial change was made in this release, the PowerCLI starting directory has now been changed from the full install path of PowerCLI to the root of the installation drive.

Business as usual

As usual updates have been made for PowerCLI to support the latest versions of the software it works with, in this case Site Recovery Manager (SRM) was updated to 6.1 and additional functionality has been added to discover SRM servers when they are deployed in a “shared recovery model” and support has been added for the vCloud Director 8.0 features which are provided by the backwards compatibility agreement of the VCD API.  All this and many more bug fixes and speed enhancements.

What an awesome release!

Same company different focus, let me know about your API wants!

For over a year I have been focused on improving VMware Command Line Interfaces including PowerCLI, vCLI, vMA, ESXCLI & DCLI.  I have tried to stay close to what the users of VMware want from a command line interface and have been delivering this as part of our current offerings and future offerings, I have been working with some of the best development teams in the industry (I may be biased) and have enjoyed talking to customers who use my products to do a variety of awesome things.

Well, none of that changes yet! I mean I am still responsible for working with the command line tools so please keep the feedback coming.

 

imageI have however recently taken over a new role and focus at VMware, one which I am highly excited to work on and make a real impact.  Around a month ago I took over the responsibility for the vSphere API and the vCloud Suite API, SDKs and CLIs.  There is a lot going on here and obviously that does not mean I am responsible for every feature which has an API!  In this new role I will be helping drive VMware APIs forward, working with customers and partners to identify gaps in our API/SDK/CLIs, define new APIs/SDKs/CLIs and work with multiple products and services within VMware to drive a better experience for people who use VMware products for automation or customers/partners who develop against our APIs to provide integration or DevOps engineers who want to just automate the entire experience and provide a constant lifecycle for their business processes or products.

Talk to me

Hopefully those who I have spoken to before, be it on twitter or via phone call or email or even over a beer know that I am an approachable guy, I may not always be able to talk about timeframes or where we are headed with our products but I promise you I will certainly take your feedback and prioritize it to produce the most valuable product to you.  Please do feel free to let me know what you think about the VMware APIs/SDKs/CLIs, the good, the bad and the ugly!

I would appreciate your quick thoughts on our APIs and what you would like to see going forward from us, feel free to use the comments on this post to pass them along or contact me via the other means mentioned.

Creating re-usable PowerCLI functions with “Onyx for the Web Client”

Onyx is a tool I have been using for years, it’s a tool that sat between the vSphere C# Client or Windows Client and the vCenter server and intercepted the SOAP calls, it would inspect these SOAP calls and convert them into PowerCLI code.  With the recent move away from the C# Client there is an obvious reason why the original Onyx Fling would not work anymore, its not like you can put it in between the web client and the vCenter service is it ?!

This tool was always a life saver for me as I’m not one to read API documentation, it was always a way to not only work out the code for me to alter and reuse but it was a great way for me to work out how VMware would do something, there is often multiple ways you can code something and if you can see what the C# client was doing to perform an action then that had to be a great place to start.

So up until now it has always been for the C# client, well today all that changes, just released is “Onyx for the Web Client” and it is awesome!

Firstly I want to thank the engineer who worked on it, Atanas Atanasov is a member of the PowerCLI team and was fantastic throughout the project taking feedback from myself and the internal testers at VMware (You know who you are and thanks!) and turning it around very fast to produce the end result which works fantastically.

So how does it work?

There are already some great blog posts out there on the official PowerCLI Blog, on Luc’s Site, Jonathans Site and on Roberts site which go into some detail so I thought I would do a quick video to show you how I like to use it.

In the video below I show how I do not know how to automate an advanced setting of a VM, I show how easy it is to record the action of changing that setting, take the code that the fling produces and then use it to firstly change a single VM to a different setting and then how easy it is to turn this into a function that can be used on all VMs in the environment.  This shows how we can create repeatable code that can be used (and shared) to achieve the task we need all with little knowledge of the API.

Download it now!

Make sure you download and install the fling in your test environment from here.

Watch how to use the fling