vCheck (Daily Report) V3

Awesome as usual.

Great job.

Juicy… Love the new script. Can’t wait for v4.

Change the name?? Daily Issue Checking?? LOL
Just every time you blog about vCheck be clear that it’s for issue checking??

hi Al , loving the script but it seems to be gettin upset calculating the cpu %ready stuff.

Note the script had been running for about 20 minutes at this point , is is posisble its hitting a timeout.

15:24 ..Checking VM CPU %RDY
Get-Stat : 04/11/2009 15:24:26 Get-Stat 1FE8AA92-9E03-450D-B247-A95E26B77
E89 The session is not authenticated.
At C:\temp\vcheckv3.ps1:827 char:54
+ $myObj.PercReady = [Math]::Round((($v | Get-Stat <<<< -S
tat Cpu.Ready.Summation -RealTime | Where {$_.Instance -eq $cpunum} | Measure-O
bject -Property Value -Average).Average)/200,1)

It does keep running. I seem to remeber having a few little bugs with the previous version – nb its running on a medium sized environment ( 21 hosts / 300 Vm’s ) – not braved it on the big one yet

might want to add an example report! Would be nice to see what it can do…

Duncan I’d forward one I did earlier but I’m embrassed as to the number of things its highlighted I need to get sorted out

oh mines got lots of issues

Outstanding report, Al! We’ve been using it in our environment and has allowed us to be more proactive in fixing items before they become huge problems. By the way, Scott Herold was just at our Portland, OR VMWare Technical Conference and he gave you some love in his presentation on TheVesi. Thanks again for the script!

Excellent Report once again …..I was thinking about maybe if you could add a section that states the size of your vmkwarnings and or vmkernel log which would kind of lead to you looking at that host as something must be wrong if that file is any type of size in terms of having data written to it correct?

none the less Thank you once again for this fabulous reporting tool

“vmkernel warning messages ov the last x days” I see this listed but not on many my reports although I have two servers that have vmkwarning logs that get about 1-2gb every few days with scsi errors and such can you explain as I am missing something not enough fuel (coffee) yet this afternoon lol

Alan

Wow yet again, just when i thought v2 was the king script, along comes v3 with all sorts of new goodies.

Thanks for putting up a donation page, i felt compelled to give something back for all your hard efforts

Cheers
Paolo aka DJLO

Works like a charm. Nice Job!

Alan – WOW Again !! You continue to amaze me .. Within a week from release of the consolidated POWERPACK of yours.

Thanks again !!

Virtually anything is possible with you guys arnd !!

Super Script! Helps us a lot to keep track of the Virtual Environment.

Only one thing, when run against a vCenter4/esx4i environment I get this error for every Cluster it finds:
Method invocation failed because [VMware.Vim.ClusterComputeResource] doesn’t contain a method named ‘RetrieveDasAdvancedRuntimeInfo’.
At D:\scripts\DailyReport\DailyReport.ps1:701 char:61
+ $SlotDetails = $Cluster.RetrieveDasAdvancedRuntimeInfo( <<<< )

Any suggestions?

Hi,
please can someone tell me how to run this script?
My Batfile looks like:
C:\WINDOWS\system32\windowspowershell\v1.0\powershell.exe -PSConsoleFile “C:\Programme\VMware\Infrastructure\vSphere PowerCLI\vim.psc1″&”C:\test\vCheckV3.psc1 vcenter”

I changed my mailserver and other parameters, but i don´t reviece a email yet.
Idea?
Big thanks

Great script, just saved me from a nasty situation that was prone to happen.

While i’m here, I would like to submit a feature request.
1) The posibility to only include some (one or more) clusters within the VI configuration and thus exclude all others. We have multiple clusters of which other departments are in control of. I don’t want to know there problems and they don’t want to know mine.

2) In the Snapshot inventory (date > snapshop creation date) can you add a column that specifies on what cluster the guest is running?

Thanks for your answer!If i post in dos box, looks loke this:

C:\WINDOWS>C:\WINDOWS\system32\windowspowershell\v1.0\powershell.exe -PSConsoleF
ile “C:\Program Files\VMware\Infrastructure\vSphere PowerCLI\vim.psc1? ” & “C:\S
cripts\VMware\DailyReport.ps1 MYVISERVER”

This comments are the problem: ″ ” & “C

I’m having problems, connecting to the VI Server, it gives me a certificate warning and I can’t connect.
Does anyone have an idea, on how to fix this?

PS C:\install\VSphere Addons> .\Status.ps1 itgvc001
11:15 Connecting to VI Server
There were one or more problems with the server certificate:

* A certification chain processed correctly, but terminated in a root certificat
e which isn’t trusted by the trust provider.

* The certificate’s CN name does not match the passed value.

Connect-VIServer : not connected
At C:\install\VSphere Addons\Status.ps1:483 char:29
+ $VIServer = Connect-VIServer <<<< $VISRV
Connect-VIServer : not connected
At C:\install\VSphere Addons\Status.ps1:494 char:30
+ $VIServer = Connect-VIServer <<<< $VISRV

@Thomas Paulsen
Updated Powershell to v2, now I have a more detailed error message:

PS C:\install\VSphere Addons> .\Status.ps1 itgvc001
11:33 Connecting to VI Server
There were one or more problems with the server certificate:

* A certification chain processed correctly, but terminated in a root certificat
e which isn’t trusted by the trust provider.

* The certificate’s CN name does not match the passed value.

Connect-VIServer : not connected
At C:\install\VSphere Addons\Status.ps1:483 char:29
+ $VIServer = Connect-VIServer <<<< $VISRV
+ CategoryInfo : NotSpecified: (:) [Connect-VIServer], ViServerCo
nnectionException
+ FullyQualifiedErrorId : VMware.VimAutomation.Types.ViServerConnectionExc
eption,VMware.VimAutomation.Commands.ConnectVIServer

Connect-VIServer : not connected
At C:\install\VSphere Addons\Status.ps1:494 char:30
+ $VIServer = Connect-VIServer <<<< $VISRV
+ CategoryInfo : NotSpecified: (:) [Connect-VIServer], ViServerCo
nnectionException
+ FullyQualifiedErrorId : VMware.VimAutomation.Types.ViServerConnectionExc
eption,VMware.VimAutomation.Commands.ConnectVIServer

Virtu-Al :
@Harold
Do your clusters have HA enabled ?

Thanks for the quick response.
This was my mistake,…. launched the script from VI toolkit 1.5.
RetrieveDasAdvancedRuntimeInfo is a vSphere PowerCLI 4.0 method.
I have upgraded and now it runs fine!

@Thomas Paulsen
Problem has been solved by installing VI Toolkit 1.5

But the CN Name certificate problem still exists, although it does not prevent the script from doing it’s job.

More detail on the “error” warning, now with VI Toolkit 1.5:

WARNING: There were one or more problems with the server certificate:

* The X509 chain could not be built up to the root certificate.

* The certificate’s CN name does not match the passed value.

Hi, no, not realy!

There are couple of Requests too

1)Is it possible to report the Average utilization of the vSphere Server say for a specific period of time so that we also will know how over / under utilized is the vSphere Server ?

2 ) Also , is there a way to run this for multiple Virtual Center Servers in the same schedule and generate a single report ?

Alan, I have a feature request

Can you add a cluster memory section:
- How much Memory is available in the cluster?
- Huw much Memory is granted to VMs in this cluster?

Why? I need to know when I take one host in maintenance, can the rest of the hosts in the cluster handle these vm’s, without memory overcommitment.
(We have a N+1 hosts policy, no memory overcommitment)

Thanks,
Harold

Alan, thanks for the great script. We are using it daily to check our 125 ESX 3.5 hosts. The only issue I have seen so far is that it takes a long time to process the VMs with CPU ready over x% part. Never seems to finish.

I too was also interested in a cluster memory section that would report memory usage in the event of host failures.

Is was working on adding this to the script.
# —- Cluster Capacity —-
$MyReport += Get-CustomHeader “1″ “Cluster Utilization”

$ClusterInfo = @()
$clusters = Get-Cluster | Sort Name
ForEach ($cluster in $clusters)
{
Write-Output $cluster
$ClusterCapacity = “” | Select-Object “ClusterName”, “TotalHosts”, “TotalMemGB”, “UsedMemGB”, “PercMemUsed”, “PercMemUsed1Fail”, “PercMemUsed2Fail”
$esxhosts = Get-VMHost -Location $cluster

ForEach ($VMhostView in ($esxhosts | Get-View))
{
$TotalMemorySize += $vmhostView.Hardware.MemorySize
$TotalMemoryUsed += $vmhostView.Summary.QuickStats.OverallMemoryUsage
}

$NumHostsInCluster = ($esxhosts | Measure-Object).Count
$TotalMemorySizeGB = [math]::Round($TotalMemorySize/1GB,$digits)
$TotalMemoryUsedGB = [math]::Round($TotalMemoryUsed/1024,$digits)
$TotalMemorySizeGB1Host = [math]::Round($TotalMemorySizeGB / $NumHostsInCluster)
$TotalMemorySizeGB1Failure = [math]::Round(($NumHostsInCluster-1)*$TotalMemorySizeGB1Host)
$TotalMemorySizeGB2Failure = [math]::Round(($NumHostsInCluster-2)*$TotalMemorySizeGB1Host)
$PercentMemoryUsed = [math]::Round(($TotalMemoryUsedGB / $TotalMemorySizeGB) * 100, 0)
$PercentMemoryUsed1Failure = [math]::Round(($TotalMemoryUsedGB / $TotalMemorySizeGB1Failure) * 100, 0)
$PercentMemoryUsed2Failure = [math]::Round(($TotalMemoryUsedGB / $TotalMemorySizeGB2Failure) * 100, 0)

if ($NumHostsInCluster -lt 8){$PercentMemoryUsed2Failure = “n/a”}

$ClusterCapacity.ClusterName = $cluster
$ClusterCapacity.TotalHosts = $NumHostsInCluster
$ClusterCapacity.TotalMemGB = $TotalMemorySizeGB
$ClusterCapacity.UsedMemGB = $TotalMemoryUsedGB
$ClusterCapacity.PercMemUsed = $PercentMemoryUsed
$ClusterCapacity.PercMemUsed1Fail = $PercentMemoryUsed1Failure
$ClusterCapacity.PercMemUsed2Fail = $PercentMemoryUsed2Failure
$ClusterInfo+=$ClusterCapacity

$TotalMemorySize = 0
$TotalMemoryUsed = 0
$NumHostsInCluster = 0
$TotalMemorySizeGB = 0
$TotalMemoryUsedGB = 0
$TotalMemorySizeGB1Host = 0
$TotalMemorySizeGB1Failure = 0
$TotalMemorySizeGB2Failure = 0
$PercentMemoryUsed = 0
$PercentMemoryUsed1Failure = 0
$PercentMemoryUsed2Failure = 0

}

$MyReport += Get-HTMLTable $ClusterInfo
$MyReport += Get-CustomHeaderClose

This code probably needs more work, but I am still learning.

I was looking to have a separate report that would be used for Capacity Planning with sections on the report that would have Cluster Utilization based on Memory, CPU, Storage and VM info.

Once againg thanks for your great work!

Hi Alan,

Great work on the vCheck script! I added some extra functionality that checks probems that I encountered on my company’s VMware environment:

# —- check for vms with unknown status —-
If ($ShowUnknownVMstatus){
Write-CustomOut “..Checking for VMs with unknown status”
$vmstatus = @($FullVM | Select Name, @{N=”OverallStatus”;E={$_.overallstatus}} | where {$_.OverallStatus -eq ‘gray’})
If (($vmstatus | Measure-Object).count -gt 0) {
$MyReport += Get-CustomHeader “1″ “VMs with unknown status : $($vmstatus.count)”
$MyReport += Get-HTMLTable $vmstatus
$MyReport += Get-CustomHeaderClose
}
}
# —- check for vms with runtime questions —-
If ($ShowVMQuestions){
Write-CustomOut “..Checking for VMs with runtime questions”
$vmquestion = @($FullVM | Select Name, @{N=”Text”;E={$_.Runtime.Question.Text}} | where {$_.Text -ne $null})
If (($vmquestion | Measure-Object).count -gt 0) {
$MyReport += Get-CustomHeader “1″ “VMs with runtime questions : $($vmquestion.count)”
$MyReport += Get-HTMLTable $vmquestion
$MyReport += Get-CustomHeaderClose
}
}
# —- check for vms with low memory reservation —-
If ($ShowLowMemLimit){
Write-CustomOut “..Checking for VMs with memory reservation lower than assigned RAM”
$vmlowmemlimit = @($VM | Select Name, MemoryMB, @{N=”MemlimitMB”;E={Get-VMResourceConfiguration -VM $_ | %{$_.MemLimitMB}}} | where {$_.MemlimitMB -ne -1 -AND $_.MemlimitMB -lt $_.MemoryMB})
If (($vmlowmemlimit | Measure-Object).count -gt 0) {
$MyReport += Get-CustomHeader “1″ “VMs with low memory reservation : $($vmlowmemlimit.count)”
$MyReport += Get-HTMLTable $vmlowmemlimit
$MyReport += Get-CustomHeaderClose
}
}

I still want to add a routine that does some resource checks. I’ll add them to this post later.

Kind regards,

Rudolf Kleijwegt

Hey Alan – thanks for this mate. The report runs well until it hits Checking VC Services:
————-
14:48 ..Checking Host Alarms
14:48 ..Checking VM Alarms
14:48 ..Checking Cluster Configuration Issues
14:48 ..Checking VC Services
Get-WmiObject : Access is denied. (Exception from HRESULT: 0×80070005 (E_ACCESS
DENIED))
At C:\Documents and Settings\dan.tracey\Desktop\VC Diagnostics\vCheck.ps1:280 c
har:28
+ $Services = get-wmiobject <<<< win32_service -ComputerName $VISRV |
Where {$_.DisplayName -like "VMware*" }
14:48 ..Checking VC Error Event Logs
Get-WmiObject : Access is denied. (Exception from HRESULT: 0×80070005 (E_ACCESS
DENIED))
At C:\Documents and Settings\dan.tracey\Desktop\VC Diagnostics\vCheck.ps1:937 c
har:31
+ $ErrLogs = @(Get-WmiObject <<<=’” + $Conv
Date + “‘”) | Where {$_.Message -like “*VMware*”} | Select @{N=”TimeGenerated”;
E={$_.ConvertToDateTime($_.TimeGenerated)}}, Message)
14:48 ..Checking VC Warning Event Logs
Get-WmiObject : Access is denied. (Exception from HRESULT: 0×80070005 (E_ACCESS
DENIED))
At C:\Documents and Settings\dan.tracey\Desktop\VC Diagnostics\vCheck.ps1:954 c
har:32
+ $WarnLogs = @(Get-WmiObject <<<=’” + $C
onvDate + “‘”) | Where {$_.Message -like “*VMware*”} | Select @{N=”TimeGenerate
d”;E={$_.ConvertToDateTime($_.TimeGenerated)}}, Message )
14:48 ..Checking vmkernel logs for warnings
14:49 ..Sending Email
Exception calling “Send” with “1″ argument(s): “Failure sending mail.”
At C:\Documents and Settings\dan.tracey\Desktop\VC Diagnostics\vCheck.ps1:151 c
har:14
+ $mailer.send( <<<

———
I’m a bit of a PowerCLI beginner so all help is very much appreciated!

Dan

@Virtu-Al
Hi Al – ever so sorry – I should have RTFM – all great now. Thanks again! Dan

Thank You Alan!

Hi Alan,
first of all I would like to say that you do a good job and your scripts help us to do our work better!

I find your new feature “Dead Lun Path” grate, but maybe there is a Bug.
The script told me that we have a problem with some vmhbas.

esx2f1.douglas-informatik.de vmhba1:0:1 Dead
esx2f1.douglas-informatik.de vmhba1:1:1 Dead
esx2f1.douglas-informatik.de vmhba1:2:1 Dead

That was right and we fixed the physical problem so far, but your script told us still the same problem. I can’t find any dead path at the service console.
Have you any idea?

Best Marcel

oh no…. The bug I reported for NTP checks if you have multi NTP servers didn’t make it into this version… (strange with all these daily checks etc, you only have 1 NTP configured in your environment??)

Any chance it can make it into V4, explain code on how to fix it was provided in the v2 section.

@bitsorbytes
Ignore that… just seen vCheck 3.1 posting…. doh!

Liking version 3, Al. One “issue”… I am not a power shell expoert, but line 485 appears to be incorrect ($VIServer = Connect-VIServer $VISRV). Since different credentials could be set (i.e. not use logged in user/domain), it apepars more arguments should be supplied here (I have crashed and burned updating it myself). Thoughts?

Charlie

@cxo
Again my PS savvy-ness is not great, but I came across a solution, almost from this URL: http://powershellcommunity.org/Forums/tabid/54/aff/5/aft/4295/afv/topic/afpg/1/Default.aspx
I could not use “$creds.Password”, but everything else worked when I put in “-Passwrd Clear-Text-PW”. So solution is not complete, but getting closer. Thoughts?