VMware Log Insight – Duplicate Hosts Listed

After returning from a VMware Design course and having learned that Log Insight was now free to valid vCenter licence holders, I set straight to work deploying it into our development environment. However, despite having under 25 hosts Log Insight soon started complaining about exceeding the allowed 25 OSI limit. ESXi hosts were being displayed in the statistics with both their defined hostname, but also what appeared to be their FQDN (e.g. vmhosta-01 and vmhosta-01.domain.com).

Checking on the hosts themselves (esxcli system hostname get) there was no domain or FQDN set, but they do have FQDNs in DNS, and vCenter knows them by their FQDN.

To fix the issue, I made things consistent by setting the domain and FQDN on all hosts using PowerCLI:

You can verify whether the change has helped quite easily, by visiting: https://your-loginsight/admin/hosts – the ‘Last Received Event’ for hosts without FQDNs simply growing greater and greater. After 12 hours, the ‘Average Active OSIs’ (as reported by https://your-loginsight/admin/license) had dropped slightly, but it’s got a way to go before being under 25.

PowerCLI 6.0 and PSModulePath

This article is to look at the problems with PSModulePath when installing PowerCLI 6.0 on Windows 10 (I’m aware that it is not yet certified for Win 10). The problem does not occur with the NetApp Data ONTAP PowerShell Toolkit (as noted at the bottom).

On a clean installation of Windows 10 we can observe the following behaviour with regard to PSModulePath:

The first command translates to $env:PSModulePath. When PowerShell loads, it dynamically sets this session only variable equal to:
$home\Documents\WindowsPowerShell\Modules ($home is something like C:\Users\username)
$pshome\Modules ($pshome is the location of PowerShell, typically C:\Windows\System32\WindowsPowerShell\v1.0)
Based on the output above it also appends the contents of the system wide, SYSTEM environment variable for PSModulePath.

The second command retrieves USER environment variable called PSModulePath (available to all users of the system), and as you can see there is none set. If there were one, it would prefixed to $env:PSModulePath (i.e. it would be the first path visible in the output of the first command).

The last command here retrieves the SYSTEM environment variable called PSModulePath. It is also present in the GUI here:
Capture

$env:PSModulePath is a rather important variable that is used not only for querying for available modules, but also module autoloading and tab completion of commands.

Sadly after manually installing PowerCLI (version 6.0.0-3056836) by running the exe as a local administrator as many would do, the behaviour of PowerShell on load is changed (some might call it broken!). Running the same query above we see:

The SYSTEM environment variable for PSModulePath is modified, nothing is created for the USER environment variable for PSModulePath, and critically PowerShell no longer performs any customisation of $env:PSModulePath! The paths:
C:\Users\DefaultAdmin\Documents\WindowsPowerShell\Modules
C:\Program Files\WindowsPowerShell\Modules

are now gone.

This post on the VMware Community Forums 7 months ago seems to suggest the behaviour of PowerCLI on Windows 8.1 is different (or perhaps an older version of PowerCLI?):

The PowerCLI 6.0 installer does create a USER PSModulePath variable if one does not already exist.

… we can see at least on Windows 10, under the current version, it does not.

There is advice on the VMware PowerCLI blog under the v6 release which suggests:

The problem with this is that when calling SetEnvironmentVariable there is no target specified so it only updates the local session variable (i.e. $env:PSModulePath). This would therefore be required in every script (not ideal). Also we’ve established that installing PowerCLI has broken the population of this variable somehow. The best workaround is already in the comments; running the above code but targeting the SYSTEM variable for PSModulePath (i.e. [Environment]::SetEnvironmentVariable(“PSModulePath”,$p,’Machine’)).

Now, if I repeat this process after installing the Netapp Data ONTAP PowerShell Toolkit (3.2.1.68) (accepting a UAC prompt), I get the perfect behaviour:

I think the appropriate clever people at VMware and NetApp need to have a chat with each other :)

Note: If I right click the PowerCLI installer and “Run as Administrator” I get the same behaviour.
Note: The Cisco UCS PowerTool also breaks the PSModule behaviour (using version CiscoUcs-PowerTool-1.5.1.0).

Office 365, PowerShell, SKUs and Service Plans

Working with Office 365 licensing and PowerShell can get a little confusing with SKU names like “STANDARDWOFFPACK_IW_FACULTY” and plan names like “MCOSTANDARD” (which is Skype, of all things!). In order to help reporting and management, I wrote a couple of advanced functions to translate SkuPartNumber and ServicePlanName into friendly names as shown in the Office 365 / O365 Portal. They accept pipeline input. Note the “LU” prefix in the function name; this is our chosen prefix to avoid naming conflicts/overlap with any official cmdlets. I work in Education, so these names are specific to educational plans, but you may adapt them to suit your needs. I’m also happy to add more to the functions if this serves as a useful reference.

The Curious Case of PowerShell Module Autoloading with Multiple Nested Modules

As part of a new project at work I wanted to move towards converting our PowerShell function libraries into PowerShell modules. After some discussion we decided that rather than having multiple functions within a singular .PS1 file (and dot sourcing to pull it in), we wanted one function per file and to pull those in using a module manifest. The obvious and more common alternative to this of course is having all your functions in a singular .PSM1 file. I’m aware that in the wider world feelings are split on this matter but for our environment the multiple file approach suits us better. According to the official Microsoft documentation, it is possible to create a Module Manifest with multiple “NestedModules” (i.e. the NestedModules parameter accepts an array). Apparently these can be .PSM1, .PS1 or .DLL files.

Using this approach however resulted in unusual behaviour; functions defined in the second and subsequent NestedModules do not autoload. The following example assumes that you have correctly modified $env:PSModulePath to include the location of your custom modules.

TestModule.psd1

Mod1.psm1

Mod2.psm1

Both Get-Module -ListAvailable and Test-ModuleManifest -Path C:\CustomModules\TestModule\TestModule.psd1 show:

As you can see, ExportedCommands only shows a function from Mod1.psm1 and the function available in Mod2.psm1 (Set-Something) is missing. As such tab completion / autocomplete / autoload only works for Get-Something. If I were to add an additional function to Mod1.psm1, we’d see an output like this:

… but again, nothing from Mod2.psm1.

What if we run Get-Command -Module TestModule?

Commands/functions from both modules appear and now Set-Something tab completes / autocompletes / autoloads. This persists across PowerShell sessions; if I close PowerShell, reopen it, and modify $env:PSModulePath I can still tab complete my Set-Something function from the second NestedModule. This is PowerShell command caching in action, and I have to give credit to this post by the extremely knowledgeable Dr Tobias Weltner (which references “Module Command Discovery” and a “Secret Command Cache”) for helping me discover this. Unfortunately there’s very little information online about PowerShell caching so I set about hunting for this magical secret cache and came across a folder called “CommandAnalysis” located at “C:\Users\Username\AppData\Local\Microsoft\Windows\PowerShell\CommandAnalysis”.

Closing PowerShell, renaming this folder, and reopening PowerShell results in this folder being recreated. On my system a total of 79 files are created (PowerShell_AnalysisCacheIndex and many files prefixed with PowerShell_AnalysisCacheEntry_). After I update $env:PSModulePath and run Get-Module -ListAvailable an additional 18 files are created, and as expected commands from Mod1.psm1 of my custom module tab complete.

As well as Get-Command -Module TestModule seeming to force a parse of the Mod2.psm1 and make the functions available and persist across sessions, the same happens if you explicitly load the module (but this defeats the purpose of the autoload feature?). It’s worth noting that testing shows this cache to have a TTL value; I explicitly imported the module, was able to execute Set-Something, and was able to do this between PowerShell sessions but upon reopening PowerShell the following morning I was not.

I also noticed that if I were to call a function from the first nested module (Mod1.psm1), this would also result in functions from the second nested module being available, but this time it was only for that session.

I have tried using .PS1 files instead of .PSM1 and the behaviour is the same. I’ve also tried using the following:
TestModule.psd1

TestModule.psm1

.. but this is worse, with no commands being exported.

Including Export-ModuleMember in each PSM1 file doesn’t help.

It is possible to include scripts in the module manifest on the ScriptsToProcess line and this seems to solve the problem, but I suspect the entire code is read into memory rather than the command names being cached. This also doesn’t addresses the original problem.

Just for full disclosure, here is a dump of my $PSVersionTable variable:

There is a discussion I’m involved in on Stack Overflow around this issue. As I was not the originator of the post, I decided to write this blog article to clarify what I’d observed and tried (as well as being free to insert code which you cannot easily do in the comments section).

PowerShell and Notify My Android (NMA)

Here is a PowerShell function which wraps around the Notify My Android notification API call. You can use it in your environment to Notify Android devices when you like.

Please note that line 57 below, that reads as

should actually be [ x m l ] $webpage (without spaces) so you will need to manually correct this (the limitations of the syntax highlighter plugin prevents this from displaying correctly).

Generating Remote Desktop Connection Manager (RDCMan) files with PowerShell

When I stumbled across this article on IT Pro in June this year a knowing smile crept across my face. The author had discovered an extremely handy tool for managing multiple remote desktop connections and seemingly puzzled over why it wasn’t more well known; my thoughts exactly. I found this utility about 3 years ago when searching for an easier way to manage remote desktop connections to multiple Windows servers. It made absolutely no sense to have to open multiple unique Remote Desktop / mstsc.exe windows to every server you wanted to work on. I was very relieved to find RDCMan produced by Microsoft.

The downside to this simple tool though is that you have to add servers manually, one by one. With some 200 servers (at the time) this was a potentially painstaking task, and one that I didn’t want to undertake if there was an easier alternative. The .rdg file produced by RDCMan is actually just an XML file (discovered by dropping the file into Notepad) so I figured that it’d be quite straightforward to automate the production of this using PowerShell, grabbing a list of servers from AD (or .csv, .txt or wherever). Thankfully Jan Egil Ring had done some of the work for me, but I modified it slightly and turned it into a more suitable script for our environment. Pass the function a username, an array of computer objects and an output path and you get an RDCMan file with the computers of your choice for the user of your choice. Rather than call the script multiple times from Task Scheduler (i.e defining the parameters at the script level) I just schedule it once (on Monday morning) and within the script call the function multiple times for all the users who need a file.

Here’s the script:

<#
	.SYNOPSIS 
	This generates a Remote Desktop Manager file for computer objects within Active Directory.
	.DESCRIPTION
	This generates a Remote Desktop Manager file for computer objects within Active Directory.
	Objects are generated using Microsoft's Active Directory module.
	Pass values required to the function rather than the script.
	It is based off "New-RDCManFile.ps1" by: Jan Egil Ring.
	.PARAMETER debugScript
	Switch on Write-Debug output. Default is No.
	.EXAMPLE
	C:\PS> New-RDCManFile.ps1
	.NOTES
	Author: Jan Egil Ring | Robin Malik
#>


# Leave previous two lines blank
#*=============================================================================
#* PARAMETER DECLARATION
#*=============================================================================

#*=============================================================================
#* REVISION HISTORY
#*=============================================================================
#* Date: 	YYYY-MM-DD
#* Author:	Your Name
#* Purpose:	Why and how you modified the script in brief. Do not delete
#*			previous revision history blocks.
#*
#* Date: 	YYYY-MM-DD
#* Author:	Your Name
#* Purpose:	Why and how you modified the script in brief. Do not delete
#*			previous revision history blocks.
#*=============================================================================

#*=============================================================================
#* DEFINE GLOBAL VARIABLES
#*=============================================================================
$startDateTime = Get-Date
$EnableEmail = 1
$DebugPreference = "SilentlyContinue"

if($debugScript -eq "Yes"){
	$DebugPreference = "Continue"	# Write-Debug commands.
	$EnableEmail = Read-Host("Enable email, 0 = No, 1 = Yes [0/1]: ")
}

#*=============================================================================
#* IMPORT SNAPINS AND MODULES
#*=============================================================================
try
{
	Import-Module ActiveDirectory -ErrorAction Stop
}
catch
{
	Write-Error $($Error[0].Exception.Message)
	# Send email or whatever...
	exit
}

#*=============================================================================
#* IMPORT LIBRARIES
#*=============================================================================

#*=============================================================================
#* EXCEPTION HANDLER
#*=============================================================================

#*=============================================================================
#* FUNCTION LISTINGS
#*=============================================================================

#*=============================================================================
#* Function:	New-LURDCMFile
#* ============================================================================
function New-RDCManFile
{
	<#
	.SYNOPSIS 
	This generates a Remote Desktop Manager file for computer objects within Active Directory.
	.DESCRIPTION
	This generates a Remote Desktop Manager file for computer objects within Active Directory.
	.PARAMETER username
	This username that you wish to be present in the RDG file by default.
	.PARAMETER outputPath
	The output path for the file (e.g. D:\).
	.PARAMETER computerArray
	Array of computer objects from Active Directory.
	.EXAMPLE
	Verb-LUServiceNoun -param1 "foo" -param2 "bar"
	.NOTES
	Author: Your Name
	#>
	
	
	# Leave previous two lines blank
	param(
		[Parameter(Mandatory=$true,HelpMessage="Admin account.")]
		[String]
		$username,

		[Parameter(Mandatory=$true,HelpMessage="Output Path for file.")]
		[String]
		$outputPath,
		
		[Parameter(Mandatory=$true,HelpMessage="Array of computers.")]
		[Array]
		$computerArray
	)
	
	
# Create a template XML. This needs to be indented to the margin so that the output XML file has no indent.
$template = @' 
<?xml version="1.0" encoding="utf-8"?> 
<RDCMan schemaVersion="1"> 
    <version>2.2</version> 
    <file> 
        <properties> 
            <name></name> 
            <expanded>True</expanded> 
            <comment /> 
            <logonCredentials inherit="FromParent" /> 
            <connectionSettings inherit="FromParent" /> 
            <gatewaySettings inherit="FromParent" /> 
            <remoteDesktop inherit="FromParent" /> 
            <localResources inherit="FromParent" /> 
            <securitySettings inherit="FromParent" /> 
            <displaySettings inherit="FromParent" /> 
        </properties> 
        <group> 
            <properties> 
                <name></name> 
                <expanded>True</expanded> 
                <comment /> 
                <logonCredentials inherit="None"> 
                    <userName></userName> 
                    <domain></domain> 
                    <password storeAsClearText="False"></password> 
                </logonCredentials> 
                <connectionSettings inherit="FromParent" /> 
                <gatewaySettings inherit="None"> 
                    <userName></userName> 
                    <domain></domain> 
                    <password storeAsClearText="False" />
                    <enabled>False</enabled> 
                    <hostName /> 
                    <logonMethod>4</logonMethod> 
                    <localBypass>False</localBypass> 
                    <credSharing>False</credSharing> 
                </gatewaySettings> 
                <remoteDesktop inherit="FromParent" /> 
                <localResources inherit="FromParent" /> 
                <securitySettings inherit="FromParent" /> 
                <displaySettings inherit="FromParent" /> 
            </properties> 
            <server> 
                <name></name> 
                <displayName></displayName> 
                <comment /> 
                <logonCredentials inherit="FromParent" /> 
                <connectionSettings inherit="FromParent" /> 
                <gatewaySettings inherit="FromParent" /> 
                <remoteDesktop inherit="FromParent" /> 
                <localResources inherit="FromParent" /> 
                <securitySettings inherit="FromParent" /> 
                <displaySettings inherit="FromParent" /> 
            </server> 
        </group> 
    </file> 
</RDCMan> 
'@ 
	
	$outputFile = $outputPath + "-$username" + ".rdg"
	
	# Output $template to a temporary XML file:
	$template | Out-File $home\RDCMan-template.xml -encoding UTF8 
	 
	# Load the XML template into XML object: 
	$xml = New-Object xml 
	$xml.Load("$home\RDCMan-template.xml") 
	 
	# Set the file properties:
	$file = (@($xml.RDCMan.file.properties)[0]).Clone() 
	$file.name = $domain 
	$xml.RDCMan.file.properties | Where-Object { $_.Name -eq "" } | ForEach-Object  { [void]$xml.RDCMan.file.ReplaceChild($file,$_) } 
	 
	# Set the group properties 
	$group = (@($xml.RDCMan.file.group.properties)[0]).Clone() 
	$group.name = $env:userdomain 
	$group.logonCredentials.Username = "$username"
	$group.logonCredentials.Domain = $domain

	$xml.RDCMan.file.group.properties | Where-Object { $_.Name -eq "" } | ForEach-Object  { [void]$xml.RDCMan.file.group.ReplaceChild($group,$_) } 
	 
	# Use template to add servers from Active Directory to the XML  
	$server = (@($xml.RDCMan.file.group.server)[0]).Clone()
	
	$computerArray | ForEach-Object {
	$server = $server.clone()
	[string]$server.DisplayName = $_.Name
	[string]$server.Name = $_.DNSHostName

	$xml.RDCMan.file.group.AppendChild($server) > $null} 
	# Remove template server 
	$xml.RDCMan.file.group.server | Where-Object { $_.Name -eq "" } | ForEach-Object { [void]$xml.RDCMan.file.group.RemoveChild($_) } 
	 
	# Save the XML object to a file 
	$xml.Save($outputFile) 
	 
	# Remove the temporary XML file:
	Remove-Item $home\RDCMan-template.xml -Force	
}

#*=============================================================================
#* END OF FUNCTION LISTINGS
#*=============================================================================

#*=============================================================================
#* SCRIPT BODY
#*=============================================================================
$domain = $Env:USERDOMAIN
# Base output path:
$outputPath = "C:\RDCMan"

# Example to get a list of MemberServers and Domain Controllers:
$computerObjects1 = Get-ADComputer -SearchBase "OU=MemberServers,DC=lunet,DC=lboro,DC=ac,DC=uk" -LDAPFilter "(operatingsystem=*Windows server*)"  | Select-Object -property name,dnshostname
$computerObjects2 = Get-ADComputer -SearchBase "OU=Domain Controllers,DC=lunet,DC=lboro,DC=ac,DC=uk" -LDAPFilter "(operatingsystem=*Windows server*)" | Select-Object -property name,dnshostname
$allComputers = $computerObjects1 + $computerObjects2 | Sort-Object

# Call the function to generate the file:
$filePrefix = "allservers"
New-RDCManFile -username "useraccount-admin" -outputPath "$outputPath\$filePrefix" -computerArray $allComputers
New-RDCManFile -username "useraccount2-admin" -outputPath "$outputPath\$filePrefix" -computerArray $allComputers


# Example to output a list of all SQL servers (from an AD security group):
$filePrefix = "sqlservers"
$sqlservers = Get-ADGroupMember -Identity "sql-servers" | Get-ADComputer | Select-Object -property name,dnshostname | Sort-Object -Property name
# Call the function to generate the file:
New-RDCManFile -username "useraccount-admin" -outputPath "$outputPath\$filePrefix" -computerArray $sqlservers

# Optional block to output script execution time:
#$endDateTime = Get-Date
#$scriptExecutionMin = ($endDateTime.Subtract($startDateTime).Minutes)
#$scriptExecutionSec = ($endDateTime.Subtract($startDateTime).Seconds)	
#Write-Output "Script execution time: $scriptExecutionMin min $scriptExecutionSec sec."
#*=============================================================================
#* END SCRIPT BODY
#*=============================================================================

#*=============================================================================
#* END OF SCRIPT
#*=============================================================================

Yet Another Invoke-SSH PowerShell Function (thanks PS Fab)

As part of a writing a decommission server PowerShell script at work, I had a requirement for a quick and easy SSH function to connect to our NetBackup server at work and remove the server from the backup system (Symantec if you’re reading this, please can we have a PowerShell module? *wishful thinking*…). Not quite needing the entire functionality provided by this module (based on the SSH.NET Library), I came across a function on PS Fab. One fantastic thing about this function is that supports the automatic acceptance of an SSH key when you connect to a host for the first time.

I made a few changes to the code, amended comments that referenced plist rather than plink and put it into a more standard function form with examples (that you might be able to add to an existing library). This is the result:

Function Invoke-SSH 
{
	<#
	.SYNOPSIS 
	Uses Plink.exe to SSH to a host and execute a list of commands.
	.DESCRIPTION
	Uses Plink.exe to SSH to a host and execute a list of commands.
	.PARAMETER hostname
	The host you wish to connect to.
	.PARAMETER username
	Username to connect with.
	.PARAMETER password
	Password for the specified user account.
	.PARAMETER commandArray
	A single, or list of commands stored in an array object.
	.PARAMETER plinkAndPath
	The location of the plink.exe including the executable (e.g. F:\tools\plink.exe)
	.PARAMETER connectOnceToAcceptHostKey
	If set to true, it will accept the remote host key (use when connecting for the first time)
	.EXAMPLE
	Invoke-SSH -username root -hostname centos-server -password Abzy4321! -plinkAndPath "F:\tools\plink.exe" -commandArray $commands -connectOnceToAcceptHostKey $true
	.EXAMPLE
	Invoke-SSH -username root -hostname centos-server -password Abzy4321! -plinkAndPath "F:\tools\plink.exe" -commandArray ifconfig -connectOnceToAcceptHostKey $true
	.NOTES
	Author: Robin Malik
	Source: Modified from: http://www.zerrouki.com/invoke-ssh/
	#>
	
	Param(
		[Parameter(Mandatory=$true,HelpMessage="Enter a host to connect to.")]
		[string]
		$hostname,
		
		[Parameter(Mandatory=$true,HelpMessage="Enter a username.")]
		[string]
		$username,
		
		[Parameter(Mandatory=$true,HelpMessage="Enter the password.")]
		[string]
		$password,
		
		[Parameter(Mandatory=$true,HelpMessage="Provide a command or comma separated list of commands")]
		[array]
		$commandArray,
		
		[Parameter(Mandatory=$true,HelpMessage="Path to plink (e.g. F:\tools\plink.exe).")]
		[string]
		$plinkAndPath,
		
		[Parameter(HelpMessage="Accept host key if connecting for the first time (the default is `$false)")]
		[string]
		$connectOnceToAcceptHostKey = $false
	)
	
	$target = $username + '@' + $hostname
	$plinkoptions = "-ssh $target -pw $password"
	 
	# On first connect to a host, plink will prompt you to accept the remote host key. 
	# This section will login and accept the host key then logout:
	if($ConnectOnceToAcceptHostKey)
	{
		$plinkCommand  = [string]::Format('echo y | & "{0}" {1} exit', $plinkAndPath, $plinkoptions )
		$msg = Invoke-Expression $plinkCommand
	}
	
	# Build the SSH Command by looping through the passed value(s). Append exit in order to logout:
	$commandArray += "exit"
	$commandArray | % { $remoteCommand += [string]::Format('{0}; ', $_) }
	
	# Format the command to pass to plink:
	$plinkCommand = [string]::Format('& "{0}" {1} "{2}"', $plinkAndPath, $plinkoptions , $remoteCommand)
	 
	# Execute the command and display the output:
	$msg = Invoke-Expression $plinkCommand
	Write-Output $msg
}

Copy and paste this function into a PowerShell window, and then test it with the below code (changing where appropriate of course):

$plinkAndPath = "F:\tools\plink\plink.exe"
$username = "root"
$password = "Adk3453#5341!"
$hostname = "centos-server"
# Commands to execute:
$Commands = @()
$Commands += "ifconfig"
$Commands += "ls"
Invoke-SSH -username $username -hostname $hostname -password $password -plinkAndPath $plinkAndPath -commandArray $Commands -connectOnceToAcceptHostKey $true