I live in PowerShell and my memory is pretty good. There are a number of common PowerShell commands and one-liners or functions that I use a lot and I can remember them. However, then there are the ones I use less regularly and I often find myself trying to recall the last time I used them in a script, in order to locate that script just to get those couple of lines. So I’m posting a bunch of them in this post, if for nothing else to help me find them quickly. Let’s consider this as my common PowerShell Snippets and Volume 1.
Unix Time
For a number of API’s I interact with I need to provide the current time in Unix format as part of the API request. This online liner around Get-Date does that.
$utime = [int][double]::Parse((Get-Date -UFormat %s))
the output looks like this
1551906706
URL Encode
Often you need to encode a URL or query. The following shows taking a query and URL encoding it.
$query = "attributes.firstname='Darren' AND attributes.lastname='Robinson'" Add-Type -AssemblyName System.Web $queryEncoded = [System.Web.HttpUtility]::UrlEncode($query)
The encoded query then looks like
attributes.firstname%3d%27Darren%27+AND+attributes.lastname%3d%27Robinson%27
Basic Authentication Header
The following will create a Basic Authentication Header from a ClientID and Client Secret that can then be used with Invoke-RestMethod or Invoke-WebRequest
$clientID = 'abcd1234567'
$clientSecret = 'abcd12345sdkslslfjahd'
$Bytes = [System.Text.Encoding]::utf8.GetBytes("$($clientID):$($clientSecret)")
$encodedAuth =[Convert]::ToBase64String($Bytes)
$header = @{Authorization = "Basic $($encodedAuth)" }
You then use the $header variable in your web request e.g
invoke-restmethod -method get -uri "https://webservice.com" -headers $header
Converting a String to Proper/Title Case
Sometimes you get a string that is SHOUTING at you. Or just badly formatted and you need to make it look as it should. The following will convert a string to Title Case
$Surname = (Get-Culture).textinfo.totitlecase("BaDlY-ForMAtted SurNAME".tolower())
The BaDlY-ForMAtted SurNAME will then become
Badly-Formatted Surname
TLS
I have written this up before in more detail for a slightly different scenario here. But often the quick one-liner to force PowerShell to use TLS 1.2 for web requests is
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Splitting a Large Collection into manageable chunks
Way to regularly I have a data-set of 100k+ objects. In order to parallelise the processing of that data into multiple threads I need to break that 100k+ collection into smaller chunks.
The following takes LARGEDATASET, say a collection of 10’s or 100’s of thousands of objects and splits it into collections of 1000.
$counter = [pscustomobject] @{ Value = 0 }
$groupSize = 1000
$groups = $LARGEDATASET | Group-Object -Property { [math]::Floor($counter.Value++ / $groupSize) }
You can then determine how many it created with
$groups.Count
and then checkout each Group by incrementing through the collection array
$groups[0].Group
Parallel Processing
As an extension of splitting a large collection into smaller collections you may then want to process a collection with multiple threads (e.g in Parallel).
The following snippet leverages the Invoke-Parallel Function from Rambling Cookie Monster that I’ve mentioned previously such as here.
Change the Throttle switch for the number of threads. Change the timeout if required (if for instance you are smashing your (not someone else’s) API.
& .\PATH-TO\invoke-parallel.ps1
$result = Invoke-Parallel -InputObject $CollectionOfObjects -throttle 10 -runspaceTimeout 60 -ImportVariables -ImportModules -ScriptBlock {
try {
$query = Invoke-RestMethod -method Get -uri "https://api.application.com" -headers $header
} catch {
write-host $_
}
@($query)
}
$result.Count
Join Large Collections
This I mentioned previously in this post here. This snippet is the ability to quickly join objects between two large collections. It utilises the Join-Object function again from Rambling Cookie Monster
$reportData = Join-Object -Left $collection1 -Right $collection2 -LeftJoinProperty UserPrincipalName -RightJoinProperty UserPrincipalName -Type AllInLeft
Un-escaping a JSON Document
Using ConvertTo-JSON will escape special characters as shown below.
{"description": "Australian G\u0026f Logistics Ltd"}
Using the following will unescape the JSON document (if the system you’re interfacing with doesn’t unescape on consumption).
$jsonString | ConvertTo-Json | % { [System.Text.RegularExpressions.Regex]::Unescape($_) }
The JSON document will then become
{"description": "Australian G&f Logistics Ltd"}