Bespoke learnings from an Identity & Access Management Consultant

I live in PowerShell and my memory is pretty good. There are a number of common PowerShell commands and one-liners or functions that I use a lot and I can remember them. However, then there are the ones I use less regularly and I often find myself trying to recall the last time I used them in a script, in order to locate that script just to get those couple of lines. So I’m posting a bunch of them in this post, if for nothing else to help me find them quickly. Let’s consider this as my common PowerShell Snippets and Volume 1.

Unix Time

For a number of API’s I interact with I need to provide the current time in Unix format as part of the API request. This online liner around Get-Date does that.

$utime = [int][double]::Parse((Get-Date -UFormat %s))

the output looks like this

1551906706

URL Encode

Often you need to encode a URL or query. The following shows taking a query and URL encoding it.

$query = "attributes.firstname='Darren' AND attributes.lastname='Robinson'"
Add-Type -AssemblyName System.Web
$queryEncoded = [System.Web.HttpUtility]::UrlEncode($query)

The encoded query then looks like

attributes.firstname%3d%27Darren%27+AND+attributes.lastname%3d%27Robinson%27

Basic Authentication Header

The following will create a Basic Authentication Header from a ClientID and Client Secret that can then be used with Invoke-RestMethod or Invoke-WebRequest

$clientID = 'abcd1234567'
$clientSecret = 'abcd12345sdkslslfjahd'
$Bytes = [System.Text.Encoding]::utf8.GetBytes("$($clientID):$($clientSecret)")
$encodedAuth =[Convert]::ToBase64String($Bytes)
$header = @{Authorization = "Basic $($encodedAuth)" }

You then use the $header variable in your web request e.g

invoke-restmethod -method get -uri "https://webservice.com" -headers $header

Converting a String to Proper/Title Case

Sometimes you get a string that is SHOUTING at you. Or just badly formatted and you need to make it look as it should. The following will convert a string to Title Case

$Surname = (Get-Culture).textinfo.totitlecase("BaDlY-ForMAtted SurNAME".tolower())

The BaDlY-ForMAtted SurNAME will then become

Badly-Formatted Surname

TLS

I have written this up before in more detail for a slightly different scenario here. But often the quick one-liner to force PowerShell to use TLS 1.2 for web requests is

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

Splitting a Large Collection into manageable chunks

Way to regularly I have a data-set of 100k+ objects. In order to parallelise the processing of that data into multiple threads I need to break that 100k+ collection into smaller chunks.

The following takes LARGEDATASET, say a collection of 10’s or 100’s of thousands of objects and splits it into collections of 1000.

$counter = [pscustomobject] @{ Value = 0 }
$groupSize = 1000
$groups = $LARGEDATASET | Group-Object -Property { [math]::Floor($counter.Value++ / $groupSize) }

You can then determine how many it created with

$groups.Count

and then checkout each Group by incrementing through the collection array

$groups[0].Group

Parallel Processing

As an extension of splitting a large collection into smaller collections you may then want to process a collection with multiple threads (e.g in Parallel).

The following snippet leverages the Invoke-Parallel Function from Rambling Cookie Monster that I’ve mentioned previously such as here.

Change the Throttle switch for the number of threads. Change the timeout if required (if for instance you are smashing your (not someone else’s) API.

& .\PATH-TO\invoke-parallel.ps1
$result = Invoke-Parallel -InputObject $CollectionOfObjects -throttle 10 -runspaceTimeout 60 -ImportVariables -ImportModules -ScriptBlock {
   try {
        $query = Invoke-RestMethod -method Get -uri "https://api.application.com" -headers $header
      } catch {
        write-host $_
      }
   @($query)
}
$result.Count

Join Large Collections

This I mentioned previously in this post here.  This snippet is the ability to quickly join objects between two large collections. It utilises the Join-Object function again from Rambling Cookie Monster

$reportData = Join-Object -Left $collection1 -Right $collection2 -LeftJoinProperty UserPrincipalName -RightJoinProperty UserPrincipalName -Type AllInLeft

Un-escaping a JSON Document

Using ConvertTo-JSON will escape special characters as shown below.

{"description": "Australian G\u0026f Logistics Ltd"}

Using the following will unescape the JSON document (if the system you’re interfacing with doesn’t unescape on consumption).

$jsonString | ConvertTo-Json | % { [System.Text.RegularExpressions.Regex]::Unescape($_) }

The JSON document will then become

{"description": "Australian G&f Logistics Ltd"}

That’s it for Vol 1 of Darren’s PowerShell snippets. I’ll start compiling others as I search for them and don’t find them in this Vol.