How to configure a Graphical PowerShell Dev/Admin/Support User Interface for Azure/Office365/Microsoft Identity Manager

During the development of an identity management solution I find myself with multiple PowerShell/RDP sessions connected to multiple environments using different credentials often to obtain trivial data/information. It is easy to trip yourself up as well with remote powershell sessions to differing environments. If only there was a simple UI that could front-end a set of PowerShell modules and make those simple queries quick and painless. Likewise to allow support staff to execute a canned set of queries without providing them elevated permissions.

I figured someone would have already solved this problem and after some searching with the right keywords I found the powershell-command-executor-ui from bitsofinfo . Looking into it he had solved a lot of the issues with building a UI front-end for PowerShell with the powershell-command-executor and the stateful-process-command-proxy. That solution provided the framework for what I was thinking. The ability to provide a UI for PowerShell using powershell modules including remote powershell was exactly what I was after. And it was built on NodeJS and AngularJS so simple enough for some customization.

Introduction

In this blog post I’ll detail how I’ve leveraged the projects listed above for integration with;

Initially I had a vision of serving up the UI from an Azure WebApp. NodeJS on Azure WebApp’s is supported, however with all the solution dependencies I just couldn’t get it working.

My fallback was to then look to serve up the UI from a Windows Server 2016 Nano Server. I learnt from my efforts that a number of the PowerShell modules I was looking to provide a UI for, have .NET Framework dependencies. Nano Server does not have full .NET Framework support. Microsoft state to do so would mean the server would no longer be Nano.

For now I’ve deployed an Azure Windows Server 2016 Server secured by an Azure NSG to only allow my machine to access it. More on security later.

Overview

Simply, put the details in Github for the powershell-command-executor provide the architecture and integration. What I will detail is the modifications I’ve made to utilize the more recent AzureADPreview PowerShell Module over the MSOL PowerShell Module. I also updated the dependencies of the solution for the latest versions and hooked it into Microsoft Identity Manager. I also made a few changes to allow different credentials to be used for Azure and Microsoft Identity Manager.

Getting Started

I highly recommend you start with your implementation on a local development workstation/development virtual machine. When you have a working version you’re happy with you can then look at other ways of presenting and securing it.

NodeJS

NodeJS is the webserver for this solution. Download NodeJS for your Windows host here. I’m using the 64-bit version, but have also implemented the solution on 32-bit. Install NodeJS on your local development workstation/development virtual machine.

You can accept all the defaults.

Following the installation of NodeJS download the powershell-command-executor-ui from GitHub. Select Clone 0r Download, Download ZIP and save it to your machine.

Right click the download when it has finished and select Extract All. Select Browse and create a folder at the root of C:\ named nodejs. Extract powershell-command-executor-ui.

Locate the c:\nodejs\powershell-command-executor-ui-master\package.json file.

Using an editor such as Notepad++ update the package.json file ……

…… so that it looks like the following. This will utilise the latest versions of the dependencies for the solution.

From an elevated (Administrator) command prompt in the c:\nodejs\powershell-command-executor-ui-master directory run “c:\program files\nodejs npm” installThis will read the package.json file you edited and download the dependencies for the solution.

You can see in the screenshot below NodeJS has downloaded all the items in package.json including the powershell-command-executor and stateful-process-command-proxy.

When you now list the directories under C:\nodejs\powershell-command-executor-ui-master\node_modules you will see those packages and all their dependencies.

We can now test that we have a working PowerShell UI NodeJS website. From an elevated command prompt whilst still in the c:\nodejs\powershell-command-executor-ui-master directory run “c:\Program Files\nodejs\node.exe” bin\www

Open a browser on the same host and go to “http://localhost:3000”. You should see the default UI.

Configuration and Customization

Now it is time to configure and customize the PowerShell UI for our needs.

The files we are going to edit are:

  • C:\nodejs\powershell-command-executor-ui-master\routes\index.js
    • Update Paths to the encrypted credentials files used to connect to Azure, MIM. We’ll create the encrypted credentials files soon.
  • C:\nodejs\powershell-command-executor-ui-master\public\console.html
    • Update for your customizations for CSS etc.
  • C:\nodejs\powershell-command-executor-ui-master\node_modules\powershell-command-executor\O365Utils.js
    • Update for PowerShell Modules to Import
    • Update for Commands to make available in the UI

We also need to get a couple of PowerShell Modules installed on the host so they are available to the site. The two I’m using I’ve mentioned earlier. With WMF5 intalled using Powershell we can simply install them as per the commands below.

Install-Module AzureADPreview
Install-Module LithnetRMA

In order to connect to our Microsoft Identity Manager Synchronization Server we are going to need to enable Remote Powershell on our Microsoft Identity Manager Synchronization Server. This post I wrote here details all the setup tasks to make that work. Test that you can connect via RPS to your MIM Sync Server before updating the scripts below.

Likewise for the Microsoft Identity Manager Service Server. Make sure after installing the LithnetRMA Powershell Module you can connect to the MIM Service using something similar to:

# Import LithnetRMA PS Module
import-module lithnetrma

# MIM AD User Admin
$username = "mimadmin@mim.mydomain.com"
# Password 
$password = "Secr3tSq1rr3l!" | convertto-securestring -AsPlainText -Force
# PS Creds
$credentials = New-Object System.Management.Automation.PSCredential $Username,$password

# Connect to the FIM service instance
# Will require an inbound rule for TCP 5725 (or your MIM Service Server Port) in you Resource Group Network Security Group Config
Set-ResourceManagementClient -BaseAddress http://mymimportalserver.:5725 -Credentials $credentials

 

\routes\index.js

This file details the encrypted credentials the site uses. You will need to generate the encrypted credentials for your environment. You can do this using the powershell-credentials-encryption-tools. Download that script to your workstation and unzip it. Open the credentialEncryptor.ps1 script using an Administrator PowerShell ISE session.

I’ve changed the index.js to accept two sets of credentials. This is because your Azure Admin Credentials are going to be different from your MIM Administrator Credentials (both in name and password). The username for my Azure account looks something like myname@mycompany.com whereas for MIM it is Domainname\Username.

Provide an account name for your Azure environment and the associated password.

The tool will create the encrypted credential files.

Rename the encrypted.credentials file to whatever makes sense for your environment. I’ve renamed it creds1.encrypted.credentials.

Now we re-run the script to create another set of encrypted credentials. This time for Microsoft Identity Manager. Once created, rename the encrypted.credentials file to something that makes sense in your environment. I’ve renamed the second set to creds2.encrypted.credentials.

We now need to copy the following files to your UI Website C:\nodejs\powershell-command-executor-ui-master directory:

  • creds1.encrypted.credentials
  • creds2.encrypted.credentials
  • decryptUtil.ps1
  • secret.key

Navigate back to Routes.js and open the file in an editor such as Notepad++

Update the index.js file for the path to your credentials files. We also need to add in the additional credentials file.

The changes to the file are, the paths to the files we just copied above along with the addition var PATH_TO_ENCRYPTED_RPSCREDENTIALS_FILE for the second set of credentials used for Microsoft Identity Manager.

var PATH_TO_DECRYPT_UTILS_SCRIPT = "C:\\nodejs\\powershell-command-executor-ui-master\\decryptUtil.ps1";
var PATH_TO_ENCRYPTED_CREDENTIALS_FILE = "C:\\nodejs\\powershell-command-executor-ui-master\\creds1.encrypted.credentials";
var PATH_TO_ENCRYPTED_RPSCREDENTIALS_FILE = "C:\\nodejs\\powershell-command-executor-ui-master\\creds2-encrypted.credentials";
var PATH_TO_SECRET_KEY = "C:\\nodejs\\powershell-command-executor-ui-master\\secret.key";


Also to initCommands to pass through the additional credentials file


initCommands: o365Utils.getO365PSInitCommands(
 PATH_TO_DECRYPT_UTILS_SCRIPT,
 PATH_TO_ENCRYPTED_CREDENTIALS_FILE,
 PATH_TO_ENCRYPTED_RPSCREDENTIALS_FILE,
 PATH_TO_SECRET_KEY,
 10000,30000,3600000),

Here is the full index.js file for reference.

 

public/console.html

The public/console.html file is for formatting and associated UI components. The key things I’ve updated are the Bootstrap and AngularJS versions. Those are contained in the top of the html document. A summary is below.

https://ajax.googleapis.com/ajax/libs/angularjs/1.6.1/angular.min.js
https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.6.1/angular-resource.min.js
http://javascripts/ui-bootstrap-tpls-2.4.0.min.js
http://javascripts/console.js
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css">

You will also need to download the updated Bootstrap UI (ui-bootstrap-tpls-2.4.0.min.js). I’m using v2.4.0 which you can download from here. Copy it to the javascripts directory.

I’ve also updated the table types, buttons, colours, header, logo etc in the appropriate locations (CSS, Tables, Div’s etc). Here is my full file for reference. You’ll need to update for your colours, branding etc.

powershell-command-executor\O365Utils.js

Finally the O365Utils.js file. This contains the commands that will be displayed along with their options, as well as the connection information for your Microsoft Identity Manager environment.

You will need to change:

  • Line 52 for the address of your MIM Sync Server
  • Line 55 for the addresses of your MIM Service Server
  • Line 141 on-wards for what commands and parameters for those commands you want to make available in the UI

Here is an example with a couple of AzureAD commands, a MIM Sync and a MIM Service command.

Show me my PowerShell UI Website

Now that we have everything configured let’s start the site and browse to it. If you haven’t stopped the NodeJS site from earlier go to the command window and press Cntrl+C a couple of times. Run “c:\Program Files\nodejs\node.exe” bin\www again from the C:\nodejs\powershell-command-executor-ui-master directory unless you have restarted the host and now have NodeJS in your environment path.

In a browser on the same host go to http://localhost:3000 again and you should see the site as it is below.

Branding and styling from the console.html, menu options from the o365Utils.js and when you select a command and execute it data from the associated service …….

… you can see results. From the screenshot below a Get-AzureADUser command for the associated search string executed in milliseconds.

 

Summary

The powershell-command-executor-ui from bitsofinfo is a very extensible and powerful NodeJS website as a front-end to PowerShell.

With a few tweaks and updates the look and feel can be easily changed along with the addition of any powershell commands that you wish to have a UI for.

As it sits though keep in mind you have a UI with hard-coded credentials that can do whatever commands you expose.

Personally I am running one for my use only and I have it hosted in Azure in its own Resource Group with an NSG allowing outgoing traffic to Azure and my MIM environment. Incoming traffic is only allowed from my personal management workstations IP address. I also needed to allow port 3000 into the server on the NSG as well as the firewall on the host. I did that quickly using the command below.

# Enable the WebPort NodeJS is using on the firewall 
netsh advfirewall firewall add rule name="NodeJS WebPort 3000" dir=in action=allow protocol=TCP localport=3000

Follow Darren on Twitter @darrenjrobinson

How to create an Azure Function App to Simultaneously Start|Stop all Virtual Machines in a Resource Group

Just on a year ago I wrote this blog post that detailed a method to “Simultaneously Start|Stop all Azure Resource Manager Virtual Machines in a Resource Group”. It’s a simple script that I use quite a lot and I’ve received a lot of positive feedback on it.

One year on though and there are a few enhancements I’ve been wanting to make to it. Namely;

  • host the script in an environment that is a known state. Often I’m authenticated to different Azure Subscriptions, my personal, my employers and my customers.
  • prioritize the order the virtual machines startup|shutdown
  • allow for a delay between starting each VM (to account for environments where the VM’s have roles that have cross dependencies; e.g A Domain Controller, an SQL Server, Application Servers). You want the DC to be up and running before the SQL Server, and so forth
  • and if I do all those the most important;
    • secure it so not just anyone can start|stop my environments at their whim

Overview

This blog post is the first that executes the first part of implementing the script in an environment that is a known state aka implementing it as an Azure Function App. This won’t be a perfect implementation as you will see, but will set the foundation for the other enhancements. Subsequent posts (as I make time to develop the enhancements) will add the new functionality. This post covers;

  • Creating the Azure Function App
  • Creating the foundation for automating management of Virtual Machines in Azure using Azure Function Apps
  • Starting | Stopping all Virtual Machines in an Azure Resource Group

Create a New Azure Function App

First up we are going to need a Function App. Through your Azure Resource Manager Portal create a new Function App.

For mine I’ve created a new Resource Group and a new Storage Account as this solution will flesh out over time and I’d like to keep everything organised.

Now that we have the Azure App Plan setup, create a New PowerShell HTTP Trigger Function App.

Give it a name and hit Create.

 

Create Deployment Credentials

In order to get some of the dependencies into the Azure Function we need to create deployment credentials so we can upload them. Head to the Function App Settings and choose Go to App Service Settings.

Create a login and give it a password. Record the FTP/Deployment username and the FTP hostname along with your password as you’ll need this in the next step.

Upload our PowerShell  Modules and Dependencies

Just as my original PowerShell script did I’m using the brilliant Invoke Parallel Powershell Script from Rambling Cookie Monster. Download it from that link and save it to your local machine.

Connect to your Azure Function App using your favourite FTP Client using the credentials you created earlier. I’m using WinSCP. Create a new sub-directory under /site/wwwroot/ named “bin” as shown below.

Upload the Invoke-Parallel.ps1 file from wherever you extracted it to on your local machine to the bin folder you just created in the Function App.

We are also going to need the AzureRM Powershell Modules. Download those via Powershell to your local machine (eg. Save-Module -Name AzureRM -Path c:\temp\azurerm). There are a lot of modules obviously and you’re not going to need them all. At a minimum for this solution you’ll need;

  • AzureRM
  • AzureRM.profile
  • AzureRM.Compute

Upload them under the bin directory also as shown below.

Test that our script dependencies are accessible

Now that we have our dependent modules uploaded let’s test that we can load and utilise them. Below is commands to load the Invoke-Parallel script and test that it has loaded by getting the Help.

# Load the Invoke-Parallel Powershell Script
. "D:\home\site\wwwroot\RG-Start-Stop-VirtualMachines\bin\Invoke-Parallel.ps1"

# See if it is loaded by getting some output
Get-Help Invoke-Parallel -Full

Put those lines into the code section, hit Save and Run and select Logs to see the output. If successful you’ll see the help. If you don’t you probably have a problem with the path to where you put the Invoke-Parallel script. You can use the Kudu Console from the Function App Settings to get a command line and verify your path.

Mine worked successfully. Now to test our AzureRM Module Loads. Update the Function to load the AzureRM Profile PSM as per below and test you have your path correct.

# Import the AzureRM Powershell Module
import-module 'D:\home\site\wwwroot\RG-Start-Stop-VirtualMachines\bin\AzureRM.profile\2.4.0\AzureRM.Profile.psm1'
Get-Help AzureRM

Success. Fantastic.

Create an Azure Service Principal

In order to automate the access and control of the Azure Virtual Machines we are going to need to connect to Azure using a Service Principal with the necessary permissions to manage the Virtual Machines.

The following script does just that. You only need to run this as part of the setup for the Azure Function so we have an account we can use for our automation tasks. Update line 6 for your naming and the password you want to use. I’m assigning the Service Principal the “DevTest Labs User” Azure Role (Line 17) as that allows the ability to manage the Virtual Machines. You can find a list of the available roles here.

Take note of the key outputs from this script. You will need to note the;

  • ApplicationID
  • TenantID

I’m also securing the credential that has the permissions to Start|Stop the Virtual Machines using the example detailed here in Tao’s post.

For reference here is an example to generate the keyfile. Update your path in line 5 if required and make sure the password you supply in line 18 matches the password you supplied for the line in the script (line 6) when creating the Security Principal.

Take note of the password encryption string from the end of the script to pair with the ApplicationID and TenantID from the previous steps. You’ll need these shortly in Application Settings.

Additional Dependencies

I created another sub-directory under the function app site named ‘keys’ again using WinSCP. Upload the passkey file created above into that directory.

Whilst we’re there I also created a “logs” directory for any erroneous output (aka logfiles created when you don’t specify them) from the invoke-parallel script.

Application Variables

Using the identity information you have created and generated we will populate variables on the Function App, Application Settings that we can then leverage in our Function App. Go to your Azure Function App, Application Settings and add an application setting (with the respective values you have gathered in the previous steps) for;

  • AzureAutomationPWD
  • AzureAutomationAppID
  • AzureAutomationTennatID (bad speed typing there)

Don’t forget to click Save up the top of the Application Settings screen.

 

The Function App Script

Below is the sample script for your testing purposes. If you plan to use something similar in a production environment you’ll want to add more logging and error handling.

Testing the Function

Select the Test option from the right-hand side pane and update the request body for what the Function takes (mode and resourcegroup) as below.   Select Run and watch the logs. You will need to select Expand to get more screen real estate for them.

You will see the VM’s enumerate then the script starting them all up. My script has a 30 second timeout for the Invoke-Parallel Runspace as the VM’s will take longer than 30 seconds to startup. And you pay for use, so we want to keep this lean. Increase the timeout if you have more VM’s or latency that doesn’t see all your VM’s state transitioning.

Checking in the Azure Portal I can see my VM’s all starting up (too fast on the screenshot for the spfarm-mim host).

 

Sample Remote PowerShell Invoke Script

Below is a sample PowerShell script that is remotely calling the Azure Function and providing the info the Function takes (mode and resourcegroup) the same as we did in the Test Request Body script in the Azure Function Portal.  This time to stop the VMs.

Looking in the Azure Portal and we can see all the VMs shutting down.

 

Summary

A foundational implementation of an Azure Function App to perform orchestration of Azure Virtual Machines.

The Function App is rudimentary in that the script exits (as described in the Runspace timeout) after 30 seconds which is prior to the VMs fully returning after starting|stopping. This is because the Function App will timeout after 5mins anyway.

Now to workout the enhancements to it.

Finally, yes I have renewed/changed the Function Key so no-one else can initiate my Function 🙂

Follow Darren Robinson on Twitter

Resolving “The Microsoft Identity Manager server database could not be successfully populated” installation error

Here is yet another of those Microsoft Identity Manager installation errors that doesn’t give you much information and when looking for a resolution you can’t find an exact match through Dr Google.

Nearing the end of the Microsoft Identity Manager Service and Portal installation you receive the “The Microsoft Identity Manager server database could not be successfully populated” error.

Looking into the installation log (which I’m in the good practice of always initiating when doing an installation of the MIM Service/Portal these days eg. msiexec /i “e:\Service and Portal\Service and Portal.msi” /l*v c:\temp\install.log )  didn’t give up much information at all. Fatal Error. Dialog created.

Looking at the server that the installation was being done on I could see that it was being spanked. This server is for a customers development environment, hosted in Azure but also done rather frugally (my Virtual Machine was running, SQL, MIM Sync, all the dependencies for the MIM Portal and then the MIM Service/Portal itself). FWIW the initially provisioned VM was an Azure DS1v2 server. Seems a character may have got lost in the VM request where an Azure DS11v2 server would have been more appropriate.

I re-sized the Azure VM and actually chose to go with an Azure DS3v2 VM size. I kicked off the Microsoft Identity Manager Service & Portal installation again and ….

….. SUCCESS.

Hope this helps someone else who may find themselves in a similar position.

Follow Darren on Twitter @darrenjrobinson

Microsoft Identity Manager installation error “Internal Error 2337. 0, Microsoft.MetadirectoryServices.host.dll”

Today I was doing a fresh installation of Microsoft Identity Manger 2016 with Service Pack 1 into a new development environment. The exact binary is “en_microsoft_identity_manager_2016_with_service_pack_1_x64_dvd_9270854”

Not too far into the installation of the Microsoft Identity Manager Synchronization Server I got the “Internal Error 2337. 0, Microsoft.MetadirectoryServices.host.dll” error as shown below.

Doing a few searches didn’t throw me any bones. I could see that the installation had added the MIM Sync Server Service Account to the Logins on the SQL Server.

I then recalled that there was an updated version that was released just before Xmas (14 Dec 2016). The exact binary is “en_microsoft_identity_manager_2016_with_service_pack_1_x64_dvd_9656597” and you can get the updated Microsoft Identity Manager 2016 with SP1 media here.

Re-running the installation and ….

….. SUCCESS.

Now I’m not going to spend any time trying to figure out was is bad with the base MIM 2016 SP1 media, I’m just going to pretend it never existed and use the latest. Onward and upwards.

Follow Darren on Twitter @darrenjrobinson

 

 

 

 

 

How to use a Powershell Azure Function to Tweet IoT environment data

Overview

This blog post details how to use a Powershell Azure Function App to get information from a RestAPI and send a social media update.

The data can come from anywhere, and in the case of this example I’m getting the data from WioLink IoT Sensors. This builds upon my previous post here that details using Powershell to get environmental information and put it in Power BI.  Essentially the difference in this post is outputting the manipulated data to social media (Twitter) whilst still using a TimerTrigger Powershell Azure Function App to perform the work and leverage the “serverless” Azure Functions model.

Prerequisites

The following are prerequisites for this solution;

Create a folder on your local machine for the Powershell Module then save the module to your local machine using the powershell command ‘Save-Module” as per below.

Save-Module -Name InvokeTwitterAPIs -Path c:\temp\twitter

Create a Function App Plan

If you don’t already have a Function App Plan create one by searching for Function App in the Azure Management Portal. Give it a Name, Select Consumption so you only pay for what you use, and select an appropriate location and Storage Account.

Create a Twitter App

Head over to http://dev.twitter.com and create a new Twitter App so you can interact with Twitter using their API. Give you Twitter App a name. Don’t worry about the URL too much or the need for the Callback URL. Select Create your Twitter Application.

Select the Keys and Access Tokens tab and take a note of the API Key and the API Secret. Select the Create my access token button.

Take a note of your Access Token and Access Token Secret. We’ll need these to interact with the Twitter API.

Create a Timer Trigger Azure Function App

Create a new TimerTrigger Azure Powershell Function. For my app I’m changing from the default of a 5 min schedule to hourly on the top of the hour. I did this after I’d already created the Function App as shown below. To update the schedule I edited the Function.json file and changed the schedule to “schedule”: “0 0 * * * *”

Give your Function App a name and select Create.

Configure Azure Function App Application Settings

In your Azure Function App select “Configure app settings”. Create new App Settings for your Twitter Account, Twitter Account AccessToken, AccessTokenSecret, APIKey and APISecret using the values from when you created your Twitter App earlier.

Deployment Credentials

If you haven’t already configured Deployment Credentials for your Azure Function Plan do that and take note of them so you can upload the Twitter Powershell module to your app in the next step.

Take note of your Deployment Username and FTP Hostname.

https://dl.dropboxusercontent.com/u/76015/BlogImages/FunctionIOTWeather/AppDevSettings.png

Upload the Twitter Powershell Module to the Azure Function App

Create a sub-directory under your Function App named bin and upload the Twitter Powershell Module using a FTP Client. I’m using WinSCP.

From the Applications Settings option start Kudu.

Traverse the folder structure to get the path do the Twitter Powershell Module and note it.

Update the code to replace the sample from the creation of the Trigger Azure Function as shown below to import the Twitter Powershell Module. Include the get-help lines for the module so we can see in the logs that the modules were imported and we can see the cmdlets they contain.

Validating our Function App Environment

Update the code to replace the sample from the creation of the Trigger Azure Function as shown below to import the Twitter Powershell Module. Include the get-help line for the module so we can see in the logs that the module was imported and we can see the cmdlets they contain. Select Save and Run.

Below is my output. I can see the output from the Twitter Module.

Function Application Script

Below is my sample script. It has no error handling etc so isn’t production ready, but gives a working example of getting data in from an API (in this case IoT sensors) and sends a tweet out to Twitter.

Viewing the Tweet

And here is the successful tweet.

Summary

This shows how easy it is to utilise Powershell and Azure Function Apps to get data and transform it for use in other ways. In this example a social media platform. The input could easily be business data from an API and the output a corporate social platform such as Yammer.

Follow Darren on Twitter @darrenjrobinson

How to use a Powershell Azure Function App to get RestAPI IoT data into Power BI for Visualization

Overview

This blog post details using a Powershell Azure Function App to get IoT data from a RestAPI and update a table in Power BI with that data for visualization.

The data can come from anywhere, however in the case of this post I’m getting the data from WioLink IoT Sensors. This builds upon my previous post here that details using Powershell to get environmental information and put it in Power BI.  Essentially the major change is to use a TimerTrigger Azure Function to perform the work and leverage the “serverless” Azure Functions model. No need for a reporting server or messing around with Windows scheduled tasks.

Prerequisites

The following are the prerequisites for this solution;

  • The Power BI Powershell Module
  • Register an application for RestAPI Access to Power BI
  • A Power BI Dataset ready for the data to go into
  • AzureADPreview Powershell Module

Create a folder on your local machine for the Powershell Modules then save the modules to your local machine using the powershell command ‘Save-Module” as per below.

Save-Module -Name PowerBIPS -Path C:\temp\PowerBI
Save-Module -Name AzureADPreview -Path c:\temp\AzureAD 

Create a Function App Plan

If you don’t already have a Function App Plan create one by searching for Function App in the Azure Management Portal. Give it a Name, Select Consumption Plan for the Hosting Plan so you only pay for what you use, and select an appropriate location and Storage Account.

Register a Power BI Application

Register a Power BI App if you haven’t already using the link and instructions in the prerequisites. Take a note of the ClientID. You’ll need this in the next step.

Configure Azure Function App Application Settings

In this example I’m using Azure Functions Application Settings for the Azure AD AccountName, Password and the Power BI ClientID. In your Azure Function App select “Configure app settings”. Create new App Settings for your UserID and Password for Azure (to access Power BI) and our PowerBI Application Client ID. Select Save.

Not shown here I’ve also placed the URL’s for the RestAPI’s that I’m calling to get the IoT environment data as Application Settings variables.

Create a Timer Trigger Azure Function App

Create a new TimerTrigger Azure Powershell Function App. The default of a 5 min schedule should be perfect. Give it a name and select Create.

Upload the Powershell Modules to the Azure Function App

Now that we have created the base of our Function App we’re going to need to upload the Powershell Modules we’ll be using that are detailed in the prerequisites. In order to upload them to your Azure Function App, go to App Service Settings => Deployment Credentials and set a Username and Password as shown below. Select Save.

Take note of your Deployment Username and FTP Hostname.

Create a sub-directory under your Function App named bin and upload the Power BI Powershell Module using a FTP Client. I’m using WinSCP.

To make sure you get the correct path to the powershell module from Application Settings start Kudu.

Traverse the folder structure to get the path to the Power BI Powershell Module and note the path and the name of the psm1 file.

Now upload the Azure AD Preview Powershell Module in the same way as you did the Power BI Powershell Module.

Again using Kudu validate the path to the Azure AD Preview Powershell Module. The file you are looking for is the Microsoft.IdentityModel.Clients.ActiveDirectory.dll” file. My file after uploading is located in “D:\home\site\wwwroot\MyAzureFunction\bin\AzureADPreview\2.0.0.33\Microsoft.IdentityModel.Clients.ActiveDirectory.dll”

This library is used by the Power BI Powershell Module.

Validating our Function App Environment

Update the code to replace the sample from the creation of the Trigger Azure Function as shown below to import the Power BI Powershell Module. Include the get-help line for the module so we can see in the logs that the modules were imported and we can see the cmdlets they contain. Select Save and Run.

Below is my output. I can see the output from the Power BI Module get-help command. I can see that the module was successfully loaded.

Function Application Script

Below is my sample script. It has no error handling etc so isn’t production ready, but gives a working example of getting data in from an API (in this case IoT sensors) and puts the data directly into Power BI.

Viewing the data in Power BI

In Power BI it is then quick and easy to select our Inside and Outside temperature readings referenced against time. This timescale is overnight so both sensors are reading quite close to each other.

Summary

This shows how easy it is to utilise Powershell and Azure Function Apps to get data and transform it for use in other ways. In this example a visualization of IoT data into Power BI. The input could easily be business data from an API and the output a real time reporting dashboard.

Follow Darren on Twitter @darrenjrobinson

 

 

 

 

Using an Azure Function to search the FIM/MIM Metaverse, create a Set and update the Set membership in the the FIM/MIM Service

Introduction

This is the third and last post in this series of integrating Microsoft Identity Manager with Azure Functions.

The first detailed how to use an Azure Function to retrieve data from the MIM Service Server. The second detailed how to use an Azure Function to retrieve data from the MIM Sync (Metaverse) Server.

This third post combines the two and then performs an action in the MIM Service. The practical purpose of this could be functions like “find all users in location y” and “enable them for entitlement x” or “add an attribute value on each of their objects”.

Overview

The reasoning for the two stage approach is that in my experience it is a lot easier to search the Metaverse than the MIM Service to find an object(s), but also the Metaverse has all the information about objects whereas the MIM Service is a ShadowVerse of the Metaverse containing a subset of the managed objects metadata.

Moving forward then the architecture is a hybrid of the first two posts that introduced the concepts associated with integrating MIM with Azure Functions. As per the other two posts this is a base working example and concept.

Prerequisites

The prerequisites are the same as for the 1st and 2nd posts in this series. You’ll need to work through those examples to setup the dependencies and prerequisites. From there you can create one more Azure Function that brings everything together. That’s what I’m covering in this post.

Therefore the prerequisites are;

  • Azure Tenant and a Function Plan
  • Microsoft Identity Manager implementation
  • Remote Powershell configured for your MIM Sync Server
  • Lithnet FIM MIIS Automation Powershell Module installed on your MIM Sync Server
  • The necessary Firewall Rules on your MIM Sync Server and your Azure Network Security Group (assuming your MIM Infrastructure is in Azure) to allow Azure Functions to communicate with MIM Sync and Service Servers

This Example performs the following

In this example the HTTP Trigger Azure Function;

  • Takes input for ObjectType, Attribute, AttributeValue, SetName
  • Searches the MIM Sync Metaverse for the input ObjectType, with the AttributeValue in the Attribute
  • Connects to the MIM Service
  • Creates the Set based of the input SetName if it doesn’t exist
  • Adds the objects from the search to the Set
  • Returns the objects added to the Set

In a real world implementation you’d do the above with a criteria based set. This post is a concept of search and find, performing a create and updating. That has many practical applications.

Create your new Azure Function

Just like the other two posts, we’re going to create a new Powershell HTTP Trigger Azure Function.

Upload the Lithnet RMA PS Module to your new Azure Function (as per blog post 1 in this series). You should also be using protected credentials now as well. So upload your username/password encryption key.

Here is the Azure Function Powershell Script that performs the process detailed above.

Test it out. Looks good. 88 users matched the value of Sydney in their location attribute.

Verify that the Set was created and the membership updated.

Test calling the Azure Function remotely

Now that it is all working in the Azure Function, lets try doing it from Powershell remotely. This time I’m again looking for Person objects that have Sydney in their location attribute and I’ll create a set named Sydney-NSW and put them in it.

Brilliant, that works nicely. Let’s verify that the Set was created and has the correct number of users in it. Yes, a perfect match.

Summary

Putting Azure Functions and Powershell together along with the Lithnet Powershell Modules opens up a world of possibilities for automation and integration of the MIM Service without the need for any additional infrastructure or any considerable effort.

Experiment and let me know what you do with this style of integration.

Follow Darren on Twitter @darrenjrobinson

Remotely managing your FIM/MIM Synchronisation Server using Powershell and the Lithnet MIIS Automation Powershell Module

Background

I’ve been using Ryan’s Lithnet MIIS Automation Powershell Module for a few months now as you’ve likely seen from some of my blog posts.

The module and its installer direct you to install the module on your FIM/MIM Synchronisation Server. This all makes perfect sense as the FIM/MIM Synchronsation Sever is more of your traditional server application. However we are no longer living in that kind of IT world. Consultants, Administrators, Architects, DevOps etc all want the flexibility to use their own workstations, administrative workstations, automation services etc.

So how do we apply that to the FIM/MIM Synchronisation Server ? Well, with the Lithnet MIIS Automation Powershell Module installed on your FIM/MIM Synchronisation Server you can, thanks to the wonderful thing that is Powershell. Thanks Jeffrey Snover.

Overview

In this blog post I’ll detail the quick and easy steps to enable you to remotely administer, orchestrate, report and query your FIM/MIM Sync Server and Metaverse using Powershell and the Lithnet Automation Powershell Module.

The diagram below outlines the topology. Essentially a standard MIM Sync Sever deployment in an Active Directory Domain. An Admin with a workstation in the same domain with domain credentials.

Prerequisites

It should be pretty obvious by now, but you’ll need;

  • A FIM/MIM Synchronisation Server
    • at least one connected system with a configuration that populates your Metaverse with holograms
  • Download and install the Lithnet MIIS Automation Powershell Module on your FIM/MIM Sync Server
  • The account you will use to connect to the FIM/MIM Sync Server must be in the Administrators Group on the FIM/MIM Sync Server
  • The account you will use to connect to the FIM/MIM Sync Server must be in the FIM/MIM Admins Role Group

Enable the MIM Sync Server for Remote Powershell

In a domain environment as described above this is straight forward. On your FIM/MIM Sync Server we need to enable Powershell Remoting. This is so we can leverage the Lithnet MIIS Automation Powershell module (that is a prerequisite that you’ve already installed right).

On the FIM/MIM Synchronisation Server open Powershell (as Administrator) and execute the command  Enable-PSRemoting -Force 

Test from another server in your network that you can access the MIM Sync Server. I did this from my MIM Service Server.

Establishing a Remote Powershell Session to your FIM/MIM Sync Server

Now you’re ready to start a remote session into your FIM/MIM Sync Server. Take the following snippets and put them into an Administrator Powershell ISE session, modify for your FIM/MIM Sync server name and your Admin username (if you’re not already in a session with that privileged account) and try connecting.

Success we’re connected, a remote session.

Now let’s run a couple of queries using two of the cmdlets from the Lithnet MIIS Automation PS Module. One to get a user and the other to get the MA Stats for the Twitter MA.

Success. Brilliant. Simple.

Troubleshooting

Server User Role Permissions

If you are authenticating with an account without enough permissions for Remote Powershell you’ll get the following message. Access is Denied. Whilst you would expect that putting the user account into the “Remote Management Users” would/should be enough, in my experience you need to have the account you’re connecting with in the Administrators group on the FIM/MIM Server. If there is another method of least privilege please let me know.

 

MIM Sync Permissions

If you aren’t in a FIM/MIM Role for the tasks you are looking to perform, you will get an error similar to that below. You can see I could connect to the MIM Sync Server with Remote Powershell, but could not run the Get-MVObject cmdlet.

If you are in the FIM/MIM Operators Role Group you’d think you could return an object. No. You get an error message like the one below.

When the same account is in the FIM/MIM Admins Role Group, Success.

 

Summary

That is the quick start guide to using Remote Powershell and the Lithnet MIIS Automation Powershell Module to manage your FIM/MIM Sync Server. Automate and Manager away.

You should now think about additional security and restricting what hosts can connect to your FIM/MIM Sync Server using RPS. See Restricting WinRM Hosts here.

 

Follow Darren on Twitter @darrenjrobinson

Using Azure Functions with the Lithnet MIIS Automation Powershell Module to query your Microsoft Identity Manager Metaverse

This is the 2nd blog continuing on from this post which is an introduction to using Azure Functions with the Lithnet FIM/MIM Powershell Modules. If you haven’t read that one please do so to get up to speed before this one as it has more detail around the setup.

Overview

This post details similar functionality to the first post but with integration to the FIM/MIM Synchronisation Server and the FIM/MIM Metaverse rather than the FIM/MIM Service.

The solution is based around an Azure Function that;

  • takes a HTTP WebRequest that contains a payload with the ObjectType, AttributeName and AttributeValue to search for in the Metaverse
  • The Azure Function uses Remote Powershell to call the Lithnet MIIS Automation Powershell Module installed on the FIM/MIM Sync Server
  • The Lithnet Powershell Module takes the query from the Azure Function, executes the query and returns the result to the Azure Function and the requesting client
  • Note: My MIM Infrastructure is all located in Azure so there are configuration steps in this solution to allow access into my Azure environment. If your FIM/MIM infrastructure is elsewhere you’ll need to transpose the appropriate firewall rules for your architecture

Let’s get started.

Prerequisites

The prerequisites for this solution are;

  • An Azure Tenant
  • FIM/MIM Sync Server (as per the diagram above) with data in your Metaverse from a connected directory service (such as Active Directory)
  • I’ll also be using the awesome Lithnet MIIS Powershell Module from here for Microsoft FIM/MIM from Ryan Newington. A fantastic contribution to the FIM/MIM community
    • You’ll need to download and install it on your FIM/MIM Synchronisation Server. This differs from the Lithnet Module from the first post in this series as this one is specific to the Metaverse not the FIM/MIM Service.

Enable Powershell Remoting on the FIM/MIM Sync Server

On the FIM/MIM Sync Server where we will be sending requests from the Function App we need to enable Powershell Remoting. This is so we can leverage the Lithnet MIIS Automation Powershell module (that is a prerequisite to be installed on your FIM/MIM Sync Server).

On the FIM/MIM Synchronisation Server open Powershell (as Administrator) and execute the command  Enable-PSRemoting -Force 

 

Test from another server in your network that you can access the MIM Sync Server. I did this from my MIM Service Server.

 

PSRemote Inbound Security Rule (Azure NSG)

Using Powershell Remote means we need to have an incoming rule into the Azure Network where my MIM Sync Server is located to allow connections from Azure Functions to my MIM Sync Server. Create an Inbound Rule in your Azure Network Security Group for TCP Port 5986 as per the rule below.

 

Create a Self Signed Cert on the FIM/MIM Sync Server

To secure the connection using Remote Powershell we will secure the HTTPS connection with a certificate. This is because the Azure Function is not a member of the domain where your FIM/MIM Sync Server is located. In this example I’m using a self-signed certificate.

In Powershell (as Administrator) on your FIM/MIM Sync Server run the following command where the DNSName is the DNS name of your FIM/MIM Sync that will resolve from Azure Functions to your FIM/MIM Sync server.

New-SelfSignedCertificate -DnsName mymimsyncserver.westus.cloudapp.azure.com -CertStoreLocation Cert:\LocalMachine\My

Create a Remote Powershell HTTPS Listener

Copy the thumbprint from the self-signed certificate above and use it along with the DNS name of your FIM/MIM Sync Server to run the following command in an Administrator command prompt on your FIM/MIM Sync Server.

winrm create winrm/config/Listener?Address=*+Transport=HTTPS @{Hostname=”mymimsyncserver.westus.cloudapp.azure.com”;CertificateThumbp
rint=”536E41D6089F35ABCDEFD8C52BE754EFF0B279B”}

 

Allow Powershell Remote (HTTPS) through your firewall on your FIM/MIM Sync Server

In an Administrator command prompt run the following command to create a new inbound firewall rule for the Remote Powershell session from your Azure Function.

netsh advfirewall firewall add rule name="WinRM-HTTPS" dir=in localport=5986 protocol=TCP action=allow

Check that the new firewall rule was created successfully.

 

Create your HTTP Request Function

Create a new HTTP Trigger Function choosing Powershell as the language. More detailed steps to do this is in the first post in this series here.

Search FIM/MIM Metaverse Function App Script

Here is the base script to get you started. This differs a little from the first blog post example in that I’ve secured the username and password for connection to my MIM Sync Server. Details on how to do that are also linked to in the first blog post.

Also in this example I’m running Remote Powershell to execute the command on the FIM/MIM Sync Server as that is where the Lithnet MIIS Automation Powershell Module is installed and needs to run.

The following script;

  • Takes an HTTP request with Object Type, AttributeName, AttributeValue
  • It uses a Script Block to take the input variables from the HTTP request and perform a a Powershell Remote command (in this example Get-MVObject)
  • Returns the object to the output

Save the function once you’ve added the script (and updated it for your credentials, target FIM/MIM Sync Server etc).

Bring up the Test dialog and give the script some input values in the Request Body that will result in a successful query result from your Metaverse. Select Run. If you’ve done everything correctly you’ll see an object returned from the Metaverse.

Test the Function App

 

Execute the Azure Function from an HTTP Trigger

Now lets try it remotely. Here is a quick Powershell query to the Azure Function using the Powershell Invoke Rest Method using the same input to the Azure Function. And huzzah a returned object.

Summary

This concept provides a framework to allow a plethora of possibilities all possible through a combination of Azure Functions and the Lithnet MIIS Automation PS Module. The Lithnet MIIS PS Module provides all the functionality you get from being on the MIM Sync Server, but now you can retrieve information remotely or trigger functions remotely.

Follow Darren on Twitter @darrenjrobinson

 

 

 

Microsoft Identity Manager Service and Portal Setup Wizard ended prematurely

Last week I was installing the Microsoft Identity Manager Service and Portal on a relatively fresh build of a Windows 2012R2 Server that also included an automated installation of SharePoint Server 2013 w/ SP1.

After going through all the installation configuration options and having the installation start I got the extremely helpful “Setup Wizard ended prematurely” error message.

Having been in this situation previously on other installs (but for different reasons) I knew it was time to kick off the installation again from the command prompt with logging to an installation log file as shown below.

msiexec /i “e:\Service and Portal\Service and Portal.msi” /l*v c:\temp\install.log 

Providing all the same installation configuration options again the install kicked off again and obviously returned the same “Setup Wizard ended prematurely” error message. But this time I had the installation log file to find out why.

Reading through the log file I found the System.Reflection.TargetInvocationException error as highlighted in the screenshot below. A SharePoint issue. The http://localhost could not be found. What? I could access the SharePoint Central Administration site and I could see everything I’d normally expect.

Until I looked in the Alternate Access Mappings.

For whatever reason my only entry was for the MIM Portal and Service URL that I was trying to install.

I updated the Alternate Access Mappings to included an additional mapping for http://localhost as per the screenshot below.

 

Running the installation process again (and re-entering all the setup configuration) and everything went through as it should. Hope this helps by saving someone else an hour or so.

Follow Darren on Twitter @darrenjrobinson