Remotely managing your FIM/MIM Synchronisation Server using Powershell and the Lithnet MIIS Automation Powershell Module

Remotely Administering and Orchestrating Microsoft Identity Manager Sync Server with Lithnet using Remote PowerShell

Background

I’ve been using Ryan’s Lithnet MIIS Automation Powershell Module for a few months now as you’ve likely seen from some of my blog posts.

The module and its installer direct you to install the module on your FIM/MIM Synchronisation Server. This all makes perfect sense as the FIM/MIM Synchronsation Sever is more of your traditional server application. However we are no longer living in that kind of IT world. Consultants, Administrators, Architects, DevOps etc all want the flexibility to use their own workstations, administrative workstations, automation services etc.

So how do we apply that to the FIM/MIM Synchronisation Server ? Well, with the Lithnet MIIS Automation Powershell Module installed on your FIM/MIM Synchronisation Server you can, thanks to the wonderful thing that is Powershell. Thanks Jeffrey Snover.

Overview

In this blog post I’ll detail the quick and easy steps to enable you to remotely administer, orchestrate, report and query your FIM/MIM Sync Server and Metaverse using Powershell and the Lithnet Automation Powershell Module.

The diagram below outlines the topology. Essentially a standard MIM Sync Sever deployment in an Active Directory Domain. An Admin with a workstation in the same domain with domain credentials.

Lithnet Microsoft Identity Manager Remote PowerShell
Lithnet Microsoft Identity Manager Remote PowerShell

Prerequisites

It should be pretty obvious by now, but you’ll need;

  • A FIM/MIM Synchronisation Server
    • at least one connected system with a configuration that populates your Metaverse with holograms
  • Download and install the Lithnet MIIS Automation Powershell Module on your FIM/MIM Sync Server
  • The account you will use to connect to the FIM/MIM Sync Server must be in the Administrators Group on the FIM/MIM Sync Server
  • The account you will use to connect to the FIM/MIM Sync Server must be in the FIM/MIM Admins Role Group

Enable the MIM Sync Server for Remote Powershell

In a domain environment as described above this is straight forward. On your FIM/MIM Sync Server we need to enable Powershell Remoting. This is so we can leverage the Lithnet MIIS Automation Powershell module (that is a prerequisite that you’ve already installed right).

On the FIM/MIM Synchronisation Server open Powershell (as Administrator) and execute the command  Enable-PSRemoting -Force 

Test from another server in your network that you can access the MIM Sync Server. I did this from my MIM Service Server.

Establishing a Remote Powershell Session to your FIM/MIM Sync Server

Now you’re ready to start a remote session into your FIM/MIM Sync Server. Take the following snippets and put them into an Administrator Powershell ISE session, modify for your FIM/MIM Sync server name and your Admin username (if you’re not already in a session with that privileged account) and try connecting.

Success we’re connected, a remote session.

Now let’s run a couple of queries using two of the cmdlets from the Lithnet MIIS Automation PS Module. One to get a user and the other to get the MA Stats for the Twitter MA.

Success. Brilliant. Simple.

Troubleshooting

Server User Role Permissions

If you are authenticating with an account without enough permissions for Remote Powershell you’ll get the following message. Access is Denied. Whilst you would expect that putting the user account into the “Remote Management Users” would/should be enough, in my experience you need to have the account you’re connecting with in the Administrators group on the FIM/MIM Server. If there is another method of least privilege please let me know.

MIM Sync Permissions

If you aren’t in a FIM/MIM Role for the tasks you are looking to perform, you will get an error similar to that below. You can see I could connect to the MIM Sync Server with Remote Powershell, but could not run the Get-MVObject cmdlet.

If you are in the FIM/MIM Operators Role Group you’d think you could return an object. No. You get an error message like the one below.

When the same account is in the FIM/MIM Admins Role Group, Success.

Summary

That is the quick start guide to using Remote Powershell and the Lithnet MIIS Automation Powershell Module to manage your FIM/MIM Sync Server. Automate and Manager away.

You should now think about additional security and restricting what hosts can connect to your FIM/MIM Sync Server using RPS. See Restricting WinRM Hosts here.

 

Follow Darren on Twitter @darrenjrobinson

Using Azure Functions with the Lithnet MIIS Automation Powershell Module to query your Microsoft Identity Manager Metaverse

Solution Arch Azure Function Remote Identity Manager Metaverse Search

This is the 2nd blog continuing on from this post which is an introduction to using Azure Functions with the Lithnet FIM/MIM Powershell Modules. If you haven’t read that one please do so to get up to speed before this one as it has more detail around the setup.

Overview

This post details similar functionality to the first post but with integration to the FIM/MIM Synchronisation Server and the FIM/MIM Metaverse rather than the FIM/MIM Service.

 

Solution Arch Azure Function Remote Identity Manager Metaverse Search
Solution Arch Azure Function Remote Identity Manager Metaverse Search

The solution is based around an Azure Function that;

  • takes a HTTP WebRequest that contains a payload with the ObjectType, AttributeName and AttributeValue to search for in the Metaverse
  • The Azure Function uses Remote Powershell to call the Lithnet MIIS Automation Powershell Module installed on the FIM/MIM Sync Server
  • The Lithnet Powershell Module takes the query from the Azure Function, executes the query and returns the result to the Azure Function and the requesting client
  • Note: My MIM Infrastructure is all located in Azure so there are configuration steps in this solution to allow access into my Azure environment. If your FIM/MIM infrastructure is elsewhere you’ll need to transpose the appropriate firewall rules for your architecture

Let’s get started.

Prerequisites

The prerequisites for this solution are;

  • An Azure Tenant
  • FIM/MIM Sync Server (as per the diagram above) with data in your Metaverse from a connected directory service (such as Active Directory)
  • I’ll also be using the awesome Lithnet MIIS Powershell Module from here for Microsoft FIM/MIM from Ryan Newington. A fantastic contribution to the FIM/MIM community
    • You’ll need to download and install it on your FIM/MIM Synchronisation Server. This differs from the Lithnet Module from the first post in this series as this one is specific to the Metaverse not the FIM/MIM Service.

Enable Powershell Remoting on the FIM/MIM Sync Server

On the FIM/MIM Sync Server where we will be sending requests from the Function App we need to enable Powershell Remoting. This is so we can leverage the Lithnet MIIS Automation Powershell module (that is a prerequisite to be installed on your FIM/MIM Sync Server).

On the FIM/MIM Synchronisation Server open Powershell (as Administrator) and execute the command  Enable-PSRemoting -Force 

Test from another server in your network that you can access the MIM Sync Server. I did this from my MIM Service Server.

PSRemote Inbound Security Rule (Azure NSG)

Using Powershell Remote means we need to have an incoming rule into the Azure Network where my MIM Sync Server is located to allow connections from Azure Functions to my MIM Sync Server. Create an Inbound Rule in your Azure Network Security Group for TCP Port 5986 as per the rule below.

Create a Self Signed Cert on the FIM/MIM Sync Server

To secure the connection using Remote Powershell we will secure the HTTPS connection with a certificate. This is because the Azure Function is not a member of the domain where your FIM/MIM Sync Server is located. In this example I’m using a self-signed certificate.

In Powershell (as Administrator) on your FIM/MIM Sync Server run the following command where the DNSName is the DNS name of your FIM/MIM Sync that will resolve from Azure Functions to your FIM/MIM Sync server.

New-SelfSignedCertificate -DnsName mymimsyncserver.westus.cloudapp.azure.com -CertStoreLocation Cert:\LocalMachine\My

Create a Remote Powershell HTTPS Listener

Copy the thumbprint from the self-signed certificate above and use it along with the DNS name of your FIM/MIM Sync Server to run the following command in an Administrator command prompt on your FIM/MIM Sync Server.

winrm create winrm/config/Listener?Address=*+Transport=HTTPS @{Hostname=”mymimsyncserver.westus.cloudapp.azure.com”;CertificateThumbp
rint=”536E41D6089F35ABCDEFD8C52BE754EFF0B279B”}

Allow Powershell Remote (HTTPS) through your firewall on your FIM/MIM Sync Server

In an Administrator command prompt run the following command to create a new inbound firewall rule for the Remote Powershell session from your Azure Function.

netsh advfirewall firewall add rule name="WinRM-HTTPS" dir=in localport=5986 protocol=TCP action=allow

Check that the new firewall rule was created successfully.

Create your HTTP Request Function

Create a new HTTP Trigger Function choosing Powershell as the language. More detailed steps to do this is in the first post in this series here.

Search FIM/MIM Metaverse Function App Script

Here is the base script to get you started. This differs a little from the first blog post example in that I’ve secured the username and password for connection to my MIM Sync Server. Details on how to do that are also linked to in the first blog post.

Also in this example I’m running Remote Powershell to execute the command on the FIM/MIM Sync Server as that is where the Lithnet MIIS Automation Powershell Module is installed and needs to run.

The following script;

  • Takes an HTTP request with Object Type, AttributeName, AttributeValue
  • It uses a Script Block to take the input variables from the HTTP request and perform a a Powershell Remote command (in this example Get-MVObject)
  • Returns the object to the output

Save the function once you’ve added the script (and updated it for your credentials, target FIM/MIM Sync Server etc).

Bring up the Test dialog and give the script some input values in the Request Body that will result in a successful query result from your Metaverse. Select Run. If you’ve done everything correctly you’ll see an object returned from the Metaverse.

Test the Function App

Execute the Azure Function from an HTTP Trigger

Now lets try it remotely. Here is a quick Powershell query to the Azure Function using the Powershell Invoke Rest Method using the same input to the Azure Function. And huzzah a returned object.

Summary

This concept provides a framework to allow a plethora of possibilities all possible through a combination of Azure Functions and the Lithnet MIIS Automation PS Module. The Lithnet MIIS PS Module provides all the functionality you get from being on the MIM Sync Server, but now you can retrieve information remotely or trigger functions remotely.

Follow Darren on Twitter @darrenjrobinson

 

 

 

Microsoft Identity Manager Service and Portal Setup Wizard ended prematurely

Last week I was installing the Microsoft Identity Manager Service and Portal on a relatively fresh build of a Windows 2012R2 Server that also included an automated installation of SharePoint Server 2013 w/ SP1.

After going through all the installation configuration options and having the installation start I got the extremely helpful “Setup Wizard ended prematurely” error message.

Install Error.png

Having been in this situation previously on other installs (but for different reasons) I knew it was time to kick off the installation again from the command prompt with logging to an installation log file as shown below.

msiexec /i “e:\Service and Portal\Service and Portal.msi” /l*v c:\temp\install.log 

Install with Logging.png

Providing all the same installation configuration options again the install kicked off again and obviously returned the same “Setup Wizard ended prematurely” error message. But this time I had the installation log file to find out why.

Reading through the log file I found the System.Reflection.TargetInvocationException error as highlighted in the screenshot below. A SharePoint issue. The http://localhost could not be found. What? I could access the SharePoint Central Administration site and I could see everything I’d normally expect.

InstallLog.png

Until I looked in the Alternate Access Mappings.

Alternate Access Mappings.png

For whatever reason my only entry was for the MIM Portal and Service URL that I was trying to install.

Internal Host - Wrong.png

I updated the Alternate Access Mappings to included an additional mapping for http://localhost as per the screenshot below.

Updated Alternate Access Mapping.png

Updated Alternate Access Mapping2.png

Running the installation process again (and re-entering all the setup configuration) and everything went through as it should. Hope this helps by saving someone else an hour or so.

Follow Darren on Twitter @darrenjrobinson

Get Users/Groups/Objects from Microsoft/Forefront Identity Manager with Azure Functions and the Lithnet Resource Management Powershell Module

Leveraging Azure Functions to query Microsoft Identity Manager

Introduction

As an Identity Management consultant if I had a $1 for every time I’ve been asked “what is user x’s current status in IDAM”, “is user x active?”, “does user x have an account in y?”, “what is user x’s primary email address?”, particularly after Go Live of an IDAM solution my holidays would be a lot more exotic.

From a Service Desk perspective IDAM implementations are often a black box in the middle of the network that for the most part do what they were designed and implemented to do. However as soon as something doesn’t look normal for a user the Service Desk are inclined to point their finger at that black box (IDAM Solution). And the “what is the current value of ..”, “does user x ..” type questions start flying.

What if we could give the Service Desk a simple query interface into FIM/MIM without needing to give them access to another complicated application?

This is the first (of potentially a series) blog post on leveraging community libraries and Azure PaaS services to provide visibility of FIM/MIM information. This first post really just introduces the concept with a working example in an easy way to understand and replicate. It is not intended for production implementation without additional security and optimisation. 

Overview

The following graphic shows the concept of using Azure Functions to take requests from a client (web app, powershell, some other script) query the FIM/MIM Service and return the result. This post details the setup and configuration for the section in the yellow shaded box with the process outlined in the numbers 1, 2 & 3. This post assumes you already have your FIM/MIM implementation setup and configured according to your connected integrated applications/services such as Active Directory. In my example my connected datasource is actually Twitter.

Prerequisites

The prerequisites for this solution are;

 

Creating your Azure App Service

First up you’ll need to create an Azure App Service in your Azure Tenant. To keep everything logically structured for this example I created an Azure App Service in the same resource group that contains my MIM IaaS infrastructure (MIM Sync Server, MIM Service Server, SQL Server, AD Domain Controller etc).

In the Azure Marketplace select New (+) and search for Function App. Select the Function App item from the results and select Create.

Give your Azure App Service a name, choose the Resource Group where you want to locate it. Choose Dynamic for the Hosting Plan. This means you don’t have to worry about resource management for your Web App and you only pay for execution time which unless you put this into production and have gone crazy with it your costs should be zero as they will (should) be well under the free grant tier.  Put the application in the appropriate location such as close to your FIM/MIM resources that it’ll be interacting with and select Create.

Now that you have your Azure App Service setup, you need to create your Azure Function.

Create an HTTP Trigger Powershell Function App

In the Azure Portal locate your App Services Blade and select the Function App created in the steps above. Mine was named MIMMetaverseSearch in the example above. Select PowerShell as the Language and HttpTrigger-Powershell as the Function type.

Give your Function a name. I’ve kept it simple in this example and named it the same as my App Service Plan. Select Create.

Adding the Lithnet Powershell Module into your Function App.

As you’d expect the Powershell Function App by default only has a handful of core Powershell Modules. As we’re using something pretty specific we’ll need put the module into our Function App so we can load it and use the library.

Download and save the Lithnet Resource Management Powershell Module to your local machine. Something like the Powershell command below will do that.

Next follow this great blog post here from Tao to upload the Lithnet RMA PS Module you downloaded earlier into your function directory. I used WinSCP as my FTP client as I’ve shown below to upload the Lithnet RMA PS Module.

FTP to the host for your App Service and navigate to the /site/wwwroot/

Create a bin folder and upload via FTP the Lithnet RMA PS Module.

Using Kudu navigate to the path and version of the Lithnet RMA PS Module.
I’m using v1.0.6088 and my app is named MIMMetaverseSearch so MY path is D:\home\site\wwwroot\MIMMetaverseSearch\bin\LithnetRMA\1.0.6088

Note: the Lithnet RMA PS Module is 64-bit so you’ll need to configure your Web App for 64-bit as per the info in the same blog you followed to upload the module here.

Test loading the Lithnet RMA PS Module in your Function App

In your Function App select </> Develop. Remove the sample script and in your first line import the Lithnet RMA PS Module using the path from the previous step. Then, to check that it loads add a line that references a cmdlet in the module. I used Help Get-Resource. Select Save then Run.

If you’ve done everything correct when you select Run and look at the Logs you’ll see the module was loaded and the Help Get-Resource command was run in the Logs.

Allow your Function App to access your FIM/MIM Service Server

Even though you have logically placed your Function App in the same Resource Group (if you did it like I have) you’ll need to actually allow the Function App that is running in a shared PaaS environment to connect to your FIM/MIM Service Server.

Create an inbound rule in your Network Security Group to allow access to your FIM/MIM Service Server. The example below isn’t as secure as it could (and should) be as it allows access from anywhere. You should restrict the source of the request(s) accordingly. I’m just showing how to quickly get a working example. TCP Port 5725 is required to access your MIM Service Server. Enter the details as per below and select Ok.

Using an Azure Function to query FIM/MIM Service

Note: Again, this is an example to quickly show the concept. In the script below your credentials are in the script in clear text (and of course those below are not valid). For anything other than validating the concept you must protect your credentials. A great example is available here in Tao’s post.

The PS Azure Function gets the incoming request and converts it from JSON. In my request which you’ll see in the next step I’m passing in “displayName” and “objectType”.

In this example I’m using Get-Resource from the Lithnet RMA to get an object from the FIM/MIM Service. First you need to open a connection to the FIM/MIM Service Server. On my Azure IaaS MIM Service Server I’ve configured a DNS name so you can see I’m using that name in line 17 to connect to it using the unsecured credentials from earlier in the script. If you haven’t set up a DNS name for your FIM/MIM Service Server you can use the Public IP Address instead.

Line 20 queries for the ObjectType and DisplayName passed into the Function (see calling the Function in the next step) and returns the response in line 22. Again this is just an example. There is no error checking, validation or anything. I’m just introducing the concept in this post.

Testing your Function App

Now that you have the function script saved, you can test it from the Function App itself. Select Test from the options up in the right from your function. Change the Request Body for what the Function is expecting. In my case displayname and objectType. Select Run and in the Logs if you’ve got everything configured correctly (like inbound network rules, DNS name, your FIM/MIM Service Server is online, your query is for a valid resource) you should see an object returned.

Calling the Function App from a Client

Now that we have our Function App all setup and configured (and tested in the Function App) let’s send a request to the Azure Function using the Powershell Invoke-RestMethod function. The following call I did from my laptop. It is important to note that there is no authN in this example and the function app will be using whatever credentials you gave it to execute the request. In a deployed solution you’ll need to scope who can make the requests, limit on the inbound network rules who can submit requests and of course further protect the account credentials used to connect to your FIM/MIM Service Server.

Successful Response

The following screenshot shows calling the Function App and getting the responding object. Success. In a couple of lines I created a hashtable for the request, converted it to JSON and submitted it and got a response. How powerful is that!?

Summary

Using the awesome Lithnet Resource Management PowerShell Module with Azure Functions it is pretty quick and flexible to access a wealth of information we may want to expose for business benefit.

Now if only there was an affiliation program for Azure Functions that could deposit funds for each IDAM request to an Azure Functions App into my holiday fund.

Stay tuned for more posts on taking this concept to the next level.

Follow Darren on Twitter @darrenjrobinson

Office365 Licensing Management Agent for Microsoft Identity Manager

Licensing for Office365 has always been a moving target for enterprise customers. Over the years I’ve implemented a plethora of solutions to keep licensing consistent with entitlement logic. For some customers this is as simple as everyone gets say, an E3 license. For other institutions there are often a mix of ‘E’ and ‘K’ licenses depending on EmployeeType.

Using the Granfeldt PowerShell Management Agent to import Office365 Licensing info

In this blog post I detail how I’m using Søren Granfeldt’s extremely versatile PowerShell Management Agent yet again. This time to import Office365 licensing information into Microsoft Identity Manager.

I’m bringing in the licenses associated with users as attributes on the user account. I’m also bringing in the licenses from the tenant as their own ObjectType into the Metaverse. This includes the information about each license such as how many licenses have been purchased, how many licenses have been issued etc.

Overview

I’m not showing assigning licenses. In the schema I have included the LicensesToAdd and LicensesToRemove attributes. Check out my Adding/Removing User Office365 Licences using PowerShell and the Azure AD Graph RestAPI post to see how to assign and remove licenses using Powershell. From that you can workout your logic to implement an Export flow to manage Office365 licenses.

Getting Started with the Granfeldt PowerShell Management Agent

If you don’t already have it, what are you waiting for. Go get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

Three items I had to work out that I’ll save you the pain of are;

  • You must have a Password.ps1 file. Even though we’re not doing password management on this MA, the PS MA configuration requires a file for this field. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present
  • The credentials you give the MA to run this MA are the credentials for the account that has permissions to the Office365 Tenant. Just a normal account is enough to enumerate it, but you’ll need additional permissions to assign/remove licenses.
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~2\2010\SYNCHR~1\EXTENS~2\O365Li~1

Schema.ps1

My Schema is based around the core Office365 Licenses function. You’ll need to create a number of corresponding attributes in the Metaverse Schema on the Person ObjectType to flow the attributes into. You will also need to create a new ObjectType in the Metaverse for the O365 Licenses. I named mine LicensePlans. Use the Schema info below for the attributes that will be imported and the attribute object types to make sure what you create in the Metaverse aligns, so you can import the values. Note the attributes that are multi-valued.

Import.ps1

The logic which the Import.ps1 implements I’m not going to document here as this post goes into all the details Enumerating all Users/Groups/Contacts in an Azure tenant using PowerShell and the Azure Graph API ‘odata.nextLink’ paging function

Password Script (password.ps1)

Empty as not implemented

Export.ps1

Empty as not implemented

Management Agent Configuration

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using an Office 365 account to connect to Office365 and import the user and license data.

O365MA-2.png

O365MA-3.png

As per the Schema script earlier in this post I’m bringing user licensing metadata as well as the Office365 Tenant Licenses info.

O365MA-4.png

Attributes to bring through aligned with what is specified in the Schema file.

O365MA-5.png

Flow through the attributes to the attributes I created in the Metaverse on the Person ObjectType and to the new ObjectType LicensePlans.

O365MA-6.png

Wiring it up

To finish it up you’ll need to do the usual tasks of creating run profiles, staging the connector space from Office365 and importing into the Metaverse.

Enjoy.

Follow Darren on Twitter @darrenjrobinson

A Twitter Management Agent for Microsoft Identity Manager

In the last couple of weeks I’ve been evaluating a number of different approaches/concepts for some upcoming MIM development projects. Some of these I’ve blogged about already.

Having an Identity Manager Metaverse with identity data is a key dependency to being able to validate ideas and concepts. So what’s a good source of some interesting and varied identity data with string, integer, reference, and boolean attributes? Twitter? Yeah why not. There’s an API. Should be pretty quick to get some sample data right?

In this blog post I’m going to give an overview of creating a PowerShell Management Agent to consume Twitter identities and their data into Microsoft Identity Manager. I’ll cover;

  • Obtaining Twitter user data from Twitter using Powershell and the Twitter RestAPI
  • Using Søren’s Powershell Management Agent to import Twitter user data obtained via the RestAPI
  • Manipulating the Twitter data into the MIM Synchronisation Engine

Twitter Data

Here’s an overview of my approach/rationale of what data I was looking for and how I got it from Twitter;

  • I don’t need real-time data. Just identity data
  • I need data of all different data-types
  • I need data with all the randomness that identity data often contains
  • I created a standalone script that took a seed Twitter identity (one of my accounts) and;
    • obtained the Twitter account info including the list of the Twitter accounts it followed
    • the Twitter accounts that follows it
  • The standalone script uses the Twitter RestAPI to obtain the data and respected the service and rate-limits
  • To make the Twitter API calls easy I leveraged the awesome InvokeTwitterAPI Powershell Module from Shannon Conley & Mehmet Kaya available here https://github.com/MeshkDevs/InvokeTwitterAPIs . I notice that there is an updated version from Marc R Kellerman available here https://www.powershellgallery.com/packages/InvokeTwitterAPIs/2.1/Content/InvokeTwitterAPIs.psm1 that was released after I had done most of my work. Notably it supports having multiple OAuth keys and the ratelimit restrictions. The details below leverage this updated version.

Here is what a sample of some of the data looks like in the Metaverse.

DataSample1.PNG

Pre-requisites

You need to enable your Twitter Account for API access. Follow the details here 

Getting the Seed Twitter account info

By now you should have downloaded the Twitter PowerShell API Modules and installed them. If you haven’t get WMF5 installed and run the install-module command as shown below in Line 1.

Modify the script below to give the;

  • Seed Twitter Account you want to bring in the Friends and Followers for as users into the MIM Metaverse
  • The API keys asscoiated with your Twitter account(s) you’re going to use to query the Twitter API
  • The directory you want to dump the account info out to

….. and let it loose.

Now we have two XML files with a whole bunch of Twitter accounts and their metadata. There is an almost certainty that the seed account you’ve used is both followed by twitter accounts that you also follow. We’re going to need to remove the duplicates so that when we import the Twitter accounts into MIM we don’t have duplicates.

Basic, basic script to read in both files and spit out the unique Twitter Accounts is shown below.

Using the Granfeldt PowerShell Management Agent to import Twitter Identities

Consuming data into the MIM Sync Engine obtained via PowerShell is quick and simple utilising Søren Granfeldt’s extremely versatile PowerShell Management Agent. I’m just going to cover importing the data from the XML file we generated above.

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

A few items of note are;

  • You must have a Password.ps1 file. Even though we’re not doing password management on this MA, the PS MA configuration requires a file for this field. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present
  • Same for an Export.ps1 file. I’m not doing any exports on the MA, but an export script must be present.
  • The credentials you give the MA to run this MA are irrelevant as they aren’t used as part of the import as I’m bringing in data from files generated via separate PowerShell scripts
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~2\2010\SYNCHR~1\EXTENS~2\Twitter

Import Twitter Users into Microsoft Identity Manager

Using the guidance above on the Granfeldt PSMA here are the two key scripts for the Twitter MA.

The Schema Script to expose the core Twitter identity attributes.

Schema Script

Import Script

The Import Script that takes the rationalised XML file created earlier from the friends and followers queries and populates the connector space.

Password Script

Required by the PSMA but not used as detailed earlier

Export Script

Required by the PSMA but not used as detailed earlier

Creating the Management Agent

Path to the Schema Script in 8.3 format as detailed earlier.

MA1

Path to the Import, Export and Password scripts also in 8.3 format.

MA2

Select the attributes you want to bring in to the connector space.

MA3

Any Join logic, and a Projection Rule.

MA4

Import Flow Rules to bring in the Tweeters.

MA5

Create your Run Profiles, perform a Stage and Full Sync and BAM; Tweeters in the Metaverse. Real Word Identity Data Ahoy. Exception testing commences now.

Follow Darren on Twitter @darrenjrobinson

Simple reporting from the FIM/MIM Metaverse to PowerBI using the Lithnet FIM/MIM Sync Service PowerShell Module

I have a customer that is looking to report on FIM/MIM identity information. The reports they are looking for aren’t overly complex and don’t necessarily justify the need the full FIM/MIM reporting infrastructure. So I spent a few hours over a couple of days looking at alternatives. In this blog post I give an overview of using the awesome Lithnet FIM/MIM Sync Service PowerShell Module recently released from Ryan Newington to do basic reporting on the Microsoft (Forefront) Identity Manager Metaverse into PowerBI.

I’ll briefly show how to leverage the Lithnet FIM/MIM Sync Service PowerShell Module to extract Person objects and their metadata (based on a search filter criteria) from the MIM/FIM Metaverse and output to a file for PowerBI.

I cover;

  • Building a query
  • Executing the query
  • Filtering the results for output to a file (CSV)
  • Importing to PowerBI as a dataset, creating a report showing results in a Dashboard

First up you’ll need to download and install the module from https://github.com/lithnet/miis-powershell

Using the FIM/MIM Sync Service PowerShell Module to query the Metaverse

What operators you can choose on your attribute types (boolean, string, integer, reference etc) in the Metaverse Search function in the Synchronisation Service Manager you can also perform using the Lithnet FIM/MIM Sync Service PowerShell Module.

By creating a search with multiple criteria in the Metaverse Search you can filter the results from the Metaverse.

MVQuery1.jpg

As shown below you can see that we get 302 results.

MVQuery2.jpg

So let’s import the Lithnet FIM/MIM Sync Service PowerShell Module, create a filter execute it and look at the results. As you’d expect we get the same result. Excellent.

PSQuery1.jpg

Remember that using this PowerShell automation module, the backend is still the WMI interface behind the Synchronisation Service Manager. This means you can’t for example create a query filter using “greater than/less than” if you can’t do it in the UI.

Take my Twitter FriendsCount attribute of type Number/Integer as an example.

MVObject1.jpg

I can’t create a query filter that would return results where FriendsCount > 20,000. I can only use the IsPresent, IsNotPresent and Equals.

QueryOperators1.jpg

On a sidenote the PowerShell error message will give you a hint at what operators you can use as shown below.

PSError.jpg

However, if you try and use StartsWith for an Integer attribute the search will execute but just return no results. My tip then is define your query in the Metaverse Search GUI and when you get what results you want/expect, create the equivalent query in PowerShell and validate you get the same number of results.

Final note on query filters. Multiple criteria are an AND operation filter, NOT OR.

QueryAndNotOr.jpg

Let’s do something with the results

Now that we have a query sorted let’s do something with the results. The result set is the full attribute list and values for each associated object that matched our query from the Metaverse. That’s way more info than what I and probably you need as well. So iterate through the results, pull out the attribute values that we want to do something with and export them as a CSV file.

Results1.jpg

What to do with the output ?

For this overview I’ve just chosen the local file (CSV) that I exported as part of the script as the input dataset in PowerBI. https://app.powerbi.com

On the right hand side I’ve chosen the columns that were exported to the CSV and they appear in the main window.

PBi1.jpg

Click Pin to Live Page. You’ll be prompted to save the report first so do that then I choose New Dashboard for the report. Click Pin live.

PBi2.jpg

I can then refine and get some visual reports quickly using text based queries using keywords from the dataset columns. Like Top 10 by number of friends from the dataset.

PBi3.jpg

Create a couple of queries and pin them to the Dashboard and the data comes to life.

PBi4.jpg

Summary

The Lithnet FIM/MIM Sync Service PowerShell Module provides a really easy way to expose information from the Metaverse that may satisfy many reporting and other requirements. Taking the concept further it wouldn’t be too complex to export the data to an Azure SQL DB on a schedule and have the results dynamically update on a PowerBI Dashboard.
The concept of exporting data for reporting is just one practical example using the tools. Huge thanks to Ryan for creating the Lithnet tools and publishing to the community. Keep in mind the tools disclaimer too.

Here is the sample PowerShell.

Follow Darren on Twitter @darrenjrobinson

 

Exception from HRESULT 0x80230729 creating a new FIM/MIM Management Agent

Another day, another piece of FIM/MIM experimentation. I had built a fresh MIM 2016 environment in Azure to test a few scenarios out. That all went quick and seamlessly thanks to some great templates and a few scripts. Until I came to create the management agent (the purpose of today’s experimentation).

It didn’t matter if I tried to Create a New Management Agent or Import the Management Agent. I just got “Exception from HRESULT 0x80230729”. The common element however was that the Management Agent I was creating was based off a 3rd party MA based on Microsoft’s Extensible Connectivity Management Agent (ECMA). Specifically I was using Soren Granfeldts PowerShell MA.

HResult 0x80230729

Now I’ve used this MA extensively and not had a problem previously.

So I retraced my steps, clean build, pre-requisites etc. All good. I then tried creating an MA from the out of the box connectors. That worked. I successfully created an Active Directory Management Agent.

In the Windows Application Log I found the following from when I was trying to create the PSMA. A little more to go on from that information.

AppLog

The link in the error message provides some info https://msdn.microsoft.com/en-us/library/dd409252(VS.100).aspx but it is a generic .NET article. Having experience with MIIS/ILM/FIM/MIM I figured the SyncEngine WebServices Config file would be appropriate place for the information provided in the MSDN link to go.

The Fix

The miiserver.exe.config file located in the default installation directory C:\Program Files\Microsoft Forefront Identity Manager\2010\Synchronization Service\Bin is what you need to edit.

Near the end of the misserver.exe.config file find the <runtime> section. Insert the line <loadFromRemoteSources enabled=”true”/> as shown below.

RemoteSources

Restart the Forefront Identity Manger Server Service from the Services Control Panel and you’re back in action.

Management Agent created and back to the task at hand. Happy days.

MACreated

Follow Darren on Twitter @darrenjrobinson

Consuming CSV files from an Exchange Mailbox via Exchange Web Services and FIM/MIM 2016 using the Granfeldt PowerShell MA

This solution on first look is quite random. A management agent that consumes a flat file (comma separated file) isn’t ground breaking, but when the twist is that the CSV file is in an email in an Exchange Inbox, it’s quite a different scenario.

Background

My customer uses a Cloud Service for their recruitment processes. The cloud service does have a SOAP API that I could potentially develop a FIM/MIM solution for using the Microsoft Web Services Management Agent, however my customer does not have API access to their tenant, the vendor isn’t overly responsive and I need a solution in days not weeks.

On the upside, my customer can quickly create reports in the SaaS Portal, and schedule them to be delivered (via CSV/Excel) to an email address. So, what if I was able to integrate FIM/MIM to the inbox that receives the emails with attached reports that contain the information I require and process it accordingly? This blog post is that solution.

Overview

Once a day there is a scheduled process that generates a report (CSV) of new staff from a SaaS provider. That CSV is emailed to an Inbox we created to receive these reports. Using the Granfeldt PowerShell Management Agent I created a solution that;

  • Connects to the specified Exchange Mailbox using Exchange Web Services
    • Enumerates the inbox looking for emails with attachments
    • Validates the emails with attachments by looking for the sender and attachment type we are expecting
    • Extracts the attachment to a file share
    • Moves all messages with attachments to a Processed subfolder
  • Processes the most recent report attachment (CSV) (in case the MA hasn’t run for few days or the reports start coming more than once a day) or if there is no new email message with attachment in the inbox, processes the most recent attachment we previously put in the file share
    • Each report is cumulative so the MA logic stays simple
  • Imports to MIM the new staff that are due to start in the next 7 days (to allow for all access to be setup prior to their first day of employment) and kicks off the MIM Provisioning processes
    • Triggers entitlements and access through the system accordingly (not covered in this post, but includes provisioning of mailbox, home directory, group memberships etc)

Notes:

  • The MIM Synchronisation Service Account will need access permissions to save files into the File Share
  • The MIM Server and this PSMA will require the Microsoft EWS 2.2 API to be installed on the MIM Synchronisation Server. It is available from here https://www.microsoft.com/en-us/download/details.aspx?id=42951

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

Three items I had to work out that I’ll save you the pain of are;

  • You must have a Password.ps1 and Export.ps1 file. Even though we’re not doing password management, or exporting back to the SaaS Provider on this MA, the PS MA configuration requires a file for these fields. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present.
  • The credentials you give the MA to run the scripts as, needs to be in the format of just ‘accountname’ NOT ‘domain\accountname’. I’m using the AD Account associated with the Exchange Mailbox that receives the emails with the CSV reports.
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~4\2010\SYNCHR~1\EXTENS~2\PageUp

Schema.ps1

My schema is essentially the columns that are in the CSV report that I’m importing.

Password Script (password.ps1)

Empty as described above

Import.ps1

Connect to the Exchange Mailbox, find messages from the defined user sending them where the attachment is of the expected naming and format. Extract the CSV file to a File Share. Move emails with attachments to a processed folder. Parse the CSV, perform some logic on the data and import objects and values for new employees.

Export.ps1

Empty as we’re not writing anything back to the SaaS provider.

Wiring it all together

In order to wire the functionality all together there are the usual number of configuration steps to be completed. Below I’ve shown a number of the key points associated with making it all work. This is all Synchronisation Engine MA configuration tasks. Basically create the PS MA, import attributes from the PS MA, create your MA Run Profiles and let it loose.

Management Agent Configuration

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using the same service account as my AD MA.

Password and Export scripts must be specified but as we’re not doing password management or exporting they’re empty as detailed above.

If your schema.ps1 file is formatted correctly, you can select your attributes/columns that will be coming in from the CSV file.

My join rule is simple. StaffID to AccountName in the MetaVerse.

My import flows are direct flows with a Boolean flag to kick off a bunch of declarative rules out of the Portal.

Summary

Thinking outside of the box and using the Granfeldt PowerShell MA I was able to quickly consume a CSV file from an Exchange Inbox to kick off the provisioning process.

Follow Darren on Twitter @darrenjrobinson

Managing SharePoint Online (SPO) User Profiles with FIM/MIM 2016 and the Granfeldt PowerShell MA

Forefront / Microsoft Identity Manager does not come with an out-of-the-box management agent for managing SharePoint Online.

Whilst the DirSync/AADConnect solution will allow you to synchronise attributes from your On Premise Active Directory to AzureAD, SharePoint only leverages a handful of them. It then has its own set of attributes that it leverages. Many are similarly named to the standard Azure AD attributes but with the SPS- prefix.

For example, here is a list of SPO attributes and a couple of references to associated Azure AD attributes;

  • UserProfile_GUID
  • SID
  • SPS-PhoneticFirstName
  • SPS-PhoneticLastName
  • SPS-PhoneticDisplayName
  • SPS-JobTitle
  • SPS-Department
  • AboutMe
  • PersonalSpace
  • PictureURL
  • UserName
  • QuickLinks
  • WebSite
  • PublicSiteRedirect
  • SPS-Dotted-line
  • SPS-Peers
  • SPS-Responsibility
  • SPS-SipAddress
  • SPS-MySiteUpgrade
  • SPS-ProxyAddresses
  • SPS-HireDate
  • SPS-DisplayOrder
  • SPS-ClaimID
  • SPS-ClaimProviderID
  • SPS-ClaimProviderType
  • SPS-SavedAccountName
  • SPS-SavedSID
  • SPS-ResourceSID
  • SPS-ResourceAccountName
  • SPS-ObjectExists
  • SPS-MasterAccountName
  • SPS-PersonalSiteCapabilities
  • SPS-UserPrincipalName
  • SPS-O15FirstRunExperience
  • SPS-PersonalSiteInstantiationState
  • SPS-PersonalSiteFirstCreationTime
  • SPS-PersonalSiteLastCreationTime
  • SPS-PersonalSiteNumberOfRetries
  • SPS-PersonalSiteFirstCreationError
  • SPS-DistinguishedName
  • SPS-SourceObjectDN
  • SPS-FeedIdentifier
  • SPS-Location
  • Certifications
  • SPS-Skills
  • SPS-PastProjects
  • SPS-School
  • SPS-Birthday
  • SPS-Interests
  • SPS-StatusNotes
  • SPS-HashTags
  • SPS-PictureTimestamp
  • SPS-PicturePlaceholderState
  • SPS-PrivacyPeople
  • SPS-PrivacyActivity
  • SPS-PictureExchangeSyncState
  • SPS-TimeZone
  • SPS-EmailOptin
  • OfficeGraphEnabled
  • SPS-UserType
  • SPS-HideFromAddressLists
  • SPS-RecipientTypeDetails
  • DelveFlags
  • msOnline-ObjectId
  • SPS-PointPublishingUrl
  • SPS-TenantInstanceId

My customer has AADConnect in place that is synchronising their On Premise AD to Office 365. They also have a MIM 2016 instance that is managing user provisioning and lifecycle management. I’ll be using that MIM 2016 instance to manage SPO User Profile Attributes.

The remainder of this blog post describes the PS MA I’ve developed to manage the SPO attributes to allow their SPO Online Forms etc to leverage business and organisation user metadata.

Using the Granfeldt PowerShell Management Agent to manage SharePoint Online User Profiles

In this blog post I detail how you can synchronise user attributes from your On Premise Active Directory to an associated users SharePoint Online user profile utilising Søren Granfeldt’s extremely versatile PowerShell Management Agent. Provisioning and licensing of users for SPO is performed in parallel by the DirSync/AADConnect solution. This solution just provides attribute synchronisation to SPO User Profile attributes.

Overview

In this solution I’m managing the attributes that are pertinent to the customer. If you need an additional attribute or you have created custom attributes it is easy enough to extent.

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

Three items I had to work out that I’ll save you the pain of are;

  • You must have a Password.ps1 file. Even though we’re not doing password management on this MA, the PS MA configuration requires a file for this field. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present
  • The credentials you give the MA to run this MA are the credentials for the account that has permissions to manage SharePoint Online User Profiles. More detail on that further below.
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~4\2010\SYNCHR~1\EXTENS~2\SPO

Managing SPO User Profiles

In order to use this working example there are a couple of items to note;

  • At the top of the Import and Export scripts you’ll need to enter your SPO Tenant Admin URL. If your tenant URL is ‘CORP.sharepoint.com’ then at the top of the scripts enter ‘CORP-admin.sharepoint.com’. The Import script will work with corp.sharepoint.com but the export won’t.
  • Give the account you’re using to connect to SPO via your MIM permissions to manage/update SPO User Profiles

Schema.ps1

As mentioned above I’m only syncing attributes pertinent to my customers’ requirements. That said I’ve selected a number of attributes that are potentials for future requirements.

Password Script (password.ps1)

Empty as described above

Import.ps1

A key part of the import script is connecting to SPO and accessing the full User Profile. In order to do this, you will need to install the SharePoint Online Client Components SDK. It’s available for download here https://www.microsoft.com/en-us/download/details.aspx?id=42038

The import script then imports two libraries that give us access to the SPO User Profiles.

Import-Module
‘C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.UserProfiles.dll’

Import-Module
‘C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll’

Import values for attributes defined in the schema.

Export.ps1

The business part of the MA. Basically enough to take attribute value changes from the MV to the SPO MA and export them to SPO. In the example script below I’m only exporting three attributes. Add as many as you need.

Wiring it all together

In order to wire the functionality together, I’m doing it just using the Sync Engine MA configuration as we’re relying on AADConnect to create the users in Office365, and we’re just flowing through attribute values.

Basically, create the PS MA, create your MA Run Profiles, import users and attributes from the PS MA, validate your joins and Export to update SPO attributes as per your flow rules.

Management Agent Configuration

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using the Office 365 account we gave permissions to manage user profiles in SPO earlier.

Password script must be specified but as we’re not doing password management it’s empty as detailed above.

If your schema.ps1 file is formatted correctly you can select your attributes.

I have a few join rules. In the pre-prod environment though I’m joining on WorkEmail => mail.

My import flow is just bringing back in users mobile numbers that users are able to modify in SPO. I’m exporting Title, Location and Department to SPO.

Summary

Using the Granfeldt PowerShell MA it was very easy to manage user SharePoint Online User Profile attributes.

Follow Darren on Twitter @darrenjrobinson