How to access Microsoft Identity Manager Hybrid Report data using PowerShell, Graph API and oAuth2

Hybrid Reporting is a great little feature of Microsoft Identity Manager. A small agent installed on the MIM Sync Server will send reporting data to Azure for MIM SSPR and MIM Group activities. See how to install and configure it here.

But what if you want to get the reporting data without going to the Azure Portal and looking at the Audit Reports ? Enter the Azure AD Reports and Events REST API that is currently in preview.  It took me a couple of cracks and getting this working, because documentation is a little vague especially when accessing it via PowerShell and oAuth2. So I’ve written it up and hope it helps for anyone else looking to go down this route.

Gotchas

Accessing the Reports via the API has a couple of caveats that I had to work through:

  • Having the correct permissions to access the report data. Pretty much everything you read tells you that you need to be a Global Admin. Once I had my oAuth tokens I messed around a little and a was able to also get the following from back from the API when purposely using an identity that didn’t have the right permissions. The key piece is “Api request is not from global admin or security admin or security reader role”. I authorized the WebApp using an account that is in the Security Reader Role, and can successfully access the report data.

  • Reading the documentation here on MSDN I incorrectly assumed each category was the report name. Only when I called the “https://graph.windows.net//$metadata?api-version=beta”  and looked at the list of reports I noticed each report was plural.The three that I wanted to access (and report on) are obviously the MIM Hybrid Reports;
"Name":  "mimSsgmGroupActivityEvents",
"Name":  "mimSsprActivityEvents",
"Name":  "mimSsprRegistrationActivityEvents",

Here is the full list of Reports available as of 24 May 2017.

{
    "Name":  "b2cAuthenticationCountSummary",
    "LicenseRequired":  "False"
}
{
    "Name":  "b2cMfaRequestCount",
    "LicenseRequired":  "False"
}
{
    "Name":  "b2cMfaRequestEvent",
    "LicenseRequired":  "False"
}
{
    "Name":  "b2cAuthenticationEvent",
    "LicenseRequired":  "False"
}
{
    "Name":  "b2cAuthenticationCount",
    "LicenseRequired":  "False"
}
{
    "Name":  "b2cMfaRequestCountSummary",
    "LicenseRequired":  "False"
}
{
    "Name":  "tenantUserCount",
    "LicenseRequired":  "False"
}
{
    "Name":  "applicationUsageDetailEvents",
    "LicenseRequired":  "False"
}
{
    "Name":  "applicationUsageSummaryEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "b2cUserJourneySummaryEvents",
    "LicenseRequired":  "False"
}
{
    "Name":  "b2cUserJourneyEvents",
    "LicenseRequired":  "False"
}
{
    "Name":  "cloudAppDiscoveryEvents",
    "LicenseRequired":  "False"
}
{
    "Name":  "mimSsgmGroupActivityEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "ssgmGroupActivityEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "mimSsprActivityEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "ssprActivityEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "mimSsprRegistrationActivityEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "ssprRegistrationActivityEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "threatenedCredentials",
    "LicenseRequired":  "False"
}
{
    "Name":  "compromisedCredentials",
    "LicenseRequired":  "False"
}
{
    "Name":  "auditEvents",
    "LicenseRequired":  "False"
}
{
    "Name":  "accountProvisioningEvents",
    "LicenseRequired":  "False"
}
{
    "Name":  "signInsFromUnknownSourcesEvents",
    "LicenseRequired":  "False"
}
{
    "Name":  "signInsFromIPAddressesWithSuspiciousActivityEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "signInsFromMultipleGeographiesEvents",
    "LicenseRequired":  "False"
}
{
    "Name":  "signInsFromPossiblyInfectedDevicesEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "irregularSignInActivityEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "allUsersWithAnomalousSignInActivityEvents",
    "LicenseRequired":  "True"
}
{
    "Name":  "signInsAfterMultipleFailuresEvents",
    "LicenseRequired":  "False"
}
{
    "Name":  "applicationUsageSummary",
    "LicenseRequired":  "True"
}
{
    "Name":  "userActivitySummary",
    "LicenseRequired":  "False"
}
{
    "Name":  "groupActivitySummary",
    "LicenseRequired":  "True"
}

How to Access the Reporting API using PowerShell

What you need to do is;

  • Register a WebApp
    • Assign a reply to URL of https://localhost
    • Assign it Read.Directory permissions
  • Get an oAuth2 Authentication Code using an account that is either Global Admin or in the Security Admin or Security Reader Azure Roles
  • Use your Bearer and Refresh tokens to query for the reports you’re interested in

Register your WebApp

In the Azure Portal create a new Web app/API app and assign it https://localhost as the Reply URL. Record the Application ID for use in the PowerShell script.

Assign the Read Directory data permission as shown below

Obtain a key from the Keys option on your new Web App.  Record it for use in the PowerShell script.

Generate an Authentication Code, get a Bearer and Refresh Token

Update the following script, changing Lines 5 & 6 for the ApplicationID/ClientId and Client Secret for the WebApp you created above.

Run the script and you will be prompted to authenticate. Use an account in the tenant where you created the Web App that is a Global Admin or in the Security Admin or Security Reader Azure Roles. You will need to change the location where you want the refresh.token stored (line 18).

If you’ve done everything correctly you have authenticated, got an AuthCode which was then used to get your Authorization Tokens. The value of the $Authorization variable should look similar to this;

Now you can use the Refresh token to generate new Authorization Tokens when they time out, simply by calling the Get-NewTokens function included in the script above.

Querying the Reporting API

Now that you have the necessary prerequisites sorted you can query the Reporting API.

Here are a couple of simple queries to return some data to get you started. Update the script for the tenant name of your AzureAD. With the $Authorization values from the script above you can get data for the MIM Hybrid Reports.

Joining Identities between Active Directory and Azure Active Directory using Microsoft Identity Manager

Introduction

One of the foundations of Identity Management is the ability to join an identity between disparate connected systems. As we extend our management of identities into cloud services this adds a few twists.

A key concept is to use an anchor that is persistent. Something that doesn’t change through a users life-cycle. A user’s Security IDentifier (SID) in Active Directory is perfect. It doesn’t change when a user or group may get renamed.  What gets interesting is how the SID is represented when returned using different methods. That is what I quickly cover in this post.

Overview

The defacto method for connecting your OnPremise Active Directory to Azure Active Directory is to use Azure Active Directory Connect. AADC will synchronise users and groups SID’s to the corresponding object in AAD into the onPremisesSecurityIdentifier attribute. 

When the onPremisesSecurityIdentifier  attribute is retrieved via the GraphAPI the format looks like this: S-1-5-21-3878594291-2115959936-132693609-65242

Using an Active Directory Management Agent for FIM/MIM and synchronizing the objectSID from the OnPremise AD represents the value in the Metaverse in binary format which when viewed as text looks like this:  15000005210002431664623112825230126105190232781820

Translating SID formats so we can join identities

What we need to do is translate the string representation of the SID returned from the GraphAPI and AzureAD so that we have it in a binary format. Then we can then use those attributes in join rules to match users/groups between AzureAD and our OnPremise Active Directory.

Overview-1

 

In my environments I’m using the out of the box FIM/MIM Active Directory Management Agent. For Azure AD/Office 365 I’m using the Granfeldt PowerShell Management Agent to integrate with Azure AD via the GraphAPI.

On my AzureAD PowerShell Management Agent I have an attribute named AADonPremiseSID configured with the format as Binary in my PSMA Schema.ps1 as shown below.

$obj | Add-Member -Type NoteProperty -Name "AADonPremiseSID|Binary" -Value 0x10

On my Azure AD PSMA I have the following lines in my Import.ps1 which essentially takes the value retrieved from the GraphAPI S-1-5-21-3878594291-2115959936-132693609-65242 and converts it to a binary array that in text looks something like 15000005210002431664623112825230126105190232721825400 and stores it in the AADonPremiseSID binary attribute in the connector space.

# Create SID .NET object using SID string from AAD S-1-500-........ 
$sid = New-Object system.Security.Principal.SecurityIdentifier $user.onPremisesSecurityIdentifier
 
#Create a byte array for the length of the users SID
$BinarySid = new-object byte[]($sid.BinaryLength)

#Copy the binary SID into the byte array, starting at index 0
$sid.GetBinaryForm($BinarySid, 0)

#Add the SID to the user in the connector space
$obj.Add("AADonPremiseSID",$BinarySid)

This then lets me join my users (and groups using the same method) between AD and AAD.  Essentially a line to put it into Security Identifier format, two lines to convert it to a binary array and a line to store it in the connector space. Simple when you don’t over think it.

I’m posting this because I know I’m going to need to do this often. Hope it helps someone else too.

How to create a PowerShell FIM/MIM Management Agent for AzureAD Groups using Differential Sync and Paged Imports

Introduction

I’ve been working on a project where I must have visibility of a large number of Azure AD Groups into Microsoft Identity Manager.

In order to make this efficient I need to use the Differential Query function of the AzureAD Graph API. I’ve detailed that before in this post How to create an AzureAD Microsoft Identity Manager Management Agent using the MS GraphAPI and Differential Queries. Due to the number of groups and the number of members in the Azure AD Groups I needed to implement Paged Imports on my favourite PowerShell Management Agent (Granfeldt PowerShell MA). I’ve previously detailed that before too here How to configure Paged Imports on the Granfeldt FIM/MIM PowerShell Management Agent.

This post details using these concepts together specifically for AzureAD Groups.

Pre-Requisites

Read the two posts linked to above. They will detail Differential Queries and Paged Imports. My solution also utilises another of my favourite PowerShell Modules. The Lithnet MIIS Automation PowerShell Module. Download and install that on the MIM Sync Server where you be creating the MA.

Configuration

Now that you’re up to speed, all you need to do is create your Granfeldt PowerShell Management Agent. That’s also covered in the post linked above  How to create an AzureAD Microsoft Identity Manager Management Agent using the MS GraphAPI and Differential Queries.

What you need is the Schema and Import PowerShell Scripts. Here they are.

Schema.ps1

Two object classes on the MA as we need to have users that are members of the groups on the same MA as membership is a reference attribute. When you bring through the Groups into the MetaVerse and assuming you have an Azure AD Users MA using the same anchor attribute then you’ll get the reference link for the members and their full object details.

Import.ps1

Here is my PSMA Import.ps1 that performs what is described in the overview. Enumerate AzureAD for Groups, import the active ones along with group membership.

Summary

This is one solution for managing a large number of Azure AD Groups with large memberships via a PS MA with paged imports showing progress thanks to differential sync which then allows for subsequent quick delta-sync run profiles.

I’m sure this will help someone else. Enjoy.

Follow Darren on Twitter @darrenjrobinson

How to create an AzureAD Microsoft Identity Manager Management Agent using the MS GraphAPI and Differential Queries

Introduction

In August 2016 I wrote this post on how to use PowerShell to leverage the Microsoft GraphAPI and use Differential Queries. The premise behind that post was I required a Microsoft Identity Manager Management Agent to synchronize identity information from AzureAD into Microsoft Identity Manager. However the environment it was intended for has a large AzureAD implementation and performing a Full Sync every-time is just to time consuming. Even more so with this limitation that still exists today in MIM 2016 with SP1.

In this blog post I’ll detail how to implement a PowerShell Management Agent for FIM/MIM to use the MS GraphAPI to synchronize objects into FIM/MIM, supporting Delta and Full Synchronization run profiles. I’m also using my favourite PowerShell Management Agent, the Granfeldt PowerShell Management Agent.

Pre-requsites

I’m using the ADAL helper library from the AzureADPreview PowerShell Module. Install that module on you MIM Sync Server via PowerShell (WMF5 or later) using the PowerShell command;

Install-Module AzureADPreview

Getting Started with the Granfeldt PowerShell Management Agent

If you don’t already have it, what are you waiting for. Go get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

Three items I had to work out that I’ll save you the pain of are;

  • You must have a Password.ps1 file. Even though we’re not doing password management on this MA, the PS MA configuration requires a file for this field. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present
  • The credentials you give the MA to run this MA are the credentials for the account that has permissions to the AzureAD/Office365 Tenant. Just a normal account is enough to enumerate it, but you’ll need additional permissions if you intend to write-back to AzureAD.
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~4\2010\SYNCHR~1\EXTENS~2\AzureAD

Schema.ps1

My Schema is based around enumerating and managing users from AzureAD. You’ll need to create a number of corresponding attributes in the Metaverse Schema on the Person ObjectType to flow the attributes into. Use the Schema info below for a base set of attributes that will get you started. You can add more as required. I’ve prefixed most of them with AAD for my implementation.

If you want to manage Groups or Contacts or a combination of object types, you will need to update the Schema.ps1 script accordingly.

Import.ps1

The logic that the Import.ps1 implements is the same as detailed here in my post using Differential Queries. Essentially, perform a full import and create a file with the cookie/watermark. Allow Delta Sync run profiles to be performed by requesting the GraphAPI to return only changes since the cookie/watermark.

You’ll need to update the script for your AzureAD Tenant name on Line 28. Also the path to where the cookie file will go and the debug file if your path is different to mine. Lines 11, 46 and 47.

Importing of the attributes is based around the names in the Schema.ps1 scripts. Any changes you made there will need to be reflected in the import.ps1.

Password Script (password.ps1)

Empty as not implemented

Export.ps1

Empty as not implemented in this example. If you are going to write information back to AzureAD you’ll need to put the appropriate logic into this script.

Management Agent Configuration

With the Granfeldt PowerShell Management Agent installed on your FIM/MIM Synchronisation Server, in the Synchronisation Server Manager select Create Management Agent and choose “PowerShell” from the list of Management Agents to create.

As this example is for Users, I’ve named my MA accordingly.

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using an Office 365 account to connect to AzureAD/Office365 and import the user data.

Paths to the Import, Export and Password scripts. Note: the Export and Password PS1 scripts files exist but are empty.

Object Type as configured in the Schema.ps1 file.

Attributes as configured in the Schema.ps1 file.

Anchor as per the Schema.ps1 file.

The rest of the MA configuration is up to your implementation. What you are going to join on and what attributes flow into the MV will vary based on your needs and solution. At a minimum you’d probably be looking to do a join on immutableID (after some manipulation) or UPN and flow in attributes such as AADAccountEnabled etc.

Completing the Configuration

To finalise the MA you’ll need to do the usual tasks of creating run profiles, staging the connector space from AzureAD/Office365 and syncing into the Metaverse. Once you’ve done your initial Stage/Full Sync you can perform Delta Sync’s.

Summary

A “Full Import” on a small AzureAD (~8500 Users) took 2 minutes.
A subsequent “Delta Import” with no changes took 6 seconds.

A similar implementation of the logic, but for Groups gives similar results/performance.
A  “Full Import” on a small AzureAD (~9800 Groups) took 5 minutes.
A subsequent “Delta Import” with 7 Adds (new Groups) and 157 Updates took 1 minute.

 

Follow Darren on Twitter @darrenjrobinson

Leveraging the Microsoft Graph API with PowerShell and OAuth 2.0

Background

Microsoft Graph is the evolvement of API’s into Microsoft Cloud Services. For me not being a developer, a key difference is interacting with with Graph API using OAuth 2.0 via PowerShell. Through a number of my previous posts I’ve interacted with the Graph API using client libraries such as the Microsoft.IdentityModel.Clients.ActiveDirectory library. This post details using PowerShell to talk directly to Graph API and managing Authentication and Authorization using OAuth 2.0 and Azure WebApp.

Leveraging the Graph API opens up access to the continually evolving Azure services as shown in the graphic below. Source graph.microsoft.io

Getting started with the Graph API, PowerShell and OAuth 2.0

The key difference between using a client library and going direct is you need to register and configure an Azure WebApp. It is super simple to do. Jump on over to the Office 365 App Registration Tool here. Sign in with an account associated with the Azure Tenant you are going to interact with. Depending on what you’re doing you’ll need to select the appropriate access.

Here’s the settings I selected for access to user profiles. Give the WebApp a name. This is the name that you’ll see in the OAuth Authorization step later on. I’ve highlighted the other key settings. Don’t worry about the SignIn and RedirectURL’s other than configuring HTTPS as we’ll be using PowerShell to access the WebApp.

Once you’ve registered the WebApp successfully you’ll get a Client ID and Client Secret. RECORD/WRITE THESE DOWN NOW as you won’t get access to your Client Secret again.

If you want to change what your WebApp has access to after creating it you can access it via the Classic Azure Portal. Select your Active Directory => Select Applications => Select the WebApp you created earlier  => Select Configure => (scroll to the bottom) Select Add Application. Depending on what you have in your subscription you can add access to your services as shown below.

Authenticating & Authorizing

In order to access the Graph API we need to get our Authorization Code. We only need to do this the first time.

This little script (modify with your Client ID and your Client Secret obtained earlier when we registered our WebApp) will prompt you to authenticate.

Using the account associated with the Web App you registered in the previous step authenticate.

You’ll be requested to authorize your application.

You will then have your AuthCode.

One thing to note above is admin_consent. This is from the URL passed to get the Authorization Code. &prompt=admin_consent is giving Admin Consent to all entities configured on the WebApp over just access for and to a single user.

Accessing the Graph API with OAuth 2.0 Access Token

Using your ClientID, ClientSecret and AuthCode you can now get your access token. I tripped up here and was getting Invoke-RestMethod : {“error”:”invalid_client”,”error_description”:”AADSTS70002: Error validating credentials.
AADSTS50012: Invalid client secret is provided.  Tracing back my steps and looking at my “Client Secret” I noticed the special characters in it that I hadn’t URL Encoded. After doing that I was successfully able to get my access token.

Looking at our AuthZ we can see that the Scope is what was selected when registering the WebApp.

Now using the Access Token I can query the Graph API. Here is getting my AAD Object.

Summary

In my case I now can access all users via the API. Here is what’s available. Using the Access Token and modifying the Invoke-RestMethod URI and Method (including -Body if you are doing a Post/Patch action) you are ready to rock and roll and all via PowerShell.

Your Access Token is valid for an hour. Before then you will need to refresh it. Just run the $Authorization = Invoke-RestMethod ….. line again. 

Follow Darren on Twitter @darrenjrobinson

Getting Users, Groups & Contacts via the Azure Graph API using Differential Query & PowerShell

This is the final post in a series detailing using PowerShell to leverage the Azure AD Graph API. For those catching up it started here introducing using PowerShell to access the Azure AD via the Graph API, licensing users in Azure AD via Powershell and the Graph API, and returning all objects using paging via Powershell and the Graph API.

In this post I show how to;

  • enumerate objects from Azure AD via Powershell and the Graph API, and set a delta change cookie
  • enumerate changes in Azure AD since the last query
  • return objects that have changed since the last query
  • return just the changed attributes on objects that have changed since the last query
  • get a differential sync from now delta change link

Searching through MSDN and other resources working this out I somehow stumbled upon a reference to changes in the API that detail the search filters. v1.5 and later of the API requires filters using the context ‘Microsoft.DirectoryServices.User|Group|Contact’ etc instead of ‘Microsoft.WindowsAzure.ActiveDirectory.User|Group|Contact’ which you’ll find in the few examples around. If you don’t want to return all these object types update the filter on line 21 in the script below.

Here is the script to return all Users, Groups and Contacts from the tenant along with all the other options I detail in this post. Update the following for your tenant;

  • line 6 for your tenant URI
  • line 10 for your account in the tenant
  • line 11 for the password associated with your account from line 10

Here is a sample output showing the Users, Groups, Contacts and DirectoryLinkChange objects. Note: if you have a large tenant that has been in place for a period of time it may take a while to enumerate.  In this instance you can use the Differential Sync from now option. More on that later.

Running the query again using the Differential DeltaLink from the first run now returns no results. This is as expected as no changes have been made in the tenant on the objects in our query.

Now if I make a change in the tenant and run the query again using the Differential DeltaLink I get 1 result. And I get the full object.

What if I just wanted to know the change that was made?

If we add ‘&ocp-aad-dq-include-only-changed-properties=true’ to the URI that’s exactly what we get. The object and what changed. In my case the Department attribute.

Finally as alluded to earlier there is the Differential Sync from now option. Very useful on large tenants where you can query and get all users, contacts, groups etc without using differential sync, then get the Differential Delta token for future sync queries. So I’ve used the same URI that I used as the beginning of this blog post but in the header specified ‘ocp-aad-dq-include-only-delta-token’ = “true” and as you can see I returned no results but I got the important Differential Query DeltaLink.

Summary

Using Powershell we can leverage the Azure AD Graph RestAPI and use the Differential Sync functions to efficiently query Azure AD for changes rather than needing to enumerate an entire tenant each time. Brilliant.

Follow Darren on Twitter @darrenjrobinson

Goodbye Set-MsolUser, Hello Set-AzureADUser & Azure Graph API

Recently Microsoft released the preview of the v2.0 Azure AD PowerShell cmdlets. https://azure.microsoft.com/en-us/updates/azure-ad-new-powershell-cmdlets-preview/

I’ve got a project coming up where I’m looking to change my approach for managing users in Azure using Microsoft Identity Manager. Good timing to do a quick proof of concept to manage users with the new cmdlets and directly using the Graph API in preparation to move away from the msol cmdlets.

New Modules

First up, the Azure AD v2.0 PowerShell module was released in public preview on July 13, 2016. There will likely be changes before they become GA, so keep that in mind.

The v2.0 Azure AD PowerShell Module modules themselves are available for download from here https://www.powershellgallery.com/packages/AzureADPreview/1.1.143.0

If you have Windows Management Framework v5 installed you can download and install from PowerShell (as below).

Once installed, pretty quickly you can import the module, authenticate to your tenant, retrieve a user and update a few attributes (as below).

Whilst functional it doesn’t really work for how we need to interact with Azure from an Identity Management perspective. So how can we still use PowerShell but enumerate and manipulate identities in Azure ?

Now that we have the AzureAD v2.0 module installed we can reference the Active Directory library it installs (Microsoft.IdentityModel.Clients.ActiveDirectory.dll), authenticate to our Tenant retrieve users, and update them. That’s exactly what is shown in the commands below.

Where interacting with the GraphAPI directly really shines however is at the directory services layer and the Differential Query functionality.  https://msdn.microsoft.com/en-us/Library/Azure/Ad/Graph/howto/azure-ad-graph-api-differential-query

As such this is the approach that I’ll be taking for integration of Azure with Microsoft Identity Manager for managing users for entitlements (such as Azure licensing).
I hope this though also saves a few people time in working out how to use PowerShell to manage Azure objects via the Graph API (using both the PowerShell Module or via the RestAPI).