Fast forward to 2019 and I’ve been building a bunch of reports from Microsoft Graph that aggregate data from multiple API endpoints such as /users /auditLogs and /security . A perfect time to explore deeper JSON Batching with Microsoft Graph.
Getting Started
Essentially your Graph API requests are the same, but instead of calling the API endpoint directly, you make a POST request to the Batch endpoint …
https://graph.microsoft.com/v1.0/$batch
… with your request in the body that includes the suffix for the API URI for the request, the request method and an ID. The example below is to query the User API endpoint for users whose displayName starts with the string provided.
Additional requests are added as required to build the Request Body and passed as part of the Post request to the Batch endpoint. Each additional request in the batch requires a different ID and the respective particulars for the request(s).
Batching Requests using PowerShell
Below is an example for generating a batch in PowerShell with two requests.
The responses come back under the Responses property as shown in the graphic below. The responses have the ID of the requests associated with them so you can associate a response with a request.
The request responses can then be parsed by iterating through the jobs.
Summary
Using Microsoft Graph JSON Batching we can submit multiple requests as a single job, have the requests performed on the backend in parallel and get the results back.
Keep in mind that the request can also use the DependsOn option to make a request dependent on another that must be completed prior. Also if the response contains a dataset that is paged. Check the nextLink property on the responses and make subsequent calls to return the remaining data.
What we know today as the Microsoft Graph has evolved over the last few years from a number of different API’s that were developed by different product teams within Microsoft (e.g Azure AD, Office 365, Outlook). That doesn’t mean the old ones have gone away, but it does mean that we can connect to the Microsoft Graph API and leverage the API’s we used to interface with independently.
What this means is, where information is actually coming from is obfuscated. But there is a way to find out where it is coming from.
The $whatIf option is a mostly undocumented switch that provides visibility on the source of information returned by Microsoft Graph from differing back-end API’s. The easiest way to use this is via the Graph Explorer.
Running a query using Graph Explorer for /me returns a bunch of information.
Running the same query with the $whatIf option returns what the /me call actually performs on the API.
In this case it returns the base user information as shown below in the $select query.
If I change the query to a specific user and request information that comes from different products behind the Microsoft Graph and I use the $whatIf option we can see where the data is coming from.
The screenshot below shows that the request actually executed 2 requests in parallel and merge the entity responses
The result then shows where the two queries went. In this case graph.windows.net and sharepoint.com.
{“Description”:”Execute 2 request in parallel and merge the entity responses”,”Request1“:{“Description”:”Execute HTTP request”,”Uri”:”https://graph.windows.net/v2/dcd219dd-bc68-4b9b-bf0b-4a33a796be35/users(‘MeganB@M365x214355.onmicrosoft.com’)?$select=mail”,”HttpMethod”:”GET”},”Request2“:{“Description”:”Execute HTTP request”,”Uri”:”https://m365x214355.sharepoint.com/_api/SP.Directory.DirectorySession/GetSharePointDataForUser(’48d31887-5fad-4d73-a9f5-3c356e68a038′)?$select=aboutMe”,”HttpMethod”:”GET”}}
Summary
The next time you’re trying to workout where information is coming from when using the Microsoft Graph API, try adding the $whatIf switch to your query and have a look at the Response.
Update: Oct 30 '18
Also see this post that adds support for Microsoft's updates
to the Microsoft Graph to include additional information
about Azure AD B2B Guest users.
Introduction
Earlier this year Microsoft released the Microsoft Identity Manager Azure AD B2B Management Agent. I wrote about using it to write to Azure AD in this post here. As detailed in that post my goal was to write to Azure AD using the MA. I provided an incomplete example of doing that for Guests. This post fills in the gap and unlike the note preceding the post indicates, I’ve updated my MA to use the Graph API over the Azure AD PowerShell Module. It does though work in unison with the Microsoft Azure AD B2B Management Agent.
Overview
The process is;
Using the Microsoft Azure B2B Management Agent connect to an Azure AD Tenant that contains users that you want to invite as Guests to your Tenant. Flow in the naming information for users and their email address and any other metadata that you need to drive the logic for who you wish to invite
Use my Azure AD B2B Invitation Management Agent to automate the invitation of users to your Azure AD Tenant using Azure AD B2B
My Azure AD B2B Invitation Management Agent works in two phases;
Invitation of Users as Guests
Update of Guests with naming information (Firstname, Lastname, DisplayName)
The Azure AD B2B Invite Management Agent uses my favorite PowerShell Management Agent (the Granfeldt PSMA). I’ve posted many times on how to configure it. See these posts here if you are new to it.
Prerequisites
Setting up an Azure AD User Account for the Management Agent
In your Azure AD create a New User that will be used by the Management Agent to invite users to your Azure AD. I named mine B2B Inviter as shown below.
You then want to assign them the Guest inviter role as shown below. This will be enough permissions to invite users to the Azure AD.
However depending on how you want these invitee’s to look, you probably also want their names to be kept consistent with their home Azure AD. To also enable the Management Agent to do that you need to also assign the User administrator role as shown below.
Now log in using that account to Azure AD and change the password. The account is now ready to go.
Management Agent Scripts
The Management Agent uses the Granfeldt PowerShell Management Agent. This is a cut down version of my MIM Azure AD Management Agent.
Schema Script
I’ve kept the schema small with just enough interesting info to facilitate the functionality required. Expand it if you need additional attributes and update the import.ps1 accordingly.
Import.ps1
The Import script imports users from the Azure AD Tenant that you will be inviting remote Azure AD users too (as Guests).
Change line 10 for your file path
Change line 24 for the version of an AzureAD or AzureADPreview PowerShell Module that you have installed on the MIM Sync Server so that the AuthN Helper Lib can be used. Note if using a recent version you will also need to change the AuthN calls as well as the modules change. See this post here for details.
Change line 27 for your tenant name
Change line 47/48 for a sync watermark file
The Import script also contains an attribute from the MA Schema named AADGuestUser that is a boolean attribute. I create the corresponding attribute in the MetaVerse and MIM Service Schemas for the Person/User objectClasses. This is used to determine when a Guest has been successfully created so their naming attributes can then be updated (using a second synchronisation rule).
Export.ps1
The Export script handles the creation (invitation) of users from another azure AD Tenant as Guests as well synchronizing their naming data. It doesn’t include deletion logic, but that is simple enough include a deletion API call based on your MA Deprovisioning logic and requirements.
By default I’m not sending invitation notifications. If you want to send invitation notifications change “sendInvitationMessage“= $false to $true on Line 129. You should then also change the Invitation Reply URL on line 55 to your Tenant/Application.
Change Line 10 for the path for the debug logging
Change Line 24 as per the Import Script if you are using a different version of the help lib
Change Line 27 for your Azure AD Tenant Name
Declarative Sync Rules
I’m not going to cover import flow configurations on the MS Azure AD B2B MA here. See here for that. Below details how I’ve configured my Invitation MA for the Creation/Export functions. My join rule (configured in the Sync Engine Invitation MA Config) is email address as shown below. Not the best anchor as it isn’t immutable. But as we don’t know what the DN is going to be until after it is created this is the next best thing.
Creation Sync Rule
Here are the three attributes I’m syncing to the B2B Invite Management Agent to perform the invitation. I’m using the mail attribute for the DN as it matches the anchor for the schema. We don’t know what objectID will be assigned until the directory service does it. By using email/upn once created we will get the join and won’t end up with two objects on the MA for the same user.
For Inbound I’m flowing in the AADGuestUser boolean value. You will need to create this attribute in the MetaVerse and then the MIM Service. In the MIM Service allow the Sync Service Account to manage the attribute and change the MIM Service Filter Permissions to allow Admins to use the attribute in Sets. Also on the MIM Service MA add an Export flow from the MV to the MIM Service for AADGuestUser.
Naming Update Sync Rule
The second Sync Rule is to update the guests GivenName, Surname and DisplayName. This sync rule has a dependency on the creation sync rule and has a corresponding Set, Workflow and MPR associated with value of the AADGuestUser boolean attribute populated by the Import script. If True (which it will be after successful creation and the confirming import) the second synchronization rule will be applied.
I will trigger an export flow for the three naming attributes.
Example of Inviting Guests
In this example Rick Sanchez is a member of a guest organisation and meets the criteria of my rules to be invited as a guest to our Tenant. We then, that we get an Add for Rick on the B2B Invite MA.
On export Rick is created in our Azure AD as a Guest User
Rick appears in Azure AD as a Guest via the Azure Portal.
Following the confirming import our second sync rule fires and flows through an update to DisplayName and adds GivenName and Surname.
This naming attributes are then successfully exported.
Going to the Azure AD Portal we see that Rick has indeed been updated.
Notification Emails
If you enable notification emails a generic notification email is sent like shown below. The import.ps1 MA script has them disabled by default.
Summary
Using a combination of the Microsoft Azure AD B2B Management Agent and my Azure AD B2B Invitation Management Agent you can automate the invitation of Guest users to your Azure AD Tenant.
UPDATE: August 2018
As promised below I've finally written up my
Azure AD B2B Invitation Management Agent.
You can find it in this post here.
UPDATE: June 2018
When I originally wrote this post the intent was to test
the ability of the Graph MA to export to Azure AD.
That works.
That then extended to messing with an identity type other
than member (which works to an extent) but I detailed
guests. However that is incomplete. I do have a working
solution that utilises the Graph Invitation API via my
favourite PowerShell MA (Granfeldt PS MA) and the
PowerShell cmdlet New-AzureADMSInvitation from the
Azure AD PowerShell Module.
That solution involves the MS Graph MA connected to a
Partner tenant to get visibility of partner users and
then creating relevant users in the home tenant via a
PowerShell MA and the New-AzureADMSInvitation cmdlet.
Another MS Graph MA connected to the home tenant provides
easy visibility of additional guest user attributes for
ongoing functions such as reporting and de-provisioning.
I will write that up later in July. Stay tuned and keep
the above in mind when reading this post.
Microsoft have documented a number of scenarios for implementing the management agent. The scenarios the MA has been built for are valid and I have customers that will benefit from the new MA immediately. There is however another scenario I’m seeing from a number of customers that is possible but not detailed in the release notes. That is B2B Sync between Azure Tenants; using Microsoft Identity Manager to automate the creation of Guests in an Azure Tenant.
This could be one-way or multi-way depending on what you are looking to achieve. Essentially this is the Azure equivalent of using FIM/MIM for Global Address List Sync.
Overview
The changes are minimal to the documentation provided with the Management Agent. Essentially;
ensure you enable Write Permissions to the Application you create in the AAD Tenant you will be writing too
Enable the Invite Guest users to the organization permission on the AAD Application
Create an Outbound Sync Rule to an AAD Tenant with the necessary mandatory attributes
Configure the Management Agent for Export Sync Profiles
In the scenario I’m detailing here I’m showing taking a number of users from Org2 and provisioning them as Guests in Org1.
What I’m detailing here supplements the Microsoft documentation. For configuring the base MA definitely checkout their documentation here.
Microsoft Graph Permissions
When setting up the Graph Permissions you will need to have Write permissions to the Target Azure AD for at least Users. If you plan to also synchronize Groups or Contacts you’ll need to have Write permissions for those too.
In addition as we will be automating the invitation of users from one Tenant to another we will need to have the permission ‘Invite guest users to the organization’.
With those permissions selected and while authenticated as an Administrator select the Grant Permissions button to assign those permissions to the Application.
Repeat this in both Azure AD Tenants if you are going to do bi-directional sync. If not you only need write and invite permissions on the Tenant you will be creating Guest accounts in.
Creating the Import/Inbound Sync Rules Azure Tenants
Here is an example of my Import Sync Rules to get Members (Users) in from an Azure Tenant. I have an inbound sync rule for both Azure Tenants.
Make sure you have ‘Create Resource in FIM‘ configured on the source (or both if doing bi-directional) Graph Connector.
The attribute flow rules I’ve used are below. They are a combination of the necessary attributes to create the corresponding Guest account on the associated management agent and enough to be used as logic for scoping who gets created as a Guest in the other Tenant. I’ve also used existing attributes negating the need to create any new ones.
Creating the Export/Outbound Sync Rule to a Partner B2B Tenant
For your Export/Outbound rule make sure you have ‘Create resource in external system’ configured.
There are a number of mandatory attributes that need to be flowed out in order to create Guests in Azure AD. The key attributes are;
userType = Guest
accountEnabled = True
displayName is required
password is required (and not export_password as normally required on AD style MA’s in FIM/MIM)
mailNickname is required
for dn and id initially I’m using the id (flowed in import to employeeID) from the source tenant. This needs to be provided to the MA to get the object created. Azure will generate new values on export so we’ll see a rename come back in on the confirming import
userPrincipalName is in the format of
SOURCEUPN (with @ replaced with _ ) #EXT# DestinationUPNSuffix
e.g user1_org2.com#EXT#org1.com
Here is an example of building a UPN.
Sets, Workflows and MPR’s
I didn’t need to do anything special here. I just created a Set based on attributes coming in from the source Azure Tenant to scope who gets created in the target Tenant. An MPR that looks for transition into the Set and applies the Workflow that associates the Sync Rule.
End to End
After synchronizing in from the source (B2B Org 2) the provisioning rules trigger and created the Users as Guests on B2B Org 1.
Looking at the Pending Export we can see our rules have applied.
On Export the Guest accounts are successfully created.
On the confirming import we get the rename as Azure has generated a new CN and therefore DN for the Guest user.
Looking into Azure AD we can see one of our new Guest users.
Summary
Using the Microsoft Azure B2B Graph Management Agent we can leverage it to create users from one Tenant as Azure AD Members in another Tenant. Stay tuned for another post detailed the solution detailed in the Update in the Introduction.
At the end of last week (14 Sept 2017) Microsoft announced a new Azure Active Directory feature – Managed Service Identity. Managed Service Identity helps solve the chicken and egg bootstrap problem of needing credentials to connect to the Azure Key Vault to retrieve credentials. When used in conjunction with Virtual Machines, Web Apps and Azure Functions that meant having to implement methods to obfuscate credentials that were stored within them. I touched on one method that I’ve used a lot in this post here whereby I encrypt the credential and store it in the Application Settings, but it still required a keyfile to allow reversing of the encryption as part of the automation process. Thankfully those days are finally behind us.
I strongly recommend you read the Managed Service Identity announcement to understand more about what MSI is.
This post details using Managed Service Identity in PowerShell Azure Function Apps.
Enabling Managed Service Identity on your Azure Function App
In the Azure Portal navigate to your Azure Function Web App. Select it and then from the main-pane select the Platform Features tab then select Managed service identity.
Enable Managed Service Identity
Turn the toggle the switch to On for Register with Azure Active Directory then select Save.
Enable Managed Service Identity
Back in Platform Features under General Settings select Application Settings.
Azure Function App Settings
Under Application Settings you will see a subset of the environment variables/settings for your Function App. In my environment I don’t see the Managed Service Identity variables there. So lets keep digging.
Azure Function App Settings
Under Platform Features select Console.
Azure Function App Console
When the Console loads, type Set. Scroll down and you should see MSI_ENDPOINT and MSI_SECRET.
NOTE: These variables weren’t immediately available in my environment. The next morning they were present. So I’m assuming there is a back-end process that populates them once you have enabled Managed Service Identity. And it takes more than a couple of hours
MSI Variables
Creating a New Azure Function App that uses Managed Service Identity
We will now create a new PowerShell Function App that will use Managed Service Identity to retrieve credentials from an Azure Key Vault.
From your Azure Function App, next to Functions select the + to create a New Function. I’m using a HttpTrigger PowerShell Function. Give it a name and select Create.
New Azure Function
Put the following lines into the top of your function and select Save and Run.
# MSI Variables via Function Application Settings Variables
# Endpoint and Password
$endpoint = $env:MSI_ENDPOINT
$endpoint
$secret = $env:MSI_SECRET
$secret
You will see in the output the values of these two variables.
Go to your Key Vault and select Access Polices from the left menu list.
Azure Key Vault Access Policy
Select Add new, Select Principal and locate your Function App and click Select.
Azure Key Vault Access Policy
As my vault contains multiple credential types, I enabled the policy for Get for all types. Select Ok. Then select Save.
Azure Key Vault Access Policy
We now have our Function App enabled to access the Key Vault.
Azure Key Vault Access Policy
Finally in your Key Vault, select a secret you want to retrieve via your Function App and copy out the Secret Identifier from the Properties.
Azure Key Vault Secret Identifier URI
Function App Script
Here is my Sample PowerShell Function App script that will connect to the Key Vault and retrieve credentials. Line 12 should be the only line you need to update for your Key Vault Secret that you want to retrieve. Ensure you still have the API version at the end (which isn’t in the URI you copy from the Key Vault) /?api-version=2015-06-01
When run the output if you have everything correct will look below.
Summary
We now have the basis of a script that we can use in our Azure Functions to allow us to use the Managed Service Identity function to connect to an Azure Key Vault and retrieve credentials. We’ve limited the access to the Key Vault to the Azure Function App to only GET the credential. The only piece of information we had to put in our Function App was the URI for the credential we want to retrieve. Brilliant.
Update:Dec 20 2018.
See this post that details the changes to the Azure AD
Reports and Events Rest API.
Hybrid Reporting is a great little feature of Microsoft Identity Manager. A small agent installed on the MIM Sync Server will send reporting data to Azure for MIM SSPR and MIM Group activities. See how to install and configure it here.
But what if you want to get the reporting data without going to the Azure Portal and looking at the Audit Reports ? Enter the Azure AD Reports and Events REST API that is currently in preview. It took me a couple of cracks and getting this working, because documentation is a little vague especially when accessing it via PowerShell and oAuth2. So I’ve written it up and hope it helps for anyone else looking to go down this route.
Gotchas
Accessing the Reports via the API has a couple of caveats that I had to work through:
Having the correct permissions to access the report data. Pretty much everything you read tells you that you need to be a Global Admin. Once I had my oAuth tokens I messed around a little and a was able to also get the following from back from the API when purposely using an identity that didn’t have the right permissions. The key piece is “Api request is not from global admin or security admin or security reader role”. I authorized the WebApp using an account that is in the Security Reader Role, and can successfully access the report data.
Reading the documentation here on MSDN I incorrectly assumed each category was the report name. Only when I called the “https://graph.windows.net//$metadata?api-version=beta” and looked at the list of reports I noticed each report was plural.The three that I wanted to access (and report on) are obviously the MIM Hybrid Reports;
Get an oAuth2 Authentication Code using an account that is either Global Admin or in the Security Admin or Security Reader Azure Roles
Use your Bearer and Refresh tokens to query for the reports you’re interested in
Register your WebApp
In the Azure Portal create a new Web app/API app and assign it https://localhost as the Reply URL. Record the Application ID for use in the PowerShell script.
Assign the Read Directory data permission as shown below
Obtain a key from the Keys option on your new Web App. Record it for use in the PowerShell script.
Generate an Authentication Code, get a Bearer and Refresh Token
Update the following script, changing Lines 5 & 6 for the ApplicationID/ClientId and Client Secret for the WebApp you created above.
Run the script and you will be prompted to authenticate. Use an account in the tenant where you created the Web App that is a Global Admin or in the Security Admin or Security Reader Azure Roles. You will need to change the location where you want the refresh.token stored (line 18).
If you’ve done everything correctly you have authenticated, got an AuthCode which was then used to get your Authorization Tokens. The value of the $Authorization variable should look similar to this;
Now you can use the Refresh token to generate new Authorization Tokens when they time out, simply by calling the Get-NewTokens function included in the script above.
Querying the Reporting API
Now that you have the necessary prerequisites sorted you can query the Reporting API.
Here are a couple of simple queries to return some data to get you started. Update the script for the tenant name of your AzureAD. With the $Authorization values from the script above you can get data for the MIM Hybrid Reports.
This is Part Two in the two-part blog post on managing users profile photos with Microsoft FIM/MIM. Part one here detailed managing users Azure AD/Active Directory profile photo. This post delves deeper into photos, specifically around Office 365 and the reason why you may want to manage these via FIM/MIM.
Background
User profile photos should be simple to manage. But in a rapidly moving hybrid cloud world it can be a lot more complex than it needs to be. The best summary I’ve found of this evolving moving target is from Paul Ryan here.
Using Paul’s sound advice we too are advising our customers to let users manage their profile photo (within corporate guidelines) via Exchange Online. However as described in this article photos managed in OnPremise Active Directory are synchronized to Azure AD and on to other Office365 services only once. And of course we want them to be consistent across AD DS, Azure AD, Exchange Online and all other Office365 Services.
This post details synchronizing user profile photos from Exchange Online to MIM for further synchronization to other systems. The approach uses a combination of Azure GraphAPI and Exchange Remote PowerShell to manage Exchange Online User Profile Photos.
Current State
Users historically had a photo in Active Directory. DirSync/ADSync/AzureADConnect then synchronized that to Azure AD (and once only into Office 365).
Users update their photo in Office365 (via Exchange Online and Outlook Web Access)
the photo is synchronized across Office365 Services
Desired State
An extension of the Current State is the requirement to be able to take the image uploaded by users in Exchange Online, and synchronize it back to the OnPremise AD, and any other relevant services that leverage a profile photo
Have AzureADConnect keep AzureAD consistent with the new photo obtained from Office365 that is synchronized to the OnPrem Active Directory
Sync the current photo to the MIM Portal
Synchronizing User Profile Photos with Microsoft Identity Manager
Synchronizing Office365 Profile Photos
Whilst Part-one dealt with the AzureAD side of profile photos as an extension to an existing AzureAD PowerShell Management Agent for FIM/MIM, I’ve separated out the Office365 side to streamline it and make it as efficient as possible. More on that later. As such I’ve created a new PowerShell Management Agent specifically for Office365 User Profile Photos.
I’m storing the Exchange Online photo in the MIM Metaverse as a binary object just as I did for the AzureAD photo (but in a different attribute ). I’m also storing a checksum of the photos (as I did for the AzureAD Photo, but also in a different attribute) to make it easier for comparing what is in Azure AD and Exchange Online, to then be used to determine if changes have been made (eg. user updated their profile photo).
Photo Checksum
For generating the hash of the profile photos I’m using Get-Hash from the Powershell Community Extensions. Whilst PowerShell has Get-FileHash I don’t want to write the profile photos out to disk and read them back in just to get the checksum. That slows the process up by 25%. You can get the checksum using a number of different methods and algorithms. Just be consistent and use the same method across both profile photos and you’ll be comparing apples with apples and the comparison logic will work.
Some notes on Photos and Exchange Online (and MFA)
This is where things went off on a number of tangents. Initially I tried accessing the photos using Exchange Online Remote PowerShell.
CAVEAT 1: If your Office365 Tenant is enabled for Multi-Factor Authentication (which it should be) you will need to get the Exchange Online Remote PowerShell Module as detailed here. Chances are you won’t have full Office365 Admin access though, so as long as the account you will be using is in the Recipient Management Role you should be able to go to the Exchange Control Panel using a URL like https://outlook.office365.com/ecp/?realm=<tenantname>&wa=wsignin1.0 where tenantname is something like customer.com.au From the Hybrid menu on in the right handside pane you will then be able to download the Microsoft.Online.CSE.PSModule.Client.application I had to use Internet Explorer to download the file and get it installed successfully. Once installed I used a few lines from this script here to load the Function and start my RPS session from within PowerShell ISE during solution development.
CAVEAT 2: The EXO RPS MFA PS Function doesn’t allow you to pass it your account password. You can pass it the identity you want to use, but not the password. That makes scheduled process automation with it impossible.
CAVEAT 3: The RPS session exposes the Get-UserPhoto cmdlet which is great. But the RPS session leverages the GraphAPI. The RPS PS Module doesn’t refresh it’s tokens, so if the import takes longer than 60 minutes then using this method you’re a bit stuffed.
CAVEAT 4: Using the Get-UserPhoto cmdlet detailed above, the syncing of photos is slow. As in I was only getting ~4 profile photos per minute slow. This also goes back to the token refresh issue as for pretty much any environment of the size I deal with, this is too slow and will timeout.
CAVEAT 5: You can whitelist the IP Address (or subnet) of your host so MFA is not required using Contextual IP Addressing Whitelisting. At that point there isn’t really a need to use the MFA Enabled PREVIEW EXO RPS function anyway. That said I still needed to whitelist my MIM Sync Server(s) from MFA to allow integration into the Graph API. I configured just the single host. The whitelist takes CIDR format so that looks like /32 (eg. 11.2.33.4/32)
Performance Considerations
As I mentioned above,
using the Get-UserPhoto cmdlet was slow. ~4 per minute slow
using the GraphAPI into Exchange Online and looking at each user and determining if they had a photo then downloading it, was also slow. Slow because at this customer only ~50% of their users have a photo on their mailbox. As such I was only able to retrieve ~145 photos in 25 minutes. *Note: all timings listed above were during development and actually outputting the images to disk to verify functionality.
Implemented Solution
After all my trial and error on this, here is myfinal approach and working solution;
Use the Exchange Online Remote PowerShell (non-MFA version) to query and return a collection of all mailboxes with an image *Note, add an exception for your MIM Sync host to the white-listed hosts for MFA (if your Office365 Tenant is enabled for MFA) so the process can be automated
Use the Graph API to obtain those photos
with this I was able to retrieve ~1100 profile photos in ~17* minutes (after ~2 minutes to query and get the list of mailboxes with a profile photo)
Pre-requisites
There’s a lot of info above, so let me summarize the pre-requisties;
The Granfeldt PowerShell MA
Whitelist your FIM/MIM Sync Server from MFA (if your Office 365 environment is enabled for MFA)
Add the account you will run the MA as, that will in turn connect to EXO via RPS to the Recipient Management Role
Create a WebApp for the PS MA to use to access users Profile Photos via the Graph API (fastest method)
Powershell Community Extensions to generate the image checksum
Creating the WebApp to access Office365 User Profile Photos
Go to your Azure Portal and select the Azure Active Directory Blade from the Resource Menu bar on the left. Then select App Registrations and from the Manage Section of the Azure Active Directory menu, and finally from the top of the main pane select “New Application Registration“.
Give it a name and select Web app/API as the type of app. Make the sign-in URL https://localhost and then select Create.
Record the ApplicationID that you see in the Registered App Essentials window. You’ll need this soon.
Now select All Settings => Required Permissions. Select Read all users basic profiles in addition to Sign in and read user profile. Select Save.
Under Required Permissions select Add and then select 1 Select an API, and select Office 365 Exchange Online then click Select.
Choose 2 Select Permissions and then select Read user profiles and Read all users’ basic profiles. Click Select.
Select Grant Permissions
From Settings select Keys, give your key a Description, choose a key lifetime and select Save.RECORD the key value. You’ll need this along with the WebApp ApplicationID/ClientID for the Import.ps1 script.
Using the information from your newly registered WebApp, we need to perform the first authentication (and authorization of the WebApp) to the Graph API. Taking your ApplicationID, Key (Client Secret) and the account you will use on on the Management Agent (and that you have assigned the Recipient Management Role in Exchange Online) and run the script detailed in this post here. It will authenticate you to your new WebApp via the GraphAPI after asking you to provide the account you will use on the MA and Authorizing the permissions you selected when registering the app. It will also create a refresh.token file which we will give to the MA to automate our connection. The Authorization dialog looks like this.
Creating the Management Agent
Now we can create our Management Agent using the Granfeldt PowerShell Management Agent. If you haven’t created one before checkout a post like this one, that further down the post shows the creation of a Granfeldt PSMA. Don’t forget to provide blank export.ps1 and password.ps1 files on the directory where you place the PSMA scripts.
PowerShell Management Agent Schema.ps1
PowerShell Management Agent Import.ps1
As detailed above the PSMA will leverage the WebApp to read users Exchange Profile Photos via the Graph API. The Import script also leverages Remote Powershell into Exchange Online (for reasons also detailed above). The account you run the Management Agent as will need to be added to the Recipient Management Role Group in order to use Remote PowerShell into Exchange Online and get the information required.
Take the Import.ps1 script below and update;
Update lines 11, 24 and 42 for the path to where you have put your PSMA. Mine is under the Extensions directory in a directory named EXOPhotos.
copy the refresh.token generated when authenticating and authorizing the WebApp earlier into the directory you specified in line 42 above.
Create a Debug directory under the directory you specified in lines 11,24 and 42 above so you can see what the MA is doing as you implement and debug it the first few times.
I’ve written the Import to use Paged Imports, so make sure you tick the Paged Imports checkbox on the configuration of the MA
Update Lines 79 and 80 with your ApplicationID and Client Secret that you recorded when creating your WebApp
Running the Exchange User Profile Photos MA
Now that you have created the MA, you should have select the EXOUser ObjectClass and the attributes defined in the schema. You should also create the EXOPhoto (as Binary) and EXOPhotoChecksum (as String) attributes in the Metaverse on the person ObjectType (assuming you are using the built-in person ObjectType).
Configure your flow rules to flow the EXOPhoto and EXOPhotoChecksum on the MA to their respective attributes in the MV.
Create a Stage Only run profile and run it. If you have done everything correctly you will see photos come into the Connector Space.
Looking at the Connector Space, I can see EXOPhoto and EXOPhotoChecksum have been imported.
After performing a Synchronization to get the data from the Connector Space into the Metaverse it is time to test the image that lands in the Metaverse. That is quick and easy via PowerShell and the Lithnet MIIS Automation PowerShell Module.
The file is output to the directory with the filename specified.
Opening the file reveals correctly my Profile Photo.
Summary
In Part one we got the AzureAD/Active Directory photo. In this post we got the Office365 photo.
Now that we have the images from Office365 we need to synchronize any update to photos to Active Directory (and in-turn via AADConnect to Azure AD). Keep in mind the image size limits for Active Directory and that we retrieved the largest photo available from Office365 when synchronizing the photo on. There are a number of PowerShell modules for photo manipulation that will allow you to resize accordingly.
Whilst Microsoft FIM/MIM can be used to do pretty much anything your requirements dictate, dealing with object types other than text and references can be a little tricky when manipulating them the first time. User Profile Photos fall into that category as they are stored in the directory as binary objects. Throw in Azure AD and obtaining and synchronizing photos can seem like adding a double back-flip to the scenario.
This post is Part 1 of a two-part post. Part two is here. This is essentially the introduction to the how-to piece before extending the solution past a users Active Directory Profile Photo to their Office 365 Profile Photo. Underneath the synchronization and method for dealing with the binary image data is the same, but the API’s and methods used are different when you are looking to implement the solution for any scale.
As for why you would want to do this, refer to Part two here. It details why you may want to do this.
Overview
As always I’m using my favourite PowerShell Management Agent (the Granfeldt PSMA). I’ve updated an existing Management Agent I had for Azure AD that is described here. I highly recommend you use that as the basis for the extra photo functionality that I describe in this post. Keep in mind the AzureADPreview, now AzureAD Powershell Module has change the ADAL Helper Libraries. I detail the changes here so you can get AuthN to work with the new libraries.
Therefore the changes to my previous Azure AD PowerShell MA are to add two additional attributes to the Schema script, and include the logic to import users profile photo (if they have one) in the Import script.
Schema.ps1
Take the schema.ps1 from my Azure AD PSMA here and add the following two lines to the bottom (before the $obj in the last line where I’ve left an empty line (29)).
The AADPhoto attribute of type Binary is where we will store the photo. The AADPhotoChecksum attribute of type String is where we will store a checksum of the photo for use in logic if we need to determine if images have changed easily during imports.
Add the following lines into the Import.ps1 in the section where we are creating the object to pass to the MA. After the$obj.Add(“AADCity”,$user.city)line is a good spot.
What the script below does is create a WebClient rather than use Invoke-RestMethod or Invoke-WebRequest to get the users Azure AD Profile image only if the ‘thumbnailPhoto@odata.mediaContentType’ attribute exists which indicates the user has a profile photo. I’m using the WebClient over the PowerShell Invoke-RestMethod or Invoke-WebRequest functions so that the returned object is in binary format (rather than being returned as a string), which saves having to convert it to binary or output to a file and read it back in. The WebClient is also faster for transferring images/data.
Once the image has been returned (line 8 below) the image is added to the object as the attribute AADPhoto to be passed to the MA (line 11)
Line 14 gets the checksum for the image and adds that to the AADPhotoChecksum attribute in line 16.
Other changes
Now that you’ve updated the Schema and Import scripts, you will need to;
Refresh your schema on your Azure AD PSMA to get the new attributes (AADPhoto and AADPhotoChecksum) added
Select the two new attributes in the Attributes section of your Azure AD PSMA
Create in your MetaVerse via the MetaVerse Designer two new attributes on the person (or whatever ObjectType you are using for users), for AADPhoto and AADPhotoChecksum. Make sure that AADPhoto is of type Binary and AADPhotoChecksum is of type string.
Configure your Attribute Flow on your Azure AD PSMA to import the AADPhoto and AADPhotoChecksum attributes into the Metaverse. Once done and you’ve performed an Import and Sync you will have Azure AD Photos in your MV.
How do you know they are correct ? Let’s extract one from the MV, write it to a file and have a look at it. This small script using the Lithnet MIIS Automation PowerShell Module makes it easy. First I get my user object from the MV. I then have a look at the text string version of the image (to make sure it is there), then output the binary version to a file in the C:\Temp directory.
Photos are still just bits of data. Once you know how to get them and manipulate them you can do what ever you need to with them. See Part two that takes this concept and extends it to Office 365.
Since that point in time I’ve found myself doing considerably more via PowerShell and the Graph API using oAuth. I regularly find myself leveraging previous scripts to generate a new script for the initial connection. To the point that I decided to make this simpler and provide a nice clean starting point for new scripts.
This blog post details a simple script to generate a couple of PowerShell Functions that can be the basis for integration with Graph API using PowerShell via a WebApp using oAuth2.
Overview
This script will request the necessary information required to call into the Graph API and establish a session. Specifically;
The API Endpoint. Historically there were many different API endpoints depending on what you are integrating with. Microsoft is moving to simplify this (great article here about the evolving API), but it is still a work in progress. For this example I’ll be using graph.microsoft.com which is where Microsoft are heading. If you need access to an API not currently on the Graph API see here to workout which API Endpoint fits your apps requirements. In short though typically all that changes between API’s is the Resource (API end-point) and the scope (what permissions your app will have). Variations to the primary Graph API endpoint is when you are integrating with applications such as OneNote (https://www.onenote.com/api), Office 365 Discovery Service (https://api.office.com/discovery/), One Drive etc.
The ClientID and the ClientSecret associated with your WebApp that you have registered in the Application Registration Portal
The Scope of the WebApp. To make it seamless this should be done via the WebApp registration in the Application Registration Portal and configured as part of the PowerShell web requests
Armed with this information the shell of a PowerShell script will be created that will;
Authenticate a user to Graph API via Powershell and oAuth 2.0
Request Authorization for the WebApp to access the Scope provided (if Admin approval scope is requested and the AuthN is performed by a non-admin an authorization failure message will appear detailing an Administrator must authorize).
Obtain and Authorization Code which will contain the Bearer Token and Refresh Token.
The Bearer token can be used to make Graph API calls for up to 1 hour.
The Refresh token will allow you to request a new token and allow your script to be used again to interact via Graph API without going through the Authentication process again.
The following graphic shows this flow.
Create/Register your Application
Go to the Application Registration Portal https://apps.dev.microsoft.com/ and sign in. This is the new portal for registering your apps. It will show any previous apps you registered within AzureAD and any of the new “Converged Apps” you’ve created via the new Application Registration Portal.
Select Add an app from the Converged applications list.
Give your app a name and select Create
Record the Application ID (previously known as the Client ID) and select Generate New Password.
You will be provided your Client Secret. Record this now as it is the only time you will see it. Select Ok.
By default you will get User.Read permissions on the API. That is enough for this sample. Depending on what you will do with the API you will probably need to come and change the permissions or do it dynamically via the values you supply the $resource setting in your API calls.
Select Platforms, select Web and add a reply URL of https://localhost
Scroll to the bottom of the Registration windows and select Save.
Generate your PowerShell Graph API oAuth Script
Copy the following script and put it into an Administrator PowerShell/PowerShell ISE session and run it.
It will ask you to choose a folder to output the resultant PowerShell Script to. You can create a new folder through this dialog window if require.
The script will prompt you for the Client/Application ID, Client Secret and the Reply URL you obtained when registering the Web App in the steps above.
The script will be written out to the folder you chose in the first step and it will be executed. It will prompt you to authenticate. Provide the credentials you used when you created the App in the Application Registration Portal.
You will be prompted to Authorize the WebApp. Select Accept
If you’ve executed the previous steps correctly you’ll receive an AuthCode in your PowerShell output window
You’ll then see the output for a sample query for your user account and below that the successful call for a refresh of the tokens.
Summary
In the folder you chose you will find a PowerShell script with the name Connect-to-Microsoft-Graph.ps1. You will also find a file named refresh.token. You can use the script to authenticate with your new app, but more simply use the Get-NewTokens function to refresh your tokens and then write your own API queries to your app using the tokens. Unless you change the scope you don’t need to run Get-AzureAuthN again. Just use Get-NewTokens before your API calls.
Change the scope of your app to get more information. If you add a scope that requires Admin consent (and you’re not an admin), when prompted to authenticate you will need to get an Admin to authenticate and authorize the scope. Because you’ve changed the scope you will need to run the Get-AzureAuthN function again after updating $scope (as per below) and the dependent $scopeEncoded.
As the screen shot below shows I added the Mail.Read permission. I changed the $scope in the script so that it reflected the changes e.g
I hope that makes getting started with the oAuth2 Graph API via PowerShell a lot simpler than it was for me initially, with the differing endpoints, evolving API and the associated documentation somewhere in-between.
One of the foundations of Identity Management is the ability to join an identity between disparate connected systems. As we extend our management of identities into cloud services this adds a few twists.
A key concept is to use an anchor that is persistent. Something that doesn’t change through a users life-cycle. A user’s Security IDentifier (SID) in Active Directory is perfect. It doesn’t change when a user or group may get renamed. What gets interesting is how the SID is represented when returned using different methods. That is what I quickly cover in this post.
Overview
The defacto method for connecting your OnPremise Active Directory to Azure Active Directory is to use Azure Active Directory Connect. AADC will synchronise users and groups SID’s to the corresponding object in AAD into the onPremisesSecurityIdentifier attribute.
When the onPremisesSecurityIdentifier attribute is retrieved via the GraphAPI the format looks like this: S-1-5-21-3878594291-2115959936-132693609-65242
Using an Active Directory Management Agent for FIM/MIM and synchronizing the objectSID from the OnPremise AD represents the value in the Metaverse in binary format which when viewed as text looks like this: 15000005210002431664623112825230126105190232781820
Translating SID formats so we can join identities
What we need to do is translate the string representation of the SID returned from the GraphAPI and AzureAD so that we have it in a binary format. Then we can then use those attributes in join rules to match users/groups between AzureAD and our OnPremise Active Directory.
On my AzureAD PowerShell Management Agent I have an attribute named AADonPremiseSID configured with the format as Binary in my PSMA Schema.ps1 as shown below.
On my Azure AD PSMA I have the following lines in my Import.ps1 which essentially takes the value retrieved from the GraphAPI S-1-5-21-3878594291-2115959936-132693609-65242 and converts it to a binary array that in text looks something like 15000005210002431664623112825230126105190232721825400 and stores it in the AADonPremiseSID binary attribute in the connector space.
# Create SID .NET object using SID string from AAD S-1-500-........
$sid = New-Object system.Security.Principal.SecurityIdentifier $user.onPremisesSecurityIdentifier
#Create a byte array for the length of the users SID
$BinarySid = new-object byte[]($sid.BinaryLength)
#Copy the binary SID into the byte array, starting at index 0
$sid.GetBinaryForm($BinarySid, 0)
#Add the SID to the user in the connector space
$obj.Add("AADonPremiseSID",$BinarySid)
This then lets me join my users (and groups using the same method) between AD and AAD. Essentially a line to put it into Security Identifier format, two lines to convert it to a binary array and a line to store it in the connector space. Simple when you don’t over think it.
I’m posting this because I know I’m going to need to do this often. Hope it helps someone else too.
