Consuming CSV files from an Exchange Mailbox via Exchange Web Services and FIM/MIM 2016 using the Granfeldt PowerShell MA

This solution on first look is quite random. A management agent that consumes a flat file (comma separated file) isn’t ground breaking, but when the twist is that the CSV file is in an email in an Exchange Inbox, it’s quite a different scenario.

Background

My customer uses a Cloud Service for their recruitment processes. The cloud service does have a SOAP API that I could potentially develop a FIM/MIM solution for using the Microsoft Web Services Management Agent, however my customer does not have API access to their tenant, the vendor isn’t overly responsive and I need a solution in days not weeks.

On the upside, my customer can quickly create reports in the SaaS Portal, and schedule them to be delivered (via CSV/Excel) to an email address. So, what if I was able to integrate FIM/MIM to the inbox that receives the emails with attached reports that contain the information I require and process it accordingly? This blog post is that solution.

Overview

Once a day there is a scheduled process that generates a report (CSV) of new staff from a SaaS provider. That CSV is emailed to an Inbox we created to receive these reports. Using the Granfeldt PowerShell Management Agent I created a solution that;

  • Connects to the specified Exchange Mailbox using Exchange Web Services
    • Enumerates the inbox looking for emails with attachments
    • Validates the emails with attachments by looking for the sender and attachment type we are expecting
    • Extracts the attachment to a file share
    • Moves all messages with attachments to a Processed subfolder
  • Processes the most recent report attachment (CSV) (in case the MA hasn’t run for few days or the reports start coming more than once a day) or if there is no new email message with attachment in the inbox, processes the most recent attachment we previously put in the file share
    • Each report is cumulative so the MA logic stays simple
  • Imports to MIM the new staff that are due to start in the next 7 days (to allow for all access to be setup prior to their first day of employment) and kicks off the MIM Provisioning processes
    • Triggers entitlements and access through the system accordingly (not covered in this post, but includes provisioning of mailbox, home directory, group memberships etc)

Notes:

  • The MIM Synchronisation Service Account will need access permissions to save files into the File Share
  • The MIM Server and this PSMA will require the Microsoft EWS 2.2 API to be installed on the MIM Synchronisation Server. It is available from here https://www.microsoft.com/en-us/download/details.aspx?id=42951

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

Three items I had to work out that I’ll save you the pain of are;

  • You must have a Password.ps1 and Export.ps1 file. Even though we’re not doing password management, or exporting back to the SaaS Provider on this MA, the PS MA configuration requires a file for these fields. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present.
  • The credentials you give the MA to run the scripts as, needs to be in the format of just ‘accountname’ NOT ‘domain\accountname’. I’m using the AD Account associated with the Exchange Mailbox that receives the emails with the CSV reports.
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~4\2010\SYNCHR~1\EXTENS~2\PageUp

Schema.ps1

My schema is essentially the columns that are in the CSV report that I’m importing.

Password Script (password.ps1)

Empty as described above

Import.ps1

Connect to the Exchange Mailbox, find messages from the defined user sending them where the attachment is of the expected naming and format. Extract the CSV file to a File Share. Move emails with attachments to a processed folder. Parse the CSV, perform some logic on the data and import objects and values for new employees.

Export.ps1

Empty as we’re not writing anything back to the SaaS provider.

Wiring it all together

In order to wire the functionality all together there are the usual number of configuration steps to be completed. Below I’ve shown a number of the key points associated with making it all work. This is all Synchronisation Engine MA configuration tasks. Basically create the PS MA, import attributes from the PS MA, create your MA Run Profiles and let it loose.

Management Agent Configuration

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using the same service account as my AD MA.

Password and Export scripts must be specified but as we’re not doing password management or exporting they’re empty as detailed above.

If your schema.ps1 file is formatted correctly, you can select your attributes/columns that will be coming in from the CSV file.

My join rule is simple. StaffID to AccountName in the MetaVerse.

My import flows are direct flows with a Boolean flag to kick off a bunch of declarative rules out of the Portal.

Summary

Thinking outside of the box and using the Granfeldt PowerShell MA I was able to quickly consume a CSV file from an Exchange Inbox to kick off the provisioning process.

Follow Darren on Twitter @darrenjrobinson

Provision Users for Exchange with FIM/MIM 2016 using the Granfeldt PowerShell MA, avoiding the AD MA (no-start-ma) error

Forefront / Microsoft Identity Manager provides Exchange Mailbox provisioning out of the box on the Active Directory Management Agent. I’ve used it in many many implementations over the years. However, in my first MIM 2016 implementation in late 2015 I ran into issues with something I’d done successfully many times before.

I was getting “no-start-ma” on the AD MA on export to AD. The point at which the MA sets up its connection to the Exchange environment. After some searching I found Thomas’s blog detailing the problem and a solution. In short update the MIM Sync Server to .NET 4.6. For me this was no-joy. However when MS released the first rollup update for MIM in December everything fired up and worked as normal.

Step forward a month as I was finalising development for the MIM solution I was building for my customer and my “no-start-ma” error was back when I re-enabled mailbox provisioning. Deselect the Exchange Provisioning option on the AD MA and all is good. Re-enable it and it fails. One week left of dev and I need mailbox provisioning so time for a work around whilst I lodge a Premier Support ticket.

So how can I get mailbox provisioning working reliably and quickly? I was already using Søren Granfeldt’s PowerShell MA for managing users Terminal Services configuration, Home Directories and Lync/Skype for Business. What’s one more. Look out for blog posts on using the PS MA to perform those other functions that I’ll be posting in the coming weeks.

Using the Granfeldt PowerShell Management Agent to Provision Exchange Mailboxes

In this blog post I’ll document how you can enable Mailbox Provisioning in Exchange utilising Søren Granfeldt’s extremely versatile PowerShell Management Agent. I’ll show you how to do the minimum of enabling a user with a mailbox. Understanding how this is done you can then easily then extend the functionality for lifecycle management (e.g. change account settings for POP/IMAP/ActiveSync and de-provisioning).

My Exchange PS MA is used in conjunction with an Active Directory MA and Declarative Provisioning Rules in the MIM Portal. Essentially all the AD MA does, when you enable Exchange Provisioning (when it works) is call the ‘update-recipient’ cmdlet to finish of the mailbox provisioning. My Exchange PSMA does the same thing.

Overview

There are three attributes you need to supply values for in order to then provision them a mailbox (on top of having an Active Directory account, or course);

  • mailNickName
  • homeMDB
  • homeExchangeServerName

The later two I’m flowing the appropriate values for using my Active Directory MA. I’m setting those attributes on the AD MA as I’m provisioning the AD account on that MA which then lets me set those two attributes as initial flow only. I’m doing that as over time it is highly likely that those attribute values may change with normal business as usual messaging admin tasks. I don’t want my Exchange MA stomping all over them.

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different. Configuration tasks like adding additional attributes the User Object Class in the MIM Portal, updating MPR’s, flow rules, Workflows, Sets etc are assumed knowledge and if not is easily Bing’able for you to work it out.

Three items I had to work out that I’ll save you the pain of are;

  • You must have a Password.ps1 file. Even though we’re not doing password management on this MA, the PS MA configuration requires a file for this field. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present
  • The credentials you give the MA to run the scripts as, needs to be in the format of just ‘accountname’ NOT ‘domain\accountname’. I’m using the service account that I’ve used for the Active Directory MA. The target system is the same directory service and the account has the permissions required (you’ll need to add the management agent account to the appropriate Exchange role group for user management)
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~4\2010\SYNCHR~1\EXTENS~2\Exchange

Schema Script (schema.ps1)

As I’m using the OOTB (out of the box) Active Directory MA to provision the AD account and only showing mailbox provisioning, the schema only consists of the attributes needed to know the state of the user with respect to enablement and the attributes associated with enabling and confirming a user for a mailbox.

https://gist.github.com/darrenjrobinson/ae46cdfccb825dce69b3

Password Script (password.ps1)

Empty as described above.

Import Script (Import.ps1)

Import values for attributes defined in the schema.

Export Script (Export.ps1)

The business part of the MA. Take the mailnickName attribute value flowed from FIM, (the other required attributes are populated via the AD MA) and call update-recipient to provision the mailbox.

Wiring it all together

In order to wire the functionality all together there are the usual number of configuration steps to be completed. Below I’ve shown a number of the key points associated with making it all work.

Basically create the PS MA, import attributes from the PS MA, add any additional attributes to the Portal Schema, update the Portal Filter to allow Administrators to use the attribute, update the Synchronisation MPR to allow the Sync Engine to flow in the new attribute, create the Set used for the transition, create your Synchronisation Rule, create your Mailbox Workflow, create your Mailbox MPR, create your MA Run Profiles and let it loose.

Management Agent Configuration

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using the same service account as my AD MA.

Password script must be specified but as we’re not doing password management its empty as detailed above.

If your schema.ps1 file is formatted correctly you can select your attributes.

My join rule is simple. AccountName (which as you’ll see in the Import.ps1 is aligned with sAMAccountName) to AccountName in the MetaVerse.

My import flows are a combination of logic used for other parts of my solution, a Boolean flag & Mailbox GUID to determine if the user has a mailbox or not (used for my Transition Set and my Export script).

Below is my rules extension that sets a boolean value in the MV and then flowed to the MIM Portal that I use in my Transition Set to trigger my Synchronisation Rule.

Synchronisation Rules

My Exchange Outbound Sync rule doesn’t and isn’t complex. All it is doing is sync’ing out the mailnickName attribute and applying the rule based on an MPR, Set and Workflow.

For this implementation my outbound attribute flow for mailnickName is a simple firstname.lastname format.

Set

I have a Set that I use as a ‘transition set’ to trigger provisioning to Lync. My Set looks to see if the user account exists in AD (I flow in the AD DN to an attribute in the Portal) and the mailbox status (set by the Advanced Flow Rule shown above). I also have (not shown in the screenshot) a Boolean attribute in the MIM Portal that is set based on an advanced flow rule on the AD MA that has some logic to determine if employment date as sourced from my HR Management Agent is current and the user should be active or not).

Workflow

An action based workflow that will use the trigger the Synchronisation rule for Exchange Mailbox creation.

MPR

Finally my MPR for provisioning mailboxes is based on the transition set,

and my Mailbox Workflow.

Summary

Using the Granfeldt PowerShell MA I was able to quickly abstract Mailbox Provisioning from the AD Management Agent and perform the functionality on its own MA.

 

Follow Darren on Twitter @darrenjrobinson