Consuming CSV files from an Exchange Mailbox via Exchange Web Services and FIM/MIM 2016 using the Granfeldt PowerShell MA

This solution on first look is quite random. A management agent that consumes a flat file (comma separated file) isn’t ground breaking, but when the twist is that the CSV file is in an email in an Exchange Inbox, it’s quite a different scenario.

Background

My customer uses a Cloud Service for their recruitment processes. The cloud service does have a SOAP API that I could potentially develop a FIM/MIM solution for using the Microsoft Web Services Management Agent, however my customer does not have API access to their tenant, the vendor isn’t overly responsive and I need a solution in days not weeks.

On the upside, my customer can quickly create reports in the SaaS Portal, and schedule them to be delivered (via CSV/Excel) to an email address. So, what if I was able to integrate FIM/MIM to the inbox that receives the emails with attached reports that contain the information I require and process it accordingly? This blog post is that solution.

Overview

Once a day there is a scheduled process that generates a report (CSV) of new staff from a SaaS provider. That CSV is emailed to an Inbox we created to receive these reports. Using the Granfeldt PowerShell Management Agent I created a solution that;

• Connects to the specified Exchange Mailbox using Exchange Web Services
  • Enumerates the inbox looking for emails with attachments
  • Validates the emails with attachments by looking for the sender and attachment type we are expecting
  • Extracts the attachment to a file share
  • Moves all messages with attachments to a Processed subfolder
• Processes the most recent report attachment (CSV) (in case the MA hasn’t run for few days or the reports start coming more than once a day) or if there is no new email message with attachment in the inbox, processes the most recent attachment we previously put in the file share
  • Each report is cumulative so the MA logic stays simple
• Imports to MIM the new staff that are due to start in the next 7 days (to allow for all access to be setup prior to their first day of employment) and kicks off the MIM Provisioning processes
  • Triggers entitlements and access through the system accordingly (not covered in this post, but includes provisioning of mailbox, home directory, group memberships etc)

Notes:

• The MIM Synchronisation Service Account will need access permissions to save files into the File Share
• The MIM Server and this PSMA will require the Microsoft EWS 2.2 API to be installed on the MIM Synchronisation Server. It is available from here https://www.microsoft.com/en-us/download/details.aspx?id=42951

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

Three items I had to work out that I’ll save you the pain of are;

• You must have a Password.ps1 and Export.ps1 file. Even though we’re not doing password management, or exporting back to the SaaS Provider on this MA, the PS MA configuration requires a file for these fields. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present.
• The credentials you give the MA to run the scripts as, needs to be in the format of just ‘accountname’ NOT ‘domain\accountname’. I’m using the AD Account associated with the Exchange Mailbox that receives the emails with the CSV reports.
  
• The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~4\2010\SYNCHR~1\EXTENS~2\PageUp
  

Schema.ps1

My schema is essentially the columns that are in the CSV report that I’m importing.

See the gist on github.

Password Script (password.ps1)

Empty as described above

See the gist on github.

Import.ps1

Connect to the Exchange Mailbox, find messages from the defined user sending them where the attachment is of the expected naming and format. Extract the CSV file to a File Share. Move emails with attachments to a processed folder. Parse the CSV, perform some logic on the data and import objects and values for new employees.

See the gist on github.

Export.ps1

Empty as we’re not writing anything back to the SaaS provider.

See the gist on github.

Wiring it all together

In order to wire the functionality all together there are the usual number of configuration steps to be completed. Below I’ve shown a number of the key points associated with making it all work. This is all Synchronisation Engine MA configuration tasks. Basically create the PS MA, import attributes from the PS MA, create your MA Run Profiles and let it loose.

Management Agent Configuration

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using the same service account as my AD MA.

Password and Export scripts must be specified but as we’re not doing password management or exporting they’re empty as detailed above.

If your schema.ps1 file is formatted correctly, you can select your attributes/columns that will be coming in from the CSV file.

My join rule is simple. StaffID to AccountName in the MetaVerse.

My import flows are direct flows with a Boolean flag to kick off a bunch of declarative rules out of the Portal.

Summary

Thinking outside of the box and using the Granfeldt PowerShell MA I was able to quickly consume a CSV file from an Exchange Inbox to kick off the provisioning process.

Follow Darren on Twitter @darrenjrobinson