A Twitter Management Agent for Microsoft Identity Manager

In the last couple of weeks I’ve been evaluating a number of different approaches/concepts for some upcoming MIM development projects. Some of these I’ve blogged about already.

Having an Identity Manager Metaverse with identity data is a key dependency to being able to validate ideas and concepts. So what’s a good source of some interesting and varied identity data with string, integer, reference, and boolean attributes? Twitter? Yeah why not. There’s an API. Should be pretty quick to get some sample data right?

In this blog post I’m going to give an overview of creating a PowerShell Management Agent to consume Twitter identities and their data into Microsoft Identity Manager. I’ll cover;

  • Obtaining Twitter user data from Twitter using Powershell and the Twitter RestAPI
  • Using Søren’s Powershell Management Agent to import Twitter user data obtained via the RestAPI
  • Manipulating the Twitter data into the MIM Synchronisation Engine

Twitter Data

Here’s an overview of my approach/rationale of what data I was looking for and how I got it from Twitter;

  • I don’t need real-time data. Just identity data
  • I need data of all different data-types
  • I need data with all the randomness that identity data often contains
  • I created a standalone script that took a seed Twitter identity (one of my accounts) and;
    • obtained the Twitter account info including the list of the Twitter accounts it followed
    • the Twitter accounts that follows it
  • The standalone script uses the Twitter RestAPI to obtain the data and respected the service and rate-limits
  • To make the Twitter API calls easy I leveraged the awesome InvokeTwitterAPI Powershell Module from Shannon Conley & Mehmet Kaya available here https://github.com/MeshkDevs/InvokeTwitterAPIs . I notice that there is an updated version from Marc R Kellerman available here https://www.powershellgallery.com/packages/InvokeTwitterAPIs/2.1/Content/InvokeTwitterAPIs.psm1 that was released after I had done most of my work. Notably it supports having multiple OAuth keys and the ratelimit restrictions. The details below leverage this updated version.

Here is what a sample of some of the data looks like in the Metaverse.

DataSample1.PNG

Pre-requisites

You need to enable your Twitter Account for API access. Follow the details here 

Getting the Seed Twitter account info

By now you should have downloaded the Twitter PowerShell API Modules and installed them. If you haven’t get WMF5 installed and run the install-module command as shown below in Line 1.

Modify the script below to give the;

  • Seed Twitter Account you want to bring in the Friends and Followers for as users into the MIM Metaverse
  • The API keys asscoiated with your Twitter account(s) you’re going to use to query the Twitter API
  • The directory you want to dump the account info out to

….. and let it loose.

Now we have two XML files with a whole bunch of Twitter accounts and their metadata. There is an almost certainty that the seed account you’ve used is both followed by twitter accounts that you also follow. We’re going to need to remove the duplicates so that when we import the Twitter accounts into MIM we don’t have duplicates.

Basic, basic script to read in both files and spit out the unique Twitter Accounts is shown below.

Using the Granfeldt PowerShell Management Agent to import Twitter Identities

Consuming data into the MIM Sync Engine obtained via PowerShell is quick and simple utilising Søren Granfeldt’s extremely versatile PowerShell Management Agent. I’m just going to cover importing the data from the XML file we generated above.

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

A few items of note are;

  • You must have a Password.ps1 file. Even though we’re not doing password management on this MA, the PS MA configuration requires a file for this field. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present
  • Same for an Export.ps1 file. I’m not doing any exports on the MA, but an export script must be present.
  • The credentials you give the MA to run this MA are irrelevant as they aren’t used as part of the import as I’m bringing in data from files generated via separate PowerShell scripts
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~2\2010\SYNCHR~1\EXTENS~2\Twitter

Import Twitter Users into Microsoft Identity Manager

Using the guidance above on the Granfeldt PSMA here are the two key scripts for the Twitter MA.

The Schema Script to expose the core Twitter identity attributes.

Schema Script

Import Script

The Import Script that takes the rationalised XML file created earlier from the friends and followers queries and populates the connector space.

Password Script

Required by the PSMA but not used as detailed earlier

Export Script

Required by the PSMA but not used as detailed earlier

Creating the Management Agent

Path to the Schema Script in 8.3 format as detailed earlier.

MA1

Path to the Import, Export and Password scripts also in 8.3 format.

MA2

Select the attributes you want to bring in to the connector space.

MA3

Any Join logic, and a Projection Rule.

MA4

Import Flow Rules to bring in the Tweeters.

MA5

Create your Run Profiles, perform a Stage and Full Sync and BAM; Tweeters in the Metaverse. Real Word Identity Data Ahoy. Exception testing commences now.

Follow Darren on Twitter @darrenjrobinson

Adding/Removing User Office365 Licences using PowerShell and the Azure AD Graph RestAPI

In a recent blog post here I posted about the Azure AD v2.0 Preview Powershell cmdlets that are currently in preview. These update the functionality the current MSOL cmdlets provide whilst also supporting features they don’t (such as managing users with MFA).

The Azure AD v2.0 cmdlets interface with the Azure AD Graph API and this week I tried using the Set-AzureADUserLicense cmdlet to add/remove licenses from users in a test tenant. With no sample documentation for syntax I didn’t kick any goals so I figured I’d just go straight to using the Azure AD Graph API to get the job done direct from Powershell instead.

In this post I’m going to show you how to add/remove Office365 licenses from users using PowerShell and the Azure AD Graph API.

As per my other post linked above if you’ve installed the Azure AD Preview Powershell module you’ll have the Microsoft.IdentityModel.Clients.ActiveDirectory.dll which we can leverage via Powershell to then connect to the Azure AD Graph API. Chances are you’ll have Microsoft.IdentityModel.Clients.ActiveDirectory.dll though if you also have the AzureRM Modules installed or TFS. Just search your Program Files sub-directories.

# the default path to where the Azure AD Preview PS Module puts the Libs
'C:\Program Files\WindowsPowerShell\Modules\AzureADPreview\1.1.143.0\Microsoft.IdentityModel.Clients.ActiveDirectory.dll'
# TFS Path
'C:\Program Files\Common Files\microsoft shared\Team Foundation Server\14.0\Microsoft.IdentityModel.Clients.ActiveDirectory.dll'
# Azure RM Cmdlets
'C:\Program Files\WindowsPowerShell\Modules\AzureRM.ApiManagement\1.1.2\Microsoft.IdentityModel.Clients.ActiveDirectory.dll'

Below you’ll find what you need to script your connection to Azure AD via the GraphAPI using the dll discussed above. Change $tenantID $username and $password to reflect you tenant and credentials.

Now that we’ve authenticated let’s enumerate our licenses. Bascially we make a RestAPI call to “https://graph.windows.net/{0}/subscribedSkus?api-version=1.6” as below. The particular license I want to add in my tenant is ExchangeStandard_Student. A key difference from the MSOL cmdlets though is adding or removing a license via the Graph API we reference the License skuId rather than skuPartNumber. The last line gets the skuId for my license.

Now to find the users that need to have the license assigned. Below in line 4 I search for users that are account enabled using this URI “https://graph.windows.net/{0}/users?$filter=accountEnabled eq true &api-version=1.6″  You can modify the filter for your criteria.

I then iterate through the users and find the users that aren’t assigned the license identified earlier above. I also exclude the AADConnect account for this tenant.

The GraphAPI expects the body with the info of licenses to be added or removed in a hashtable. So create the hashtable and convert it to JSON which is the format for Azure AD Graph API requires. Then for each of the unlicensed users we call the Azure AD GraphAPI URI “https://graph.windows.net/myorganization/users/$usertolicense`/assignLicense?api-version=1.6” and assign the license . In my environment it processed users at just over 1 user/sec.

A similar approach to remove licenses from users. A subtle difference is you only need to specify the skuId for removal as shown below. Note: you can add and remove licenses in the same call (if say you’re switching users over from one license plan to another).

All together for quick copy and paste. Update it for your tenant and creds. And remark out the Add or Remove depending on what you’re looking to do. If you’re looking to add/remove multiple licenses just add them to the hashtable/array.

Follow Darren on Twitter @darrenjrobinson

Goodbye Set-MsolUser, Hello Set-AzureADUser & Azure Graph API

Recently Microsoft released the preview of the v2.0 Azure AD PowerShell cmdlets. https://azure.microsoft.com/en-us/updates/azure-ad-new-powershell-cmdlets-preview/

I’ve got a project coming up where I’m looking to change my approach for managing users in Azure using Microsoft Identity Manager. Good timing to do a quick proof of concept to manage users with the new cmdlets and directly using the Graph API in preparation to move away from the msol cmdlets.

New Modules

First up, the Azure AD v2.0 PowerShell module was released in public preview on July 13, 2016. There will likely be changes before they become GA, so keep that in mind.

The v2.0 Azure AD PowerShell Module modules themselves are available for download from here https://www.powershellgallery.com/packages/AzureADPreview/1.1.143.0

If you have Windows Management Framework v5 installed you can download and install from PowerShell (as below).

Once installed, pretty quickly you can import the module, authenticate to your tenant, retrieve a user and update a few attributes (as below).

Whilst functional it doesn’t really work for how we need to interact with Azure from an Identity Management perspective. So how can we still use PowerShell but enumerate and manipulate identities in Azure ?

Now that we have the AzureAD v2.0 module installed we can reference the Active Directory library it installs (Microsoft.IdentityModel.Clients.ActiveDirectory.dll), authenticate to our Tenant retrieve users, and update them. That’s exactly what is shown in the commands below.

Where interacting with the GraphAPI directly really shines however is at the directory services layer and the Differential Query functionality.  https://msdn.microsoft.com/en-us/Library/Azure/Ad/Graph/howto/azure-ad-graph-api-differential-query

As such this is the approach that I’ll be taking for integration of Azure with Microsoft Identity Manager for managing users for entitlements (such as Azure licensing).
I hope this though also saves a few people time in working out how to use PowerShell to manage Azure objects via the Graph API (using both the PowerShell Module or via the RestAPI).

PowerShell error “Run Login-AzureRmAccount to login.” in AzureRM when already logged in

Usually when I’m writing PowerShell scripts I do it from a development virtual machine with a known environment state. However, yesterday I was trying to do something simple and quick and was writing it on my everyday laptop.

My script was using Windows Management Framework 5.0 and I was creating a new burn environment in AzureRM. I was authenticated and could query and enumerate most of my AzureRM environment, however I was getting erroneous responses on some cmdlets and was unable to create a new resource group. Essentially whenever I tried to perform anything of value to what I was trying to achieve PowerShell would return ”Run Login-AzureRmAccount to login.”

I was authenticated and all looks as it should.

LoggedIn

Query ARM RG and get the error, ”Run Login-AzureRmAccount to login.”

LoginError

I started digging to find out what my environment was looking like. Did I have WMF5 installed?  $PSVersionTable showed I did and all looked as it should.

PSEnv

What modules did I have installed? Get-InstalledModule | Out-GridView

PSModules

Lots of differing versions is what I saw *Note: screenshot above is after resolving my issue.

So what fixed my issue? I ran Update-Module and watched PowerShell update my modules and get my environment back into spec. A restart of my laptop and back in action.

What caused my issue? I’m thinking I updated an individual module at some point and dependencies were missed.

Hope this helps someone else and saves some of those “this should just be working, why isn’t it” moments.

Consuming CSV files from an Exchange Mailbox via Exchange Web Services and FIM/MIM 2016 using the Granfeldt PowerShell MA

This solution on first look is quite random. A management agent that consumes a flat file (comma separated file) isn’t ground breaking, but when the twist is that the CSV file is in an email in an Exchange Inbox, it’s quite a different scenario.

Background

My customer uses a Cloud Service for their recruitment processes. The cloud service does have a SOAP API that I could potentially develop a FIM/MIM solution for using the Microsoft Web Services Management Agent, however my customer does not have API access to their tenant, the vendor isn’t overly responsive and I need a solution in days not weeks.

On the upside, my customer can quickly create reports in the SaaS Portal, and schedule them to be delivered (via CSV/Excel) to an email address. So, what if I was able to integrate FIM/MIM to the inbox that receives the emails with attached reports that contain the information I require and process it accordingly? This blog post is that solution.

Overview

Once a day there is a scheduled process that generates a report (CSV) of new staff from a SaaS provider. That CSV is emailed to an Inbox we created to receive these reports. Using the Granfeldt PowerShell Management Agent I created a solution that;

  • Connects to the specified Exchange Mailbox using Exchange Web Services
    • Enumerates the inbox looking for emails with attachments
    • Validates the emails with attachments by looking for the sender and attachment type we are expecting
    • Extracts the attachment to a file share
    • Moves all messages with attachments to a Processed subfolder
  • Processes the most recent report attachment (CSV) (in case the MA hasn’t run for few days or the reports start coming more than once a day) or if there is no new email message with attachment in the inbox, processes the most recent attachment we previously put in the file share
    • Each report is cumulative so the MA logic stays simple
  • Imports to MIM the new staff that are due to start in the next 7 days (to allow for all access to be setup prior to their first day of employment) and kicks off the MIM Provisioning processes
    • Triggers entitlements and access through the system accordingly (not covered in this post, but includes provisioning of mailbox, home directory, group memberships etc)

Notes:

  • The MIM Synchronisation Service Account will need access permissions to save files into the File Share
  • The MIM Server and this PSMA will require the Microsoft EWS 2.2 API to be installed on the MIM Synchronisation Server. It is available from here https://www.microsoft.com/en-us/download/details.aspx?id=42951

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

Three items I had to work out that I’ll save you the pain of are;

  • You must have a Password.ps1 and Export.ps1 file. Even though we’re not doing password management, or exporting back to the SaaS Provider on this MA, the PS MA configuration requires a file for these fields. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present.
  • The credentials you give the MA to run the scripts as, needs to be in the format of just ‘accountname’ NOT ‘domain\accountname’. I’m using the AD Account associated with the Exchange Mailbox that receives the emails with the CSV reports.
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~4\2010\SYNCHR~1\EXTENS~2\PageUp

Schema.ps1

My schema is essentially the columns that are in the CSV report that I’m importing.

Password Script (password.ps1)

Empty as described above

Import.ps1

Connect to the Exchange Mailbox, find messages from the defined user sending them where the attachment is of the expected naming and format. Extract the CSV file to a File Share. Move emails with attachments to a processed folder. Parse the CSV, perform some logic on the data and import objects and values for new employees.

Export.ps1

Empty as we’re not writing anything back to the SaaS provider.

Wiring it all together

In order to wire the functionality all together there are the usual number of configuration steps to be completed. Below I’ve shown a number of the key points associated with making it all work. This is all Synchronisation Engine MA configuration tasks. Basically create the PS MA, import attributes from the PS MA, create your MA Run Profiles and let it loose.

Management Agent Configuration

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using the same service account as my AD MA.

Password and Export scripts must be specified but as we’re not doing password management or exporting they’re empty as detailed above.

If your schema.ps1 file is formatted correctly, you can select your attributes/columns that will be coming in from the CSV file.

My join rule is simple. StaffID to AccountName in the MetaVerse.

My import flows are direct flows with a Boolean flag to kick off a bunch of declarative rules out of the Portal.

Summary

Thinking outside of the box and using the Granfeldt PowerShell MA I was able to quickly consume a CSV file from an Exchange Inbox to kick off the provisioning process.

Follow Darren on Twitter @darrenjrobinson

Managing SharePoint Online (SPO) User Profiles with FIM/MIM 2016 and the Granfeldt PowerShell MA

Forefront / Microsoft Identity Manager does not come with an out-of-the-box management agent for managing SharePoint Online.

Whilst the DirSync/AADConnect solution will allow you to synchronise attributes from your On Premise Active Directory to AzureAD, SharePoint only leverages a handful of them. It then has its own set of attributes that it leverages. Many are similarly named to the standard Azure AD attributes but with the SPS- prefix.

For example, here is a list of SPO attributes and a couple of references to associated Azure AD attributes;

  • UserProfile_GUID
  • SID
  • SPS-PhoneticFirstName
  • SPS-PhoneticLastName
  • SPS-PhoneticDisplayName
  • SPS-JobTitle
  • SPS-Department
  • AboutMe
  • PersonalSpace
  • PictureURL
  • UserName
  • QuickLinks
  • WebSite
  • PublicSiteRedirect
  • SPS-Dotted-line
  • SPS-Peers
  • SPS-Responsibility
  • SPS-SipAddress
  • SPS-MySiteUpgrade
  • SPS-ProxyAddresses
  • SPS-HireDate
  • SPS-DisplayOrder
  • SPS-ClaimID
  • SPS-ClaimProviderID
  • SPS-ClaimProviderType
  • SPS-SavedAccountName
  • SPS-SavedSID
  • SPS-ResourceSID
  • SPS-ResourceAccountName
  • SPS-ObjectExists
  • SPS-MasterAccountName
  • SPS-PersonalSiteCapabilities
  • SPS-UserPrincipalName
  • SPS-O15FirstRunExperience
  • SPS-PersonalSiteInstantiationState
  • SPS-PersonalSiteFirstCreationTime
  • SPS-PersonalSiteLastCreationTime
  • SPS-PersonalSiteNumberOfRetries
  • SPS-PersonalSiteFirstCreationError
  • SPS-DistinguishedName
  • SPS-SourceObjectDN
  • SPS-FeedIdentifier
  • SPS-Location
  • Certifications
  • SPS-Skills
  • SPS-PastProjects
  • SPS-School
  • SPS-Birthday
  • SPS-Interests
  • SPS-StatusNotes
  • SPS-HashTags
  • SPS-PictureTimestamp
  • SPS-PicturePlaceholderState
  • SPS-PrivacyPeople
  • SPS-PrivacyActivity
  • SPS-PictureExchangeSyncState
  • SPS-TimeZone
  • SPS-EmailOptin
  • OfficeGraphEnabled
  • SPS-UserType
  • SPS-HideFromAddressLists
  • SPS-RecipientTypeDetails
  • DelveFlags
  • msOnline-ObjectId
  • SPS-PointPublishingUrl
  • SPS-TenantInstanceId

My customer has AADConnect in place that is synchronising their On Premise AD to Office 365. They also have a MIM 2016 instance that is managing user provisioning and lifecycle management. I’ll be using that MIM 2016 instance to manage SPO User Profile Attributes.

The remainder of this blog post describes the PS MA I’ve developed to manage the SPO attributes to allow their SPO Online Forms etc to leverage business and organisation user metadata.

Using the Granfeldt PowerShell Management Agent to manage SharePoint Online User Profiles

In this blog post I detail how you can synchronise user attributes from your On Premise Active Directory to an associated users SharePoint Online user profile utilising Søren Granfeldt’s extremely versatile PowerShell Management Agent. Provisioning and licensing of users for SPO is performed in parallel by the DirSync/AADConnect solution. This solution just provides attribute synchronisation to SPO User Profile attributes.

Overview

In this solution I’m managing the attributes that are pertinent to the customer. If you need an additional attribute or you have created custom attributes it is easy enough to extent.

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

Three items I had to work out that I’ll save you the pain of are;

  • You must have a Password.ps1 file. Even though we’re not doing password management on this MA, the PS MA configuration requires a file for this field. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present
  • The credentials you give the MA to run this MA are the credentials for the account that has permissions to manage SharePoint Online User Profiles. More detail on that further below.
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~4\2010\SYNCHR~1\EXTENS~2\SPO

Managing SPO User Profiles

In order to use this working example there are a couple of items to note;

  • At the top of the Import and Export scripts you’ll need to enter your SPO Tenant Admin URL. If your tenant URL is ‘CORP.sharepoint.com’ then at the top of the scripts enter ‘CORP-admin.sharepoint.com’. The Import script will work with corp.sharepoint.com but the export won’t.
  • Give the account you’re using to connect to SPO via your MIM permissions to manage/update SPO User Profiles

Schema.ps1

As mentioned above I’m only syncing attributes pertinent to my customers’ requirements. That said I’ve selected a number of attributes that are potentials for future requirements.

Password Script (password.ps1)

Empty as described above

Import.ps1

A key part of the import script is connecting to SPO and accessing the full User Profile. In order to do this, you will need to install the SharePoint Online Client Components SDK. It’s available for download here https://www.microsoft.com/en-us/download/details.aspx?id=42038

The import script then imports two libraries that give us access to the SPO User Profiles.

Import-Module
‘C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.UserProfiles.dll’

Import-Module
‘C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll’

Import values for attributes defined in the schema.

Export.ps1

The business part of the MA. Basically enough to take attribute value changes from the MV to the SPO MA and export them to SPO. In the example script below I’m only exporting three attributes. Add as many as you need.

Wiring it all together

In order to wire the functionality together, I’m doing it just using the Sync Engine MA configuration as we’re relying on AADConnect to create the users in Office365, and we’re just flowing through attribute values.

Basically, create the PS MA, create your MA Run Profiles, import users and attributes from the PS MA, validate your joins and Export to update SPO attributes as per your flow rules.

Management Agent Configuration

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using the Office 365 account we gave permissions to manage user profiles in SPO earlier.

Password script must be specified but as we’re not doing password management it’s empty as detailed above.

If your schema.ps1 file is formatted correctly you can select your attributes.

I have a few join rules. In the pre-prod environment though I’m joining on WorkEmail => mail.

My import flow is just bringing back in users mobile numbers that users are able to modify in SPO. I’m exporting Title, Location and Department to SPO.

Summary

Using the Granfeldt PowerShell MA it was very easy to manage user SharePoint Online User Profile attributes.

Follow Darren on Twitter @darrenjrobinson

HRESULT: 0x8023063D when attempting to run multiple Sync Run Profiles in MIM/FIM after applying rollup build 4.3.2124.0

A new hotfix rollup was released on the 11th of March Microsoft Identity Manager contains a number of fixes and some new functionality.

It appears that it also contains a new bug. Information about this came to my attention from Ryan Newington

The bug kicks in if you’re trying to run sync sequences on multiple MA’s simultaneously. It throws the error; “Unable to run the management agent.” Exception from HRESULT: 0x8023063D

The screenshot below shows the error when attempting to run a Full Synchronization on an MA when another MA is already running a Full Synchronization.

Note: You CAN still run Stage (Full Import) sequences on multiple MA’s simultaneously.

After rolling back my MIM Sync Server (to 4.3.2064.0, the snapshot prior to applying the 4.3.2124.0 rollup) I can again run multiple sync sequences across multiple management agents simultaneously.

Summary

If your run sequences include running multiple sync jobs running simultaneously don’t update your MIM/FIM Synchronisation Server to 4.3.2124.0.

Microsoft, can we have a fix please.

Follow Darren on Twitter @darrenjrobinson

Creating Microsoft Identity Manger (MIM) Run Profiles using PowerShell (post MIM rollup build 4.3.2124.0)

A new hotfix rollup was released on the 11th of March for Microsoft Identity Manager that contains a number of fixes and some new functionality.

One new feature according to the release notes is a new cmdlet Add-MIISADMARunProfileStep

This cmdlet allows the creation of MIM Synchronisation Management Agent Run Profiles using PowerShell.

From the MS Documentation

Add-MIISADMARunProfileStep -MAName ‘AD_MA’ -Partition ‘DC=CONTOSO,DC=COM’ -StepType ‘FI’ -ProfileName ‘ADMA_FULLIMPORT’

Possible values of the StepType parameter (short form or long one can be used):
“FI”, “FULL IMPORT”
“FS”, “FULL SYNCHRONIZATION”
“FIFS”, “FULL IMPORT AND FULL SYNCHRONIZATION”
“FIDS”, “FULL IMPORT AND DELTA SYNCHRONIZATION”
“DI”, “DELTA IMPORT”
“DS”, “DELTA SYNCHRONIZATION”
“DIDS”, “DELTA IMPORT AND DELTA SYNCHRONIZATION”
“EXP”,”EXPORT”

The neat feature of this cmdlet is that it will create the Run Profile if it doesn’t exist.

Cool, so let’s have a play with it. I installed the Rollup on the Synchronisation server in my dev environment. I then went looking for the PS module, and it can’t be seen.

That’s a bit weird. Re-read the release notes. New cmdlet. Ah, maybe the new cmdlet has been added to the old PSSnapIn ?

Yes, there it is. Add-PSSnapin MIIS.MA.Config gives us access to the new cmdlet.

Creating Run Profiles using PowerShell.

Here is a Management Agent from my Dev environment with no Run Profiles configured.

Using ISE I used the new cmdlet to create a Run Profile to run a Stage (Full Import) on the ADMA. Note: You need to put parentheses around the Partition name.

And there you go. The Run Profile didn’t exist so it got created.

A nice enhancement would be if you could create multi-step Run Profiles. That would really save some time, to be able to script that.

Follow Darren on Twitter @darrenjrobinson

Automating the simultaneous deployment of AzureRM Virtual Machines for a development environment

This post is details my method for automating the creation of AzureRM virtual machines for use in a development environment. I’m using this process to quickly standup an environment for testing configurations on.

In summary this process;

  • parallel creation of the AzureRM Virtual Machines
  • All machines have the same configuration
    • NIC, Disks etc
  • All machines are created in a new Resource Group, with associated Virtual Network

Simultaneous Creating the AzureRM Virtual Machines for MIM 2016

For my MIM 2016 Lab I’m going to create 5 Virtual Machines. Their roles are;

  • Active Directory Domain Controllers (2)
  • MIM 2016 Portal Servers (2)
  • MIM 2016 Synchronization Server & collocated SQL Server (1)

In order to stand up the Virtual Machines as quickly as possible I spawn the VM Create process in parallel leveraging the brilliant Invoke-Parallel Powershell Script from Cookie.Monster just as I did for my Simultaneous Start|Stop all AzureRM VM’s script detailed here.

VM Creation Script

In my script at the bottom of this post I haven’t included the ‘invoke-parallel.ps1’. The link for it is in the paragraph above. You’ll need to either reference it at the start of your script, or include it in your script. If you want to keep it all together in a single script include it like I have in the screenshot below.

Parts you’ll need to edit

The first part of the script defines what, where and names associated with the development environment. Where will the lab be deployed (‘Australia East’), what will be the Resource Group name (‘MIM2016-Dev10’), the virtual network name in the resource group (‘MIM-Net10’), the storage account name (‘mimdev16021610’). Update each of these for the name you’ll be using for your environment.

#Global Variables
# Where do we want to put the VM’s
$global:locName ‘Australia East’
# Resource Group name
$global:rgName ‘MIM2016-Dev10’
# Virtual Network Name
$global:virtNetwork ‘MIM-Net10’
# Storage account names must be between 3 and 24 characters in length and use numbers and lower-case letters only
$global:stName ‘mimdev16021610’
# VMName
$global:NewVM $null

The VM’s that will be deployed are added to an array. The names used here will become the ComputerName for each VM. Change for the number of machines and the names you want.

# MIM Servers to Auto Deploy
$VMRole = @()
$VMRole += ,(‘MIMPortal1’)
$VMRole += ,(‘MIMPortal2’)
$VMRole += ,(‘MIMSync’)
$VMRole += ,(‘ADDC1’)
$VMRole += ,(‘ADDC2’)

The script will then prompt you to authN to AzureRM.

# Authenticate to the Azure Portal
Add-AzureRmAccount

The script will then prompt you provide a username and password that will be associated with each of the VM’s.

# Get the UserID and Password info that we want associated with the new VM’s.
$global:cred Get-Credential -Message “Type the name and password for the local administrator account that will be created for your new VM(s).”

The Resource Group will be created. The environment subnet and virtual network will be created in the resource group.
# Create Resource Group
New-AzureRmResourceGroup -Name $rgName -Location $locName
# Create RG Storage Account
$storageAcc New-AzureRmStorageAccount -ResourceGroupName $rgName -Name $stName -Type “Standard_GRS” -Location $locName
# Create RG Subnet
$singleSubnet New-AzureRmVirtualNetworkSubnetConfig -Name singleSubnet -AddressPrefix 10.0.0.0/24
# Create RG Network
$global:vnet New-AzureRmVirtualNetwork -Name $virtNetwork -ResourceGroupName $rgName -Location $locName -AddressPrefix 10.0.0.0/16 -Subnet $singleSubnet

A configuration for each of the VM’s that we listed in the array earlier will be created and added to a new array $VMConfig

Update the -VMSize “Standard_A1” for a different Azure VM Spec and  DiskSizeInGB for a different data disk size.

# VM Config for each VM
$VMConfig = @()
# Create VMConfigs and add to an array
foreach ($NewVM in $VMRole) {
# ******** Create IP and Network for the VM ***************************************
# *****We do this upfront before the bulk create of the VM**************


$pip New-AzureRmPublicIpAddress -Name $NewVM-IP1″ -ResourceGroupName $rgName -Location $locName -AllocationMethod Dynamic
$nic New-AzureRmNetworkInterface -Name $NewVM-NIC1″ -ResourceGroupName $rgName -Location $locName -SubnetId $vnet.Subnets[0].Id -PublicIpAddressId $pip.Id
$vm New-AzureRmVMConfig -VMName $NewVM -VMSize “Standard_A1”
$vm Set-AzureRmVMOperatingSystem -VM $vm -Windows -ComputerName $NewVM -Credential $cred -ProvisionVMAgent -EnableAutoUpdate
$vm Set-AzureRmVMSourceImage -VM $vm -PublisherName MicrosoftWindowsServer -Offer WindowsServer -Skus 2012-R2-Datacenter -Version “latest”
$vm Add-AzureRmVMNetworkInterface -VM $vm -Id $nic.Id
# VM Disks. Deploying an OS and a Data Disk for each
$osDiskUri $storageAcc.PrimaryEndpoints.Blob.ToString() “vhds/WindowsVMosDisk$NewVM.vhd”
$DataDiskUri $storageAcc.PrimaryEndpoints.Blob.ToString() “vhds/WindowsVMDataDisk$NewVM.vhd”
$vm Set-AzureRmVMOSDisk -VM $vm -Name “windowsvmosdisk” -VhdUri $osDiskUri -CreateOption fromImage
$vm Add-AzureRmVMDataDisk -VM $vm -Name “windowsvmdatadisk” -VhdUri $DataDiskUri -CreateOption Empty -Caching ‘ReadOnly’ -DiskSizeInGB 10 -Lun 0

# Add the Config to an Array
$VMConfig += ,($vm)
# ******** End NEW VM ***************************************
}

And finally BAM, we let it loose. Using the Invoke-Parallel script each of the VM’s are created in AzureRM simultaneously.

# In Parallel Create all the VM’s
$VMConfig Invoke-Parallel -ImportVariables -ScriptBlockNew-AzureRmVM -ResourceGroupName $rgName -Location $locName -VM $_ }

The screenshot below is near the end of the create VM process. The MIMPortal1 VM that was first in the $VMConfig array is finished and the others are close behind.

The screenshot below shows all resources in the resource group. The Storage Account, the Network, each VM, its associated NIC and Public IP Address.

For me all that (including authN to AzureRM and providing the ‘username’ and ‘password’ for the admin account associated with each VM) executed in just under 13 minutes. Nice.

The full script? Sure thing, here it is.

Follow Darren Robinson on Twitter

Simultaneously Start|Stop all Azure Resource Manager Virtual Machines in a Resource Group

Problem

How many times have you wanted to Start or Stop all Virtual Machines in an Azure Resource Group ? For me it seems to be quite often, especially for development environment resource groups. It’s not that difficult though. You can just enumerate the VM’s then cycle through them and call ‘Start-AzureRMVM’ or ‘Start-AzureRMVM’. However, the more VM’s you have, that approach running serially as PowerShell does means it can take quite some time to complete. Go to the Portal and right-click on each VM and start|stop ?

There has to be a way of starting/shutting down all VM’s in a Resource Group in parallel via PowerShell right ?

Some searching and it seems common to use Azure Automation and Workflow’s to accomplish it. But I don’t want to run this on schedule or necessarily mess around with Azure Automation for development environments, or have to connected to the portal and kickoff the workflow.

What I wanted was a script that was portable. That lead me to messing around with ‘ScriptBlocks’ and ‘Start-Job’ functions in PowerShell. Passing variables in for locally hosted jobs running against Azure though was painful. So I found a quick clean way of doing it, that I detail in this post.

Solution

I’m using the brilliant Invoke-Parallel Powershell Script from Cookie.Monster, to in essence multi-thread and run in parallel the Virtual Machine ‘start’ and ‘stop’ requests.

In my script at the bottom of this post I haven’t included the ‘invoke-parallel.ps1’. The link for it is in the paragraph above. You’ll need to either reference it at the start of your script, or include it in your script. If you want to keep it all together in a single script include it like I have in the screenshot below.

My rudimentary PowerShell script takes two parameters;

  1. Power state. Either ‘Start’ or ‘Stop’
  2. Resource Group. The name of the Azure Resource Group containing the Virtual Machines you are wanting to start/stop. eg. ‘RG01’

<

p style=”background:white;”>Example: .\AzureRGVMPowerGo.ps1 -power ‘Start’ -azureResourceGroup ‘RG01’ or PowerShell .\AzureRGVMPowerGo.ps1 -power ‘Start’ -azureResourceGroup ‘RG01’

Note: If you don’t have a session to Azure in your current environment, you’ll be prompted to authenticate.

Your VM’s will simultaneously start/stop.

What’s it actually doing ?

It’s pretty simple. The script enumerates the VM’s in the Resource Group you’ve specified. It looks to see the status of the VM’s (Running or Deallocated) that is the inverse of the ‘Power’ state you’ve specified when running the script. It’ll start stopped VM’s in the Resource Group when you run it with ‘Start’ or it will stop all started VM’s in the Resource Group when you run it with ‘Stop’. Simples.

This script could also easily be updated to do other similar tasks. Like, delete all VM’s in a Resource Group.

Here it is

Enjoy.

Follow Darren Robinson on Twitter