Monitor the Compliance of your Office 365 Configuration with Azure DevOPS and DSC

As previously discussed in my The PaaS/SaaS DSC Paradigm article, the continuous monitoring of Software-as-a-Service configurations is something that keeps me up at night. Still to this day, when a customer asks me what the recommended way to run Office365DSC configurations and to ensure continuous compliance is, I still don’t have a good answer to provide. For on-premises system, my answer has always been to use Azure DevOPS to manage the Configuration as code, and to automate the deployments via Azure Pipelines; that really should be a no brainer for organizations. However Azure DevOPS doesn’t play that nice with SaaS configurations. I can use an Azure DevOPS Agent to compile and deploy the configuration remotely against my Office 365 tenant, but then that agent gets destroyed, and doesn’t do the frequent Consistency checks that DSC is known to be good for.

Things are about to change however. As I was sitting in a plane on my way to sunny (way too sunny this week actually) Las Vegas, I decided to try out a new approach to get around this limitation. Azure DevOPS allows you to schedule the execution of Release Pipelines. If we could somehow create a Pipeline whose sole purpose would be to test a configuration against a remote Office 365 Tenant and return an error if the configuration is not in the Desired State, we should be able to mimic something similar to what Azure Automation DSC is doing. When you come to think about it, with PowerShell 5.0 and above, the Test-DSCConfiguration cmdlet now offers a -ReferenceConfiguration parameter that can accept a path to a compiled .MOF file, and analyzes a machine based on it to check and see if it is in the Desired State. The good news for us is that the .MOF file doesn’t have to be currently applied to the machine in order for us to do the Consistency check. We can therefore have a Release Pipeline that will run on a regular basis, on an ephemeral Azure DevOPS Agent, test the configuration using Test-DSCConfiguration -ReferenceConfiguration, and report any configuration drifts.

This article describes the steps one should be taking to create 2 Azure DevOPS pipelines to “simulate” an ApplyAndMonitor Scenario. If you are looking to achieve ApplyAndAutocorrect, then it becomes even simpler since all you have to do is schedule a Pipeline that will execute the configuration on a regular basis without even bothering to check if the environment is in the desired state or not. After all, the concepts of Idempotency tell us that running the same thing over ad over again will always results in the same results don’t they?

Configuration File

For the purpose of this demo, I will use the below configuration Please note that it is “twice parameterized”. The .ps1 file accepts Username, Password, for the Global Admin Account, creates a PSCredential object out of these two pieces of information, then feeds that PSCredential object into the actual configuration. These Username and Password are fed in by the Build Pipeline at compilation time via Pipeline Variables.

param
(
[Parameter()]
[System.String]
$GlobalAdminPassword,

[Parameter()]
[System.String]
$ConfigDataPath
)

Configuration TenantConfig
{
param
(
[Parameter(Mandatory=$true)]
[System.String]
$GlobalAdminPassword
)

$password = ConvertTo-SecureString $GlobalAdminPassword -AsPlainText -Force
$GlobalAdmin = New-Object System.Management.Automation.PSCredential ($ConfigurationData.NonNodeData.GlobalAdminAccount, $password)

Import-DSCResource -ModuleName "Office365DSC"

Node $AllNodes.NodeName
{
SPOSite ReadySite
{
Title = "DemoReady"
Url = "https://[YourTenant]/sites/Ready1234191"
CentralAdminUrl = "https://[YourTenant]-admin.sharepoint.com"
Owner = $GlobalAdmin.Username
Template = "STS#0"
Ensure = "Present"
GlobalAdminAccount = $GlobalAdmin
}

TeamsTeam ReadyTest
{
DisplayName = "ReadyTest"
Ensure = "Present"
GlobalAdminAccount = $GlobalAdmin
}

TeamsTeam Ready146
{
DisplayName = "Ready146"
Ensure = "Present"
GlobalAdminAccount = $GlobalAdmin
}
}
}
TenantConfig -ConfigurationData $ConfigDataPath -GlobalAdminPassword $GlobalAdminPassword

Build Pipeline

While I will not be covering the creation of a build pipeline in this article, you should know that your build pipeline should do 3 things: Compile your .MOF file, Copy the files in the build artifacts and publish the artifacts so that your Release pipelines can consume the files. If you required additional guidance, you can always refer to my DevOPS for Office 365 article.

Push Release Pipeline

The push pipeline gets triggered via Continous Integration everytime a successful build completes. Once it does, all this pipeline does is run Start-DSCConfiguration against your configuration so that the Azure DevOPS pipeline can start configuring your tenant remotely. Again, I would refer you to the DevOPS for Office 365 article if you require additional guidance.

Monitor Pipeline

Alright, let’s get down to business.

1. Create a New Release Pipeline in your project.



2. Select an Empty job template.



3. Leave Stage 1 as the default name and click on Add an artifact.



4. Select the latest Build, leave the default values and click Add.



5. Click on the task link to add new tasks to the pipeline.



6. Click on the + button beside the Agent. When the task panel opens, search for PowerShell and finally, click on the Add button beside the PowerShell task to add a new one.



7. Select the newly added task, rename it to Monitoring Tenant, select Inline as the type of script, and paste in the following script. This will assess the configuration of your tenant against localhost (the DevOPS agent). The path here is the default build path. If you followed all steps, that path to the .mof file should be the same for you as well.

winrm quickconfig -Force
Install-Module Office365DSC -Force
$results = Test-DSCConfiguration -ReferenceConfiguration "D:\a\r1\a\_Build\Package\BuildFiles\TenantConfig\localhost.mof" -ComputerName localhost
$results
if (-not $results.InDesiredState)
{
$results.ResourcesNotInDesiredState
throw "Some resources are not in the Desired State."
}



8. Go back to the Pipeline tab and click on the Schedule Icon under the artifact.



9. This is the step where you define the frequency at which the Monitoring pipeline will automatically get triggered. Think of this as our ConfigurationModeFrequencyMins property for a Local Configuration Manager that doesn’t really exist. In my case, I will create 2 schedules, once at 6AM and once at 6PM, every day of the week. Click Save once your schedules have been defined.

In Action

Our pipelines are finally created, and everything is in place for our monitoring to take place. To showcase the monitoring process, I have manually triggered the new Azure DevOPS Release Pipeline. In my case, the tenant was already in the Desired State, so the pipeline returns successful.

Let us now force a configuration drift and see what happens. Go to your tenant’s list of team, and delete one of the two teams that are controlled by our Office365DSC configuration.

If I am to re-run the monitoring pipeline, it will now error out and if I take a look at the actual error, I will get information regarding what resources are no longer in their desired state.

I can now enjoy a good night sleep again.