Monitor the Compliance of your Office 365 Configuration with Azure DevOPS and DSC

As previously discussed in my The PaaS/SaaS DSC Paradigm article, the continuous monitoring of Software-as-a-Service configurations is something that keeps me up at night. Still to this day, when a customer asks me what the recommended way to run Office365DSC configurations and to ensure continuous compliance is, I still don’t have a good answer to provide. For on-premises system, my answer has always been to use Azure DevOPS to manage the Configuration as code, and to automate the deployments via Azure Pipelines; that really should be a no brainer for organizations. However Azure DevOPS doesn’t play that nice with SaaS configurations. I can use an Azure DevOPS Agent to compile and deploy the configuration remotely against my Office 365 tenant, but then that agent gets destroyed, and doesn’t do the frequent Consistency checks that DSC is known to be good for.

Things are about to change however. As I was sitting in a plane on my way to sunny (way too sunny this week actually) Las Vegas, I decided to try out a new approach to get around this limitation. Azure DevOPS allows you to schedule the execution of Release Pipelines. If we could somehow create a Pipeline whose sole purpose would be to test a configuration against a remote Office 365 Tenant and return an error if the configuration is not in the Desired State, we should be able to mimic something similar to what Azure Automation DSC is doing. When you come to think about it, with PowerShell 5.0 and above, the Test-DSCConfiguration cmdlet now offers a -ReferenceConfiguration parameter that can accept a path to a compiled .MOF file, and analyzes a machine based on it to check and see if it is in the Desired State. The good news for us is that the .MOF file doesn’t have to be currently applied to the machine in order for us to do the Consistency check. We can therefore have a Release Pipeline that will run on a regular basis, on an ephemeral Azure DevOPS Agent, test the configuration using Test-DSCConfiguration -ReferenceConfiguration, and report any configuration drifts.

This article describes the steps one should be taking to create 2 Azure DevOPS pipelines to “simulate” an ApplyAndMonitor Scenario. If you are looking to achieve ApplyAndAutocorrect, then it becomes even simpler since all you have to do is schedule a Pipeline that will execute the configuration on a regular basis without even bothering to check if the environment is in the desired state or not. After all, the concepts of Idempotency tell us that running the same thing over ad over again will always results in the same results don’t they?

Configuration File

For the purpose of this demo, I will use the below configuration Please note that it is “twice parameterized”. The .ps1 file accepts Username, Password, for the Global Admin Account, creates a PSCredential object out of these two pieces of information, then feeds that PSCredential object into the actual configuration. These Username and Password are fed in by the Build Pipeline at compilation time via Pipeline Variables.



Configuration TenantConfig

$password = ConvertTo-SecureString $GlobalAdminPassword -AsPlainText -Force
$GlobalAdmin = New-Object System.Management.Automation.PSCredential ($ConfigurationData.NonNodeData.GlobalAdminAccount, $password)

Import-DSCResource -ModuleName "Office365DSC"

Node $AllNodes.NodeName
SPOSite ReadySite
Title = "DemoReady"
Url = "https://[YourTenant]/sites/Ready1234191"
CentralAdminUrl = "https://[YourTenant]"
Owner = $GlobalAdmin.Username
Template = "STS#0"
Ensure = "Present"
GlobalAdminAccount = $GlobalAdmin

TeamsTeam ReadyTest
DisplayName = "ReadyTest"
Ensure = "Present"
GlobalAdminAccount = $GlobalAdmin

TeamsTeam Ready146
DisplayName = "Ready146"
Ensure = "Present"
GlobalAdminAccount = $GlobalAdmin
TenantConfig -ConfigurationData $ConfigDataPath -GlobalAdminPassword $GlobalAdminPassword

Build Pipeline

While I will not be covering the creation of a build pipeline in this article, you should know that your build pipeline should do 3 things: Compile your .MOF file, Copy the files in the build artifacts and publish the artifacts so that your Release pipelines can consume the files. If you required additional guidance, you can always refer to my DevOPS for Office 365 article.

Push Release Pipeline

The push pipeline gets triggered via Continous Integration everytime a successful build completes. Once it does, all this pipeline does is run Start-DSCConfiguration against your configuration so that the Azure DevOPS pipeline can start configuring your tenant remotely. Again, I would refer you to the DevOPS for Office 365 article if you require additional guidance.

Monitor Pipeline

Alright, let’s get down to business.

  1. Create a New Release Pipeline in your project.
  2. Create a New Azure DevOPS Pipeline

  3. Select an Empty job template.
  4. Empty Job template for Azure DevOPS Release pipeline

  5. Leave Stage 1 as the default name and click on Add an artifact.
  6. Add Azure DevOPS Release Pipeline Artifact

  7. Select the latest Build, leave the default values and click Add.
  8. Azure DevOPS Release Pipeline Artifact selection

  9. Click on the task link to add new tasks to the pipeline.
  10. Add tasks to an Azure DevOPS Release Pipeline Stage

  11. Click on the + button beside the Agent. When the task panel opens, search for PowerShell and finally, click on the Add button beside the PowerShell task to add a new one.
  12. Add a PowerShell Task to an Azure DevOps Release pipeline

  13. Select the newly added task, rename it to Monitoring Tenant, select Inline as the type of script, and paste in the following script. This will assess the configuration of your tenant against localhost (the DevOPS agent). The path here is the default build path. If you followed all steps, that path to the .mof file should be the same for you as well.
  14. winrm quickconfig -Force
    Install-Module Office365DSC -Force
    $results = Test-DSCConfiguration -ReferenceConfiguration "D:\a\r1\a\_Build\Package\BuildFiles\TenantConfig\localhost.mof" -ComputerName localhost
    if (-not $results.InDesiredState)
    throw "Some resources are not in the Desired State."

    Azure DevOps PowerShell Task inline script

  15. Go back to the Pipeline tab and click on the Schedule Icon under the artifact.
  16. Add an Azure DevOPS Release Pipeline Schedule

  17. This is the step where you define the frequency at which the Monitoring pipeline will automatically get triggered. Think of this as our ConfigurationModeFrequencyMins property for a Local Configuration Manager that doesn’t really exist. In my case, I will create 2 schedules, once at 6AM and once at 6PM, every day of the week. Click Save once your schedules have been defined.
  18. Azure DevOPS Release Pipeline Schedules

In Action

Our pipelines are finally created, and everything is in place for our monitoring to take place. To showcase the monitoring process, I have manually triggered the new Azure DevOPS Release Pipeline. In my case, the tenant was already in the Desired State, so the pipeline returns successful.

Successful Azure DevOPS Release Pipeline Execution

Let us now force a configuration drift and see what happens. Go to your tenant’s list of team, and delete one of the two teams that are controlled by our Office365DSC configuration.

Deleting a Microsoft Teams through the Web Interface

If I am to re-run the monitoring pipeline, it will now error out and if I take a look at the actual error, I will get information regarding what resources are no longer in their desired state.

Azure DevOPS Release Pipeline Task Error Details

I can now enjoy a good night sleep again.

One thought on “Monitor the Compliance of your Office 365 Configuration with Azure DevOPS and DSC

  1. This is a really great solution for ensuring your O365 configurations are all in line with the baseline.
    Assuming you want to automatise all this using Azure DevOps, which options would you have to secure your mof file (so that it doesn’t contain clear text passwords)?

Leave a Reply

Your email address will not be published. Required fields are marked *