SharePoint DSC Gotchas and Best Practices

In the Microsoft Virtual Academy on SharePoint Automation with DSC my colleague Brian Farnhill and I recorded in June of 2017, we dedicated an entire module to SharePointDSC best practices. I thought I’d take a few minutes to put the items we discussed during that session on paper. This article will act as a list of the most frequent issues and problem encountered by people using SharePoint DSC. It will be updated as the tool evolves.

Frequent Issues

Installation of Prerequisites, Binaries or Language Pack is Stuck

Symptom:

This is one of the most frequent issue people encountered when starting off with SharePoint DSC. When running the configuration using Start-DSCConfiguration, or when in Pull mode, the installation seems to get stuck when trying to install components (prerequisites, binaries, or language pack). No errors are being thrown.

Most Likely Reason:

There is some level of User Account Control set on the installation file. In most cases this issues will happen when the installation is done from a Network Drive. Upon executing the installation of the component, windows triggers a prompt for the user to confirm the execution of an executable that was launched from an untrusted location, but because this was initiates from a background thread, it is impossible to ever approve that prompt and the installation gets stuck.

In order to test if this is your issue, launch a new PowerShell session, running as the user defined in the PsDscRunAsCredential parameter of the execution block. From there, run the following line of PowerShell. If there are any prompts that appear, that means that the installation is most likely stuck because of the User Account Control.

Start-Process -FilePath <Path to the Installer file> -Wait -PassThru

Solution:

The solution here is two fold. First off, we recommend you turn off the User Account Control feature for the user running the PsDscRunAsCredential installation block in your script. As a second step, launch Internet Explorer and go to Internet Explorer Settings. From there, go to the Security tab, click on the Local Intranet Zone, then click on the Sites button. In the prompt, click on the Advanced button, this will open the window that let’s you specify addresses to identify as being part of the Local Intranet Zone. In that window, type in the location of the Network share (or other common location) where the installer file is located, and click on Add.

Using a completely DSC based solution, you could use the xSystemSecurity module to disable the User Account Control prompt as follow:

# Make sure you import the module
Import-DSCResource -ModuleName "MSFT_xSystemSecurity"

# Put this in the node's configuration;
xUAC NeverNotifyAndDisableAll
{
    Setting = "NeverNotifyAndDisableAll"
}

Best Practices

Master Node

SharePoint is a distributed system, meaning that not all components run on the same server, and that most of the content and configuration is actually stored in a SQL Server database, that all servers in the farm can access at any time. When you define a DSC configuration file, you should identify one of the servers in your farm to be what I refer to has your Master Node. this is the node on which you will define most of the components in the Farm. All other nodes, referred to as secondary nodes in this scenario, should only define resources that allow them to install the required binaries and updates, to join the farm, and the service instances to run on that server. All other components should ONLY be defined on the Master Node.For example, don’t go and define an SPSite entry on multiple node, it is not necessary. SPSite objects are defined in the Content Databases anyway, so you don’t need every server in the farm to attempt to create them.

Take another example where you wish to update the DSC definition of a certain feature on a Site Collection to be disabled instead of enabled. If you properly defined your configuration, you only have to update the DSC definition for that feature within a single node (Master Node) in the DSC configuration file, but let’s imagine you have it defined within every node instead. You now have to go and modify it multiple times within your configuration, and that opens the door to mistakes. Let’s say our farm has 3 servers, that you updated the feature’s definition for Server #1 and #2, but forgot to update the definition in server #3. To make things worst, you’ve set all 3 servers’ Local Configuration Manager (LCM) to be set to ApplyAndAutocorrect. We all know that by default, the ConfigurationMode for the LCM will check every 15 minutes to make sure it is in its desired state, but that 15 minutes may not happen exactly at the same time on each of the servers in the farm. Imagine that at 10h a.m., server one has its LCM kick in, and realize that the feature has to be turned on based on the new definition. It will go and disable the feature. Now, let’s pretend that server #3’s LCM kicks in at 10h05 a.m., but since that server doesn’t have the proper definition, it re-disables the feature that was just turned on 5 minutes ago by Server #1. You can see where this is going, and that can cause a lot of headaches for your admin team. DO NOT DECLARE components (other than SPFarm, SPIntall, SPInstallPrereqs, and SPServiceInstance) on more than one server in the configuration.

Use ApplyAndAutocorrect with Caution

We took a decision a few years back not to let SharePointDSC deal with content such as SPList, SPContentTypes, etc. However, we do deal with items such as SPSite and SPWeb that are stored in the Content Database. The real beauty behind SharePoint is that it lets users deal with their content as they see fit. Users can easily go and create a new list, play with it for a few days, and then decide to delete it after because their tests are done.

PowerShell Desired State Configuration is very powerful. When you set the Local Configuration Manager of a server to ApplyAndAutocorrect, the server will automatically attempt to bring itself back into its defined Desired State Configuration whenever it detects that it drifted away from that configuration. While this feature is very powerful, my recommendation is not to set the LCM in ApplyAndAutocorrect the moment your configuration includes either and SPWeb or SPSite resource declaration, unless you are absolutely certain and aware of what this will do to your environment.

Imagine the following scenario where you have an environment which declares 3 SPWeb (say Web1, Web2, and Web3), and for which the LCM is set to ApplyAndAutocorrect. Come spring, the Information Management (IM) team at your organization undertakes a major cleanup project where they want to get rid of sites that have not been active in the past 6 months. There 3 SPWeb sites defined in the configuration, have been identified by the IM team to be deleted. The IM manager goes in and deletes the 3 sites, then sends an email to the IM director to let him know that the cleanup has been done. Two weeks after the IM director stumbles upon SPWeb2, which the IM Manager claimed to have deleted 14 days ago. Well, the moment the site was originally deleted by the IM Manager, 15 minutes after, the LCM detected that it was no longer in its desired state, and since its LCM is set to ApplyAndAutocorrect, it automatically recreated the site. Imagine the confusion this can cause.

Do Not Have Multiple SPProductUpdate & SPConfigWizard Entries

The SPProductUpdate resource is responsible for installing the bits from a Cumulative or Public Update (.exe) onto a server. When using it in a script that configures a farm from the ground up, the recommendation is to have it executed right after the SPInstall bits have installed the SharePoint binaries, and right before using the SPFarm resource to join the server to the farm. You need to make sure that this resource is applied to EVERY server in the farm, therefore it needs to be defined for each node in your Configuration Script.

The SPConfigWizard resource is responsible for executing the PSconfig process on the servers, committing the bits back to the farm’s configuration database. Once again, it needs to be run on every server in the SharePoint farm, and should therefore appear in every node in your DSC config.

It is obvious why you do not want to have more than one instance of the SPConfigWizard resource defined per server. There are no reasons why PSconfig should be ran multiple time on the server during a DSC consistency check. You also do not want to have more than one SPProductUpdate instance per server. Cumulative and Public updates are cumulative by nature, meaning that for example, you do not need to install the September 2017 Public update before installing the October 2017 Public Update. You simply install the October 2017 PU which contains all the bits from the previous updates as well. If you define multiple SPProductUpdate instances per server, the latest PU or CU will always overwrite the previous version, so you will end up installing old versions of patches, just so that they can be overwritten by more recent ones. Always remember to only have one instance of these two resources per node in your Configuration Script.

this article will be updated on an on-going basis.

What’s New in SharePointDSC.Reverse 1.9.0.0

The latest release of the ReverseDSC Orchestrator Script for SharePoint is the biggest release we’ve had so far and it is packed with new features that will make it even easier for users to migrate their on-premises SharePoint 2013 or 2016 Farms from their on-premises data centers to the cloud. This release drastically changes the way the orchestrator script does its extraction. The following is a complete list of all improvements that have been included in the latest release.

New Features

  • Configuration Data: A new ConfigurationData.psd1 file will be generated with all extractions. This external .psd1 file exposes all parameters that are specific to the source SharePoint environment. If you are planning on deploying a copy of the extracted SharePoint Farm into a new environment (Azure, on-premises, etc.), all you need to do is update the values contained in that file to reflect the destination environment. For more information regarding what parameters are now exposed in the Configuration Data, please refer to the Configuration Data wiki Page.
  • Extraction Modes: Previous versions of the Orchestrator Script would expose a -Lite switch, which allow the extraction process to skip certain DSC Resources such as the Timer Job States, Features, SPWeb, etc. In version 1.9.0.0, we introduce the concept of Extraction Modes, where each mode will capture a different set of DSC resources. to learn more about what set of DSC Resource will be extracted in each mode, please refer to Extraction Mode wiki page.
  • Full Installation: Previous versions of the Orchestrator Script extracted resources starting at the SPFarm level, meaning that the machines you were trying to replicate your environment on had to have the SharePoint bits installed on them already for the configuration to get applied. With the new version, the SPInstallPrereqs and SPInstall resources are automatically included in the configuration file. The Configuration Data files drives whether or not these are included in the compiled MOF file using the FullInstallation. parameter. If this parameter is set to $True in the configuration data, then the compiled MOF instances will attempt to install the SharePoint Prerequisites (using the Online Mode) and the binaries on the servers being configured.

New Resources Supported

SharePointDSC.Reverse 1.9.0.0 introduces support for the following Resources:

  • SPInstallPrereqs
  • SPInstall
  • SPMachineTranslationServiceApp

Deprecated Features

the following switches have been deprecated with the 1.9.0.0 Release. The equivalent result can now be achieved 9in most part) by using the -Mode Lite parameter.

  • -Lite
  • -SkipFeatures
  • -SkipWebs
  • -SkipHealthRules
  • -SkipSearchFileType
  • -SkipTimerJobState
  • -SkipUserProfile

ReverseDSC for Files and Registry

As mentioned in multiple blog article I wrote in the past, ReverseDSC is technology agnostic, meaning that it can reverse engineer any component (Windows, Linux, MacOS, etc.) as long as there is a DSC module that exists for it. What we refer to as Orchestrator Scripts is what is technology specific and does specific operations for a specific component. You’ve seen me blog mainly about the SharePointDSC.Reverse Orchestrator Script lately, and show users how they can extract configuration of existing SharePoint Farms and replicate them elsewhere. There are however several other Orchestrator Scripts being developed at the moment including the SQLServer.Reverse one and the PSDesiredStateConfiguration.Reverse one that allows you to replicate file and registry structured on other machines using DSC.

In this article, I want to demo how the PSDesiredStateConfiguration.Reverse Orchestrator Script can be used to extract a DSC Configuration script. While the Orchestrator Script allows you to extract numerous components of an environment (Windows Features, Logs, Archives, Users, etc.) we will only focus on Files and Registry Entries within this article.

How to Install

Just like all other Orchestrator Scripts we are currently working on, the PSDesiredStateConfiguration.Reverse script is made available in the PowerShell Gallery. You can download and install it from any machine that has internet connectivity by executing the following lines of PowerShell (needs WMF 5.0).

Install-Script PSDesiredStateConfiguration.Reverse -Force

If you don’t have internet connectivity on your machine or if you don’t have access to a machine with WMF 5.0+ on it, then you will need to manually download the script for the GitHub Repository from a machine that has internet connectivity and copy it manually onto the machine. By default, the PowerShell code above downloads the script under c:\Program Files\WindowsPowerShell\Scripts. I would therefore recommend copying the script under that folder (create the Scripts folder if it doesn’t already exist).

ReverseDSC for Files

In this section we will cover the process of using the PSDesiredStateConfiguration.Reverse Orchestrator Script to extract a file structure from an existing environment as a DSC configuration script that we can use to recreate that exact file structure onto another environment. The challenge here is that in order for this file structure to be recreated on a new computer, we need to have access to the files from a central location. Let’s take the following example where I wish to extract the File structure in the path C:\MyCustomApplication from my machine.

MyCustomApplication file Structure

MyCustomApplication file Structure

The files are currently local to the machine. If we were to simply extract the file structure as a DSC Configuration Script, it would result in DSC Resources blocks similar to the following:

File 469e3fe0-b1c6-4b70-bc69-725dd2bc3739
{
    DestinationPath = 'C:\MyCustomApplication';
    Type = 'Directory';
    Recurse = $True;
    SourcePath = 'C:\MyCustomApplication';
    Ensure = 'Present';
}

This code tells DSC on the new machine to copy all files that exist under SourcePath (C:\MyCustomApplication) under the DestinationPath (C:\MyCustomApplication). Now, let’s forget that both paths are the same for one moment and try to understand the issue here. The new machine we are trying to configure with DSC knows nothing about the one we ran ReverseDSC on. It doesn’t have access to the original file repository where all the .dll, .chm, and .exe are located. Therefore we need to make sure these files we want to copy over our new machine are made available either on a network drive or with a Share from the machine we ran ReverseDSC on.

In order to solve this issue and to make it easy for users to extract a DSC Configuration script that can easily be leverage on other machines, the script accepts and array of HashTable values for the Folders parameter. The HashTable defines the following values:

Key Description
Source Root URL of the folder to scan on the local machine
SharedSource Root URL of the Shared folder or network location where the files to be copied are located.

To go back to our example above, I will be using the following values when calling the script:

Key Value
Source C:\MyCustomApplication
SharedSource \\NetworkShare\MyCustomApplication

I will then be calling the PSDesiredStateConfiguration.Reverse.ps1 script as follow:

.\PSDesiredStateConfiguration.Reverse.ps1 -Folders @{Source="C:\MyCustomApplication";SharedSource="\\NetworkShare\MyCustomApplication"}

Doing so will then result in the DSC Resource blocks having the DestinationPath match what was on the original server we ran ReverseDSC onto, and the SourcePath pointing to the shared location as follow:

File 617c1dc1-a7f6-48b6-bd69-24b4cefe2d21
{
    DestinationPath = 'C:\MyCustomApplication\Bin\Core.dll';
    Type = 'File';
    SourcePath = '\\NetworkShare\MyCustomApplication\Bin\Core.dll';
    Ensure = 'Present';
}

Upon compiling the DSC Configuration Script into a MOF file and initiating the DSC configuration on the new machine, the machine will automatically copy the files from the shared location to a local path matching the original machine on which we ran ReverseDSC onto.

It is also important to note that as mentioned above, the Folders parameter is actually an array of HashTable, meaning that multiple values could be passed to it. Assuming you had two folders you wanted to extract, you code do it as follow:

.\PSDesiredStateConfiguration.Reverse.ps1 -Folders @(@{Source="C:\Folder1";SharedSource="\\NetworkShare\Folder1"}, @{Source="C:\Folder2";SharedSource="\\NetworkShare\Folder2"})

ReverseDSC for Registry Keys

The process to use the PSDesiredStateConfiguration.Reverse Orchestrator Script against some Registry Keys is very similar to the Files process described above. The Script accepts a parameter named RegistryPaths that is an array of string that contains the paths to the Parent Registry key you wish to extract. The Registry Key paths can be entered either as the long or short form (e.g. HKLM:\System or HKEY_LOCAL_MACHINE\System).

Assuming I wanted to extract everything under the following Registry Paths:

  • HKLM:\System\CurrentControlSet\Control\Lsa
  • HKLM:\Software\dotnet

All I would need to do is call the script as follow:

.\PSDesiredStateConfiguration.Reverse.ps1 -RegistryPaths @("HKLM:\System\CurrentControlSet\Control\Lsa", "HKLM:\Software\dotnet")

That would then produce an output having DSC Resource Blocks similar to the following:

Registry 23e6bc1f-2f88-4b14-afbe-dddd8e3aa3e6
{
    ValueName = "NoLmHash";
    Key = "HKLM:\System\CurrentControlSet\Control\Lsa";
    Ensure = "Present";
    ValueType = "DWord";
    ValueData = @("1");
}

Reverse Engineer an Existing SharePoint Farm using PowerShell DSC

PowerShell Desired State Configuration (DSC) is a technology that allows you to define a configuration for an environment, and ensure it automatically gets configured tat way. It can also help you monitor the delta between the current state of the environment and what the defined configuration (a.k.a. Desired State) is. So as an example in the case of SharePoint, if you were to specify that you needed to register a Managed Account for user sp_services, and that someone was to delete that account, PowerShell DSC would automatically report the discrepancy in the logs or attempt to automatically recreated it based on how you configured your environment (Configuration Mode).

The SharePoint module for DSC is great to build new environments from the ground up. Simply specify what the Desired State should be (Web Applications, Site Collections, Service Applications, etc.) and let DSC do its job and automatically configure it. It acts as some kind of abstraction layer for the user, where they only need to define resources and their various settings for DSC to create and configure everything for them in the background. They don’t need to know the cmdlets to use, know what the objects’ properties are, and even less understand the object model behind it all. They only need to know what the DSC resource accepts.

Existing Environments

In many cases however, clients who want to get into DSC already have investments in SharePoint, where they have an existing SharePoint 2013 or 2016 farm that they would like to replicate somewhere else with DSC. Writing a DSC configuration script from scratch that represents their exact environment is a complex job and often results in the customer writing tens of thousands of lines of code. This is where the ReverseDSC orchestrator for SharePoint, called SharePointDSC.Reverse comes into play. SharePointDSC.Reverse is a PowerShell script (.ps1) that you execute against an existing SharePoint 2013 or 2016 farm to extract its PowerShell Desired State Configuration. The script will scan every component of the farm, down to the SPWeb level. This means that everything that can be defined in Central Administration, with the addition of Sites can be extracted with the SharePointDSC.Reverse. The output will be another .ps1 script that will contain the DSC configuration of your farm. That script can then be executed to compile a .MOF file that can be used to replicate your SharePoint environment elsewhere.

Other Scenarios

The ReverseDSC Orchestrator for SharePoint can also be used for several scenarios other than replicating a farm. It can be used to:

  • Extract a SharePoint 2013 Farm Configuration and replicate it as a SharePoint 2016 one;
  • Compare deltas between two extractions to see what changed between two point in time;
  • Create standalone copies of a Farm for developers;
  • Move an on-premises SharePoint Farm into Azure Infrastructure-as-a-Service;
  • On-board an existing farm onto PowerShell DSC to monitor it and ensure it remains in its desired state.

How to Use

The ReverseDSC Orchestrator for SharePoint is now officially available in the PowerShell Gallery. It can be installed, alongside all of its dependencies, by executing the following PowerShell one-liner from a machine that has internet connectivity:

Install-Script SharePointDSC.Reverse -Force 

Note that this has to be done on only one server in the farm; you don’t need to install it on all servers. If your SharePoint server doesn’t have internet connectivity, simply run the above line of PowerShell from a machine that does have internet connectivity, and manually copy the ReverseDSC, SharePointDSC from the PowerShell modules folder (C:\Program Files\WindowsPowerShell\Modules), and the SharePointDSC.Reverse.ps1 script from the PowerShell script folder (C:\Program Files\WindowsPowerShell\Scripts) onto the destination server.

Additional Resources

To learn more about the ReverseDSC Orchestrator for SharePoint, you can watch my Microsoft Virtual Academy session on the topic at the following URL: https://mva.microsoft.com/en-US/training-courses/sharepoint-automation-with-dsc-17843

SharePointDSC and ReverseDSC Microsoft Virtual Academy Session

In June of 2017, my colleague Brian Farnhill and I recorded a Full Day Microsoft Virtual Academy (MVA) session on Automating SharePoint with PowerShell Desired State Configuration. The recording is now available for free on the MVA site and covers the following topics:

  • Introduction to PowerShell Desired State Configuration
  • Advanced PowerShell DSC on-premises
  • Introduction to SharePointDSC
  • Azure Automation DSC with SharePoint
  • ReverseDSC for SharePoint
  • SharePointDSC Best Practices and Guidance

Click on the following link to watch the Full Session.