Azure VM DSC Extension

What a discovery!   I’ve been trying for weeks to get an automated SQL Server deployment working in Azure, and every time I’ve gotten close, but never exactly where I wanted.

I started with doing a Chef recipe for the installation.  This worked, but I have problems getting the gems to install correctly in my Azure VMs.   After some work with both the Chef and the MS support teams, it appears that I have a problem connecting to to obtain the gems.   The main one I need is rubyzip, as I need to pull down and extract some files.   With my DNS setup, though, it doesn’t want to work.   Seems like the site is kicking me back because of a new “.azure” top-level domain.   Kinda weird.

I then started messing with DSC.   Using the xSQLps module, I was able to install SQL Server using a push configuration pretty easily.   This was awesome, so I took that configuration and added it to a Chef recipe, as I don’t have a DSC pull server available and don’t really want one.   Chef has two methods to run DSC configurations:  dsc_script and dsc_resource.   dsc_script is kind of the “old” way, as it just runs the script on the machine, creates the MOF, and applies it.  That should’ve worked, but i couldn’t figure out how to pass a credential in an encrypted fashion.  dsc_resource is the “new” way, but I’ve been asked to hold off on using it due to needing WMF5 and some more support hashed out.   After another call with Chef, I got dsc_script working, but then I was back to my original problem, needing to download the rubyzip gem first.   Argh….

Finally, I found the Azure VM DSC Extension, which allows you, via PowerShell, inject DSC configurations to run on an Azure VM one time.   Essentially, you publish a configuration to a blob storage location and then issue a “Set-AzureVMDSCExtension” command against the VM.    This causes the vm to pull the config and run it locally, regardless of if the LCM is set to Push, Pull, or Disabled.   Could this be the key?

I created a general script, as shown below, to download the Windows SXS folder to the machines.   This is already on most machines, but there are parts missing in Azure VMs that are needed later, so previously, I had just zipped the whole thing up and put it in a blob.  (I’ve changed a couple server names in the blob….insert your own.


configuration GeneralConfig
import-dscresource -module ‘PSDesiredStateConfiguration’
import-dscresource -module ‘xPSDesiredStateConfiguration’

node (“localhost”)
File Sources
Type = “Directory”
Ensure = “Present”
DestinationPath = “C:\Sources”

# downloads SXS Folder
xRemoteFile DownloadSXS
DestinationPath = “C:\sources\”
Uri = “https://<storageaccount><container>/

# Extracts sources
Archive ExtractSXS
Ensure = “Present”  # You can also set Ensure to “Absent”
Path = “c:\sources\”
Destination = “c:\sources”



This script creates the c:\sources folder, downloads the zip to it, then extracts it.   Simple enough.

Next, you have to publish this DSC config into your blob storage.   You do this with the Publish-AzureVMDSCConfiguration cmdlet.  A couple of caveats here.   It defaults to publish it to whatever your current storage account is configured as in your subscription and it puts it in a container called windows-powershell-dsc.   You can change both with arguments to the cmdlet, and I suggest you do so so you can use the same files regardless of what your “current storage context” is later.

Once published, applying this to a machine, even an existing one, is pretty straightforward.   The following PowerShell does it.

Set-AzureVMDscExtension -ConfigurationArchive <zip file created in publish process> -ConfigurationName “GeneralConfig” -vm $vm -storagecontext $ctx | update-azurevm

$vm has to be a VM object that you get with “get-azurevm”.

On the machine, there will be a log folder under c:\windowsazure\logs\plugins\Microsoft.Powershell.dsc\<version> that will contain a “DSCExtensionHandler” file.   This will show you to outcome of the run.   To get things working the first time, you may need to apply your config multiple times until you get a successful run, but once it works, you’re ready.

Finally, in my build script, I just added a line at the end with the Set-AzureVMDSCExtension cmdlet from above.   As soon as the machine is deployed, the config is now injected and applied.

No longer do I need to try to do things with extra tools (Chef, pull server, whatever).  Now that I have it right, it will deploy the exact same every time via my script.   I can actually remove all of the Chef recipes I have to deploy the apps that I’m deploying in my “General” config, and drop all of them that deploy SQL.    Even if something gets broken down the road, redeploying is a breeze with this in place.

There you have it.   Hope this helps someone else.   My next step is to inject a custom script, or maybe even just add to the current one, to configure the SQL Server itself (default db location, log location, etc.)   That should be fun, but in the worst case, I can push that with a sqlcmd script using Chef for whatever.   Nothing to it.   🙂

DSC Adventure

Man, this PowerShell DSC stuff is such a new way to do things…..for a guy coming into the industry, it has to be intimidating.  It is to me, and I’ve been at this for 15 years.  Basically, gone are the days where you can get into IT and just click Next a bunch of times, check some boxes, and move on.   Even systems that require a bit more skill, like Group Policy, are a breeze to use compared to DSC.

Here’s why, and it’s tough for an infrastructure guy: you have to have some level of skill doing development.   You don’t have to be a full-on developer, but you better not be afraid to look at some code to troubleshoot things.   You better know how conditional statements, iterative loops, and probably even an idea of object-oriented programming, all work.    You can get by at first just by filling in some JSON files with property values and then applying them, but you’re not going to get far without creating your own custom resources or customizing existing ones.

This could be because the technology is so new, but I’m not sure.   I don’t know that the people working with it now want to make it any friendlier, and I don’t know that anyone is making some sort of GUI-based solution to create or apply them.   There are some third party systems like Chef and Puppet that do some work with them, but those aren’t the simplest things to pick up either.

As for my own progress, I’m getting there.   I’ve modified a couple of resources on my own and even created a pull request for my changes on one of them.  (I don’t think it got pulled, but whatever.)   I’m not a programmer by training, so I knew this would be a step.   I think it’s a good one, though.   The technology is slick, it’s native to Windows, and it’s far more powerful than anything else that is out there, including GPO.   The promise is that going forward it will become THE ONLY way to configure Windows servers.  I don’t know if that will become totally true or not, but so far, Snover hasn’t lied with putting the Monad Manifesto into place.

If you don’t do DSC now, you’d better get into it soon.   If you don’t even do PowerShell now, you are behind and need to get working on it.   I haven’t really found classes on these things, as Microsoft is treating it as just something Windows people have to know.   It’s not like every other technology that they create, then put out classes, exams, and certs in them.   With PowerShell and DSC, you just need to do it yourself.   The MVA courses are good and there are some decent videos out there you can create a learning plan out of.   Take a day, do a full course.   Spend the next week using it as much as you can in real life.   Take another day, do another full course.   Spend another week doing it.   Keep doing that, and within 3 months you should be damn good.   Where am I now?   On a scale of 1-10, I’d give myself an 8 in PowerShell and a 5 on DSC.   Relative to most admins I know, though, I’m a 10 on both….there just doesn’t seem to be the jump to do this stuff out there, unfortunately.   I don’t know why.

Well, I guess I do.   The learning curve is pretty steep.  But man, is it awesome once you get there.

Gotta get back

I gotta get back to posting on here regularly.   It’s really nice to have a place to come back to for information on stuff I’ve figured out.   So much with the random thoughts stuff…I just need a place for notes.

My current discovery is working with PowerShell DSC to get SQL Server installed automatically.   A couple of good links are below.   One is an example using the xSQLps module, and it works really well.   The other is information regarding securing credentials using certificates.   That’s the hard part of this, and it’s the part I’m still trying to figure out.

More to come when I figure out the cert thing.