More Fun with ARM Templates

All this time, all I’ve wanted to do with an Azure Resource Manager (ARM) template was create a bunch of identical computers.   The biggest roadblock I’ve faced is figuring out how to make the ARM template not one gigantic document with similarly-named resources in it.

Each VM requires a public IP address, a NIC, and a VM resource.   When I’ve put those into a template, if I named everything to make sense to me (like “Computer1” would have “Computer1-IP” and “Computer1-NIC” attached to it), the document gets really hard to follow.    I can copy and paste the definitions for these three objects, but eventually it gets ugly and confusing, and when something doesn’t work, it’s a pain to work around.

Enter the “count” parameter!   There’s a neat way that you can tell the ARM engine to iterate through a list of resources and create however many you want it to.   The engine breaks down the JSON file and duplicates the guts of it “n” times, assigning a suffix with the number on it at the end of the item names.

Below is my ARM template that creates as many VMs as you want, each with a public IP address and a NIC.

———————

{
“$schema”: “https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#”,
“contentVersion”: “1.0.0.0”,
“parameters”: {
“userImageStorageAccountName”: {
“type”: “string”,
“metadata”: {
“description”: “This is the name of the your storage account”
}
},
“userImageStorageContainerName”: {
“type”: “string”,
“metadata”: {
“description”: “This is the name of the container in your storage account”
}
},
“userImageVhdName”: {
“type”: “string”,
“metadata”: {
“description”: “This is the name of the your customized VHD”
}
},
“adminUserName”: {
“type”: “string”,
“metadata”: {
“description”: “UserName for the Virtual Machine”
}
},
“adminPassword”: {
“type”: “securestring”,
“metadata”: {
“description”: “Password for the Virtual Machine”
}
},
“osType”: {
“type”: “string”,
“allowedValues”: [
“windows”,
“linux”
],
“metadata”: {
“description”: “This is the OS that your VM will be running”
}
},
“vmSize”: {
“type”: “string”,
“metadata”: {
“description”: “This is the size of your VM”
}
},
“vmNameBase”: {
“type”: “string”,
“metadata”: {
“description”: “This is the size of your VM”
}
},
“vmCount”:{
“type”: “int”,
“defaultValue”: 1,
“metadata”: {
“description”: “The number of VMs to build.”
}
}
},
“variables”: {
“location”: “[resourceGroup().location]”,
“virtualNetworkName”: “VNetName”,
“addressPrefix”: “10.6.0.0/16”,
“subnet1Name”: “default”,
“subnet1Prefix”: “10.6.0.0/24”,
“publicIPAddressType”: “Dynamic”,
“vnetID”: “[resourceId(‘Microsoft.Network/virtualNetworks’,variables(‘virtualNetworkName’))]”,
“subnet1Ref”: “[concat(variables(‘vnetID’),’/subnets/’,variables(‘subnet1Name’))]”,
“userImageName”: “[concat(‘http://’,parameters(‘userImageStorageAccountName’),’.blob.core.windows.net/’,parameters(‘userImageStorageContainerName’),’/’,parameters(‘userImageVhdName’))]”
},
“resources”: [
{
“apiVersion”: “2015-05-01-preview”,
“type”: “Microsoft.Network/publicIPAddresses”,
“name”: “[concat(parameters(‘vmNameBase’),copyIndex(),’-PublicIP’)]”,
“copy”: {
“name”: “publicIPAddressCopy”,
“count”: “[parameters(‘vmCount’)]”
},
“location”: “[variables(‘location’)]”,
“properties”: {
“publicIPAllocationMethod”: “[variables(‘publicIPAddressType’)]”,
“dnsSettings”: {
“domainNameLabel”: “[concat(parameters(‘vmNameBase’),copyIndex())]”
}
}
},
{
“apiVersion”: “2015-05-01-preview”,
“type”: “Microsoft.Network/networkInterfaces”,
“name”: “[concat(parameters(‘vmNameBase’),copyIndex(),’-NIC’)]”,
“location”: “[variables(‘location’)]”,
“copy”: {
“name”: “networkInterfacesCopy”,
“count”: “[parameters(‘vmCount’)]”
},
“dependsOn”: [
“[concat(‘Microsoft.Network/publicIPAddresses/’, parameters(‘vmNameBase’),copyIndex(),’-PublicIP’)]”
],
“properties”: {
“ipConfigurations”: [
{
“name”: “ipconfig1”,
“properties”: {
“privateIPAllocationMethod”: “Dynamic”,
“publicIPAddress”: {
“id”: “[resourceId(‘Microsoft.Network/publicIPAddresses/’,concat(parameters(‘vmNameBase’),copyIndex(),’-PublicIP’))]”
},
“subnet”: {
“id”: “[variables(‘subnet1Ref’)]”
}
}
}
]
}
},
{
“apiVersion”: “2015-06-15”,
“type”: “Microsoft.Compute/virtualMachines”,
“name”: “[concat(parameters(‘vmNameBase’),copyIndex())]”,
“copy”: {
“name”: “vmCopy”,
“count”: “[parameters(‘vmCount’)]”
},
“location”: “[variables(‘location’)]”,
“dependsOn”: [
“[concat(‘Microsoft.Network/networkInterfaces/’,parameters(‘vmNameBase’),copyIndex(),’-NIC’)]”,
“[concat(‘Microsoft.Network/publicIPAddresses/’, parameters(‘vmNameBase’),copyIndex(),’-PublicIP’)]”
],
“properties”: {
“hardwareProfile”: {
“vmSize”: “[parameters(‘vmSize’)]”
},
“osProfile”: {
“computername”: “[concat(parameters(‘vmNameBase’),copyIndex())]”,
“adminUsername”: “[parameters(‘adminUsername’)]”,
“adminPassword”: “[parameters(‘adminPassword’)]”
},
“storageProfile”: {
“osDisk”: {
“name”: “[concat(parameters(‘vmNameBase’),copyIndex(),’-osDisk’)]”,
“osType”: “[parameters(‘osType’)]”,
“caching”: “ReadWrite”,
“createOption”: “FromImage”,
“image”: {
“uri”: “[variables(‘userImageName’)]”
},
“vhd”: {
“uri”: “[concat(‘http://’,parameters(‘userImageStorageAccountName’),’.blob.core.windows.net/vhds/’,parameters(‘vmNameBase’), copyIndex(),’-osDisk.vhd’)]”
}
}
},
“networkProfile”: {
“name”: “[concat(parameters(‘vmNameBase’),copyIndex(),’-networkProfile’)]”,
“networkInterfaces”: [
{
“id”: “[resourceId(‘Microsoft.Network/networkInterfaces/’,concat(parameters(‘vmNameBase’),copyIndex(),’-NIC’))]”
}
]
},
“diagnosticsProfile”: {

“bootDiagnostics”: {
“enabled”: “true”,
“storageUri”: “[concat(‘http://’,parameters(‘userImageStorageAccountName’),’.blob.core.windows.net’)]”
}
}
}
}
]
}

——————————————————-

The “copyIndex()” command is what tells the ARM engine to use the value from the loop.   Using “DependsOn” with this, means that Computer1 will depend upong Computer1-NIC and Computer1-PublicIP, so nothing will be built if the other parts aren’t ready for it.

This template is setup to be used with a user image, something custom that I’ve put up there that’s already sysprepped and ready-to-go.  I haven’t yet made this add the resources for the extensions or gotten it to join the domain after being built, but at least I have the VMs running now.

Network Security Groups

Two posts in a day?   Yes….because I’m afraid if I wait until Monday, I’ll forget this stuff.

I’ve been working with Azure Network Security Groups (NSG) today.   For the uninitiated, imagine firewall rules that you can apply to subnets within your Azure network or to specific VMs.   NSG can basically replace all the ACLs on your VM endpoints, allow you to more fully control network traffic between your subnets, and can give you granular control on your VMs.   I envision replacing all of my endpoint ACLs and all of my Windows Firewall configuration with these, once I get them right.

What did I learn today?

  1. The moment you apply an NSG to a subnet, the endpoint ACLs on any VMs within the subnet are basically deemed useless.  If you have an endpoint ACL that restrict connections, as soon as you turn on the NSG, if you don’t have the restriction configured, your connection is wide open to the Internet.
  2. Each rule is made up of a Name, Direction, Action, Source Address, Source Port, Destination Address, Destination Port, and Protocol.   This is like every other firewall rule I’ve ever put in.   One tip here, though, is that on Inbound rules, if you’re wanting to open, say, port 443, you need to put that on the Destination Port.   Leave Source Port at “Any” or “*”.    I was putting the ports in both parts, but it wouldn’t let inbound connections work at all that way.
  3. The new Azure portal is pretty nice in that you can open a subnet, see all the VMs in the subnet, and then use that to browse into the endpoints.    Much easier than in the old portal, which I don’t think could show you what VMs were on what subnets at all.
  4. Reiterated to me that you always do staging first.   I left off one endpoint from one NSG and I basically crippled SQL connections to a db server there.   Do your homework first and be careful.

These worked as advertised and I think it’s going to be the right path for me.   I’m still trying to figure out how to do things on VM-level NSGs for stuff like “RPC Dynamic Ports” and ICMP, but at least I can do some network-level stuff now.    Pretty cool.

End goal will be to have the subnet NSG and some templated VM NSG in such a condition that I can just add a VM, assign it to the right subnet, give it a VM NSG, and never have to worry about Windows Firewall on the machine.  Hell, I could even disable Windows Firewall altogether if I’m lucky.   Good times.

Formatting Disks After Deployment

I don’t usually post code snippets or whatever, but I think it’s time for me to do so.   I have some sweet little ones here and there, and I might as well publish them this way so others can use them if they need them.

One of the first difficulties I had when deploying new VMs was formatting all the disks that I attached to the machine after it was deployed.   In Azure, the system disk always formats to be the C: drive, and there’s always a D: drive attached for “temporary storage”.    However, as I was automating my server builds, I didn’t really have a way to format additional data disks that I attached.

Enter the Custom Script Extension.   This extension allows you to push a PowerShell script into a VM and have it just run.   It’s pretty slick in that it runs locally, just as if you were logged into the machine, and it puts some nice logging under

“C:\WindowsAzure\Logs\Plugins\Microsoft.Compute.CustomScriptExtension”

so you can find out what happened during the execution.

In my deployment script, after the VM is built, I simply pass in the following script to format all the additional drives and assign them the next available drive letters.

----------------------------------------------
Function SetupDisks()
{
 # This will initialize and format any uninitialized disks on the system and assign the next available drive letters to them
 
 $diskstoformat = get-disk | where-object {$_.numberofpartitions -eq 0} | sort {$_.number}
Foreach($disk in $diskstoformat)
 {
 $disknum = $disk.number
 $label = 'Data'+$disknum
$disk | Initialize-Disk -PartitionStyle MBR -ErrorAction SilentlyContinue

new-partition -disknumber $disknum -usemaximumsize -assigndriveletter:$False
$part = get-partition -disknumber $disknum -number 1
if($disknum -gt 1)
{
$part | Format-Volume -FileSystem NTFS -NewFileSystemLabel $label -AllocationUnitSize ‘65536’ -Confirm:$false -Force
}
else
{
$part | Format-Volume -FileSystem NTFS -NewFileSystemLabel $label -Confirm:$false -Force
}
end
$part | Add-PartitionAccessPath -AssignDriveLetter
}
}
———————————————
Taking a look at this, here’s what’s happening:

  1. First, I pull all the disks that have no partitions.
  2. For each one, I assign it a number and label, then initialize it.
  3. Next I create a single maximum-sized partition on the disk.
  4. If I’m on the first disk, I just format it with normal NTFS ….otherwise, I change the allocation size to 65536 to make better use of the disk.   (This isn’t necessary generally, but I do it because only my SQL VMs have more than one data disk, and I use those to store my data, logs, and backups.    My “standard” VM has an attached data disk, which I don’t use the allocation on.)
  5. Finally, I assign a drive letter.

This works every time.   What took me the longest was figuring out how to get the disks to initialize in a reliable way.   Once I got the initialization down, it was pretty straightforward.

FTP as a Service

How can FTP as a Service be so hard to find and so damn expensive when I find it?   All I need is a cloud-based storage solution that support FTP with SSL encryption.   This should be so simple for Microsoft to offer in Azure, but, alas, they don’t.

BrickFTP is a solution that appears, on the surface, to do everything I need it to do.   It’s also inexpensive, which is awesome.   The problem is, unless you go with the highest price offering, all the usernames have to be globally unique within their entire system.   So two clients can’t have a “CSMITH”.

C’mon guys….make it so the usernames only need to be unique within the customer’s individual account.   This shouldn’t be that tough to manage from your end.   In your own database, just use a composite key with the customer account number and name as your “username” within your stuff.   Don’t make your customers do some first-come-first-served thing with them.   Otherwise, the service is awesome.   It’s easy to setup and works.     I don’t want this one limitation to be a show-stopper, but at the moment, I’m afraid is.

Other FTP services are out there, but they are all pretty pricey.   Most of them are billed based upon the number of user accounts, too.   This makes sense, as I think they are seeing themselves as a kind of Dropbox replacement.    This isn’t the model I’m working in, though, as I have just a few accounts that all need managed access to the same set of files.

Argh.  More research or keep doing it myself.

 

I’ve Drunk the Kool-Aid

OK.   I admit it.   I doubted that enterprises would really move to “the cloud” as much, if not more, than the next systems guy.   No matter how the numbers were run, on-premise server systems were cheaper.   I could buy a SAN, the server blades, and all of the software licenses I needed for less money, in the long term, than running the same number of virtual servers in the cloud.

Last year, I moved over to a position in a team that has always been “cloud first.”   Upon getting there, I was handed a slew of virtual machines to take care of, some cloud-based databases and services, and a very small on-premises production system at a colocation facility.

What has made me actually drink the Kool-Aid?   It’s very simple, actually.  About a month ago, I went to a VMware User Group meeting, and I was sitting in a room of about 200 engineers from around Cincinnati.   The keynote started, and the VMware guy talked for about 45 minutes on backups.    That’s right, backups.    I left that room afterward and went to a quick meeting regarding flash storage systems.    Then I went to the vendor floor and found it full of people pushing SSD, thin clients, and “software defined” stuff.

You know what I realized?   Most of the vendors in there will still be in business in 10 years selling stuff to someone.   However, most of those 200 engineers won’t be working with the stuff those people are selling.  The people worrying about flash storage arrays and all those things are going to all work for the cloud vendors behind the scenes, and only a few of the guys in this particular VMUG will move on to work for them.

For a year, I haven’t had to worry one bit about anything the vendors in the room were selling, and yet I know I use most of that stuff indirectly every day.   I haven’t once had to figure out a way to squeeze more power out of a SAN or expand my infrastructure to handle extra VMs.   I haven’t had to concern myself with zoning a SAN to get storage to a server.   I haven’t had to worry about how to get a VM to a particular VLAN on the network.

What have I been doing?   I’ve been doing all the other stuff a Windows Server guy is supposed to be doing.    Setting up and making sure Active Directory works.   Making sure my backups and security-type things are taken care of.   Maintaining the actual software that rides on the server VMs that I have in place.   Working hard to keep up with other services, like Azure AD domain services and Security Center, as they are released to preview, so I can jump onboard and get some value out of them the day they are made GA.

THAT’S WHAT A SERVER GUY IS SUPPOSED TO BE DOING!   Bringing value to the business by spending time taking care of the software the business runs, and making sure that the data is safe, secure, and available all the time.

If you’re a server guy and haven’t jumped into the cloud, do it.   You need to do so now, because if you wait much longer, you’re going to be looking for work.   The cloud just makes things work.    Back to my first point, is it cheaper to do things in the cloud?   If you’re looking at direct costs, I don’t think it beats on-prem yet.   If you’re looking at all the indirect costs that are hidden in the on-prem systems, though, I think the cloud becomes more favorable.

Azure Security Center

Microsoft now has released to Public Preview the Azure Security Center service, and, man, am I impressed.    I love the potential in the product and am just dreamy-eyed at the products people use every day that this thing can replace.

Consider your most basic server scenario:  what a new company must have to operate.   First thing is probably a place to store files.   You probably need a print server, too (YUCK).   You probably need some way to secure the files and get some permissions on them, so if you’re like most places, you’ll need Active Directory.    You will certainly have Internet access and need antivirus and some software to manage that on your computers.   You probably need some updating software to keep all of the systems up-to-date.

In my most basic scenario, I envision about 5 servers:

  1. 2 for Active Directory for redundancy
  2. 1 for AV management
  3. 1 for Update management (WSUS or the like)
  4. 1 File/Print Server.

This isn’t even including any servers that you may need for software that your company uses.   This is just the basics.  If you’re small enough or don’t have specific requirements for local servers, you can certainly do all of this through services like Google Docs or OneDrive, plus use Windows Update to take care of the rest.   In my world, though, those don’t really cut it.   I have servers that I have to take care of, and those servers have “support server” requirements, even without a single user touching anything.

Here’s where Security Center comes in.   It can now, and has the potential to, basically eliminate the AV and Updating systems.   It also will provide means to get at suggestions for solving security events, provide configuration recommendations for your servers all the time, and even can give you audit trails for things that aren’t so easy to find in Windows alone.

Why is this such a big deal to me?   If you’re on the fence about moving to IaaS, this added service should totally move you to the cloud side.  It’s easy to operate, gives you a load of information, and can eliminate a lot of the worry you have day-to-day.

Now, I know nothing about AWS, but if you’re considering what cloud vendor to go to, you need to look at this solution in Azure.   If Amazon doesn’t have something similar, you might really want to consider going with Azure, particularly if you are going to have a lot of stable, rarely-changing VMs in our cloud.    If you are doing DevOps-type stuff only, you may not even care about this, but most of the IT departments I know still are building servers that need to stick around for a few years without worry.   For them, this is a killer app.

One thing I think is missing is backup monitoring.    Backup isn’t always looked at as part of the security team’s focus, but it sure seems to me like having secure, regular backups of your data is critical to the security of your company.   I would love for some backup monitoring and alerting to be integrated into this solution.    If that was to happen, I don’t know that I’d have to look at any other portal on a daily basis to check on things.   “One Stop Shopping”, as an old boss of mine used to say.