Azure’s Infrastructure-As-Code: ARM Templates, Validation, and Deployment Using Azure DevOps

What is ARM? 

An ARM template is a JSON file used to configure and deploy various Azure resources like VMs, AKS clusters, web apps, VNets, functions, and more to the Azure cloud. The basic idea behind Infrastructure-as-Code (IAC) is to provide the infrastructure through automation rather than using manual processes. In this Agile development world, even infrastructure code is changing and so it needs to be committed to version control repositories so it can be built/deployed using repeatable processes. The IAC fits well in the Agile development process without manual interventions by auto-validation, redeployment of the resources using Continuous Integration and Continuous deployment (CICD).

A sample ARM template is shown here:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "adminUsername": {
            "type": "string",
            "defaultValue": "ashuser",
            "metadata": {
                "description": "User name for the Virtual Machine."
            }
        },
        "adminPassword": {
            "type": "securestring",
            "defaultValue": "Ashpassword123",
            "metadata": {
                "description": "Password for the Virtual Machine."
            }
        },
        "osDiskSize": {
            "type": "int",
            "defaultValue": 1024
        }
    },
    "variables": {
        "location": "[resourceGroup().location]",
        "addressPrefix": "10.0.0.0/16",
        "subnetName": "Subnet",
        "subnetPrefix": "10.0.0.0/24",
        "storageAccountType": "Standard_LRS",
        "publicIPAddressType": "Dynamic",
        "publicIPAddressName": "[concat('my-',variables('uniqString'),'-pip')]",
        "nsgName": "[concat('my-',variables('uniqString'),'-nsg')]",
        "nicName": "[concat('my-',variables('uniqString'),'-nic')]",
        "vmName": "[concat('my-',variables('uniqString'),'-vm')]",
        "vmSize": "Standard_DS1_v2",
        "virtualNetworkName": "[concat('my-',variables('uniqString'),'-vnet')]",
        "uniqString": "[toLower(substring(uniqueString(resourceGroup().id), 0,5))]",
        "vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
        "subnetRef": "[concat(variables('vnetID'),'/subnets/',variables('subnetName'))]",
        "windowsImage": {
            "publisher": "MicrosoftWindowsServer",
            "offer": "WindowsServer",
            "sku": "2012-R2-Datacenter",
            "version": "latest"
        }
    },
    "resources": [
        {
            "apiVersion": "2017-04-01",
            "type": "Microsoft.Network/virtualNetworks",
            "name": "[variables('virtualNetworkName')]",
            "location": "[variables('location')]",
            "properties": {
                "addressSpace": {
                    "addressPrefixes": [
                        "[variables('addressPrefix')]"
                    ]
                },
                "subnets": [
                    {
                        "name": "[variables('subnetName')]",
                        "properties": {
                            "addressPrefix": "[variables('subnetPrefix')]"
                        }
                    }
                ]
            }
        }
    ]
}

Use the azure-pipelines.yml shown in full at the end of this post as the root of your repository and adjust variables to work for your situation. This yaml file will create a build pipeline for your project. Modify this file and commit it back to your repository so that your build/deploy processes are also treated as code (Process-as-Code).

Follow the instructions to create a build pipeline to validate the ARM templates.

Steps

pool:
  name: Hosted Ubuntu 1604

• Select an Azure subscription to validate Arm templates in a resource group.
• Validate an Azure Resource Manager (ARM) template for (Basic, Standard, and Premium) solutions to a resource group. You can also start, stop, delete, deallocate all Virtual Machines (VM) in a resource group.
• Build pipeline references secret variable names such as resource group name, passwords, tenant ID, etc.
• Pass template parameters in Override template parameters.
• Deployment mode should be "Validation" for validating ARM templates.
• Validate mode enables you to find problems with the template before creating actual resources.

variables:
  rgname: 'arm-resource-group’

steps:
- task: AzureResourceGroupDeployment@2
  displayName: 'standerd-template-validation'
  inputs:
    azureSubscription: 'subscription name(XXXX-XXXX-XXXX-XXXX-XXXX)'
    resourceGroupName: '$(rgname)'
    location: 'East US'
    csmFile: 'main-template.json'
    csmParametersFile: 'main-template.parameters.json'
    overrideParameters: '-solutionType "Project-Edison" -deploymenttype Standard -geo-paired-region "EastUS2" -signalRlocation eastus2 -acrDeploymentLocation "CanadaCentral" -omsWorkspaceRegion "southeastasia" -appInsightsLocation "eastus" -appInsightsLocationDr "southcentralus" -tenantId $(tid) -botAdClientId "XXXXXXXXXX" -adObjectId "XXXX-XXXX-XXX-XXX-XXXX" -adClientSecret "$(adsp)" -azureAccountName "abc@gmail.com" -azurePassword "abcacacc123@" -adminName "adminuser" -sessionId "3791180c-24c5-4290-8459-a454feee90ab" -vmUsername "adminuser" -vmPassword "Pass@1212" -aksServicePrincipalClientId "XXX-XXXXa63c-XXXX" -aksServicePrincipalClientSecret $(ksp) -signalrCapacity 1 -dockerVM Yes -githuburl "$giturl" -azureAdPreviewModuleUri "https://github.com/raw/dev/code/Azu.zip" -cosmosdbModuleUri "https://github.com/raw/dev/code.zip" -siteName "test"'
    deploymentMode: Validation
  condition: always()

• After validating the ARM templates, delete the resource group by using the following Azure CLI task.

steps:
- task: AzureCLI@1
  displayName: 'Deleting-validation-RG'
  inputs:
    azureSubscription: 'Subscription name (XXXX-XXXX-XXXX-XXXX-XXX)'
    scriptLocation: inlineScript
    inlineScript: 'az group delete -n $(rgname) --yes'

• Use this task built-in pipeline to publish the build artifacts to Azure pipelines stored in the Azure DevOps server.

steps:
- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: ARM_templates'
  inputs:
    PathtoPublish: /home/arm
    ArtifactName: 'ARM_templates'

• After everything is done, save and queue the build, and get the logs for success or failure of the build.

• Each task will generate a build log for the corresponding job.

Deploy an ARM Template to a Resource Group (CD)

In this example, we show deployment of three different solutions such as basic/standard/premium being deployed to the Azure cloud. First, a description of the solutions will be given, then the actual deployment pipeline will be discussed.

• The Basic solution will have all core components in an ARM template. Basic pricing tier efficiently works for development/test.

• The Standard solution has all the basic solution features plus has monitoring and HA (High Availability) features (Availability set, Availability zones, paired zones) to make application redundant at every level of failure, from an individual VM to the entire region.

• The Premium Solution has all the standard solution features plus automated disaster recovery in another region. Each Azure region is paired with another region within the same geography, together making a regional pair. To protect an application against a regional outage, deploy the application across multiple regions, using traffic manager to distribute internet traffic to the different regions.

Deployment Pipeline (CD)

Add Artifacts from source (Build) to deploy a release pipeline through multiple stages. Choose source type build as the one created above for validation.

Various stages can be added using the graphics tools in Azure DevOps. Note the parallel nature of these pipelines for each SKU.

ARM Template deployment (Basic, Standard, Premium):

• Select an Azure subscription to deploy Arm templates (Basic, Standard, Premium) in a resource group.

• Deployment mode should be Incremental for deploy ARM templates to a Resource group.

• Incremental mode handles deployments as incremental updates to the resource group. It leaves unchanged resources that exist in the resource group but is not specified in the template.

variables:
  basic_rg: 's_basic1'
  location: 'westus'

steps:
- task: AzureResourceGroupDeployment@2
  displayName: 'Basic Solution   Deployment'
  inputs:
    azureSubscription: 'CICD (XXXX-XXXX-XXXX)'
    resourceGroupName: '$(basic_rg)'
    location: '$(location)'
    csmFile: '$(System.DefaultWorkingDirectory)/_SnS_Build_CI/ARM_drop/maintemplate.json'
    overrideParameters: '-solutionType Basic-Solution -deploymentPrefix tere -cognitiveServicesLocation "eastus" -omsLocation "eastus" -appInsightsLocation "westus2" -locationDr westcentralus -trafficManagerName "NA" -b2cApplicationId XXXX-XXXX-XXX -b2cApplicationIdDR "NA" -b2cPolicy B2C_1_SignUp-In -b2cTenant onmicrosoft.com -b2cScope https://onmicrosoft.com/ -b2cScopeDR "NA" -videoIndexerKey "XXXX-XXXX-XXXX" -keyVaultName "NA" -keyVaultwebAppSecretName "NA" -keyVaultResourceGroup "NA" -webAppCertificatethumbPrint "NA"'
    deploymentOutputs: 'output_variables'

• After deploying the ARM templates, delete the resource group by using the following Azure CLI task.

variables:
  basic_rg: 'basic1'

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI '
  inputs:
    azureSubscription: 'CICD (XXXX-XXXX-XXXX-XXXX-XXXX)'
    scriptLocation: inlineScript
    inlineScript: 'az group delete -n $(basic_rg) --yes'

variables:
  standard_rg: 'ns_standard'

steps:
- task: AzureResourceGroupDeployment@2
  displayName: 'Standard solution '
  inputs:
    azureSubscription: 'CICD (XXX-XXXX-XXXX-XXX-XXX)'
    resourceGroupName: '$(standard_rg)'
    location: 'East US 2'
    csmFile: '$(System.DefaultWorkingDirectory)/__Build_CI/ARM_drop/main-template.json'
    overrideParameters: '-solutionType Standard-Solution -deploymentPrefix tere -cognitiveServicesLocation "eastus" -omsLocation "eastus" -appInsightsLocation "westus2" -locationDr westcentralus -trafficManagerName "traficmanagersns" -b2cApplicationId "XXXXXX" -b2cApplicationIdDR "NA" -b2cPolicy "B2C_1_b2csignup_in" -b2cTenant "snsiot.onmicrosoft.com" -b2cScope "https://abac.com " -b2cScopeDR "NA" -videoIndexerKey "XXXX-XXX-XXXX-XXX" -keyVaultName "NA" -keyVaultwebAppSecretName "NA" -keyVaultResourceGroup "NA" -webAppCertificatethumbPrint "NA"'

The standard solution will have two regions

• Primary Region(Deployment)
• Secondary Region (Re-Deployment)

• After deploying the ARM templates, delete the resource group by using the following Azure CLI task.

variables:
  standard_rg: 'ns_standard'

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI '
  inputs:
    azureSubscription: 'CICD (XXX-XXXX-XXX-XXX)'
    scriptLocation: inlineScript
    inlineScript: 'az group delete -n $(standard_rg) --yes'

variables:
  standard_rg: 'ns_premium'

steps:
- task: AzureResourceGroupDeployment@2
  displayName: 'premium solution'
  inputs:
    azureSubscription: 'CICD (XXXX-XXXX-XXXX-XXXX)'
    resourceGroupName: '$(premium_rg)'
    location: '$(location)'
    csmFile: '$(System.DefaultWorkingDirectory)/_SnS_Build_CI/ARM_drop/main-template.json'
    overrideParameters: '-solutionType Premium-Solution -deploymentPrefix "security" -cognitiveServicesLocation "eastus" -omsLocation "eastus" -appInsightsLocation "westus2" -locationDr westcentralus -trafficManagerName traficmanager -b2cApplicationId XXXX-XXXX-XXXX-XXX -b2cApplicationIdDR https://abac.com -b2cPolicy B2C_1_b2csignup_in -b2cTenant abc.onmicrosoft.com -b2cScope https://abc.com   -b2cScopeDR https://demo1  -videoIndexerKey XXXXXXX -keyVaultName "NA" -keyVaultwebAppSecretName "NA" -keyVaultResourceGroup "NA" -webAppCertificatethumbPrint "NA"'

After deploying the ARM templates, delete the resource group by using the following Azure CLI task.

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI '
  inputs:
    azureSubscription: 'CICD (XXX-XXXX-XXX-XXX)'
    scriptLocation: inlineScript
    inlineScript: 'az group delete -n $(premium_rg) --yes'

After everything is done, save and queue the release, and get the logs for success or failure of the release.

Each task will generate a deployment log for the corresponding job.

An azure-pipelines.yml is shown here:

# Build and release (CI/CD) the pipelines using a yaml editor.
# Build pipeline(CI) is to validate the ARM templates: 
pool:
  name: Hosted Ubuntu 1604
variables:
  rgname: 'abc'
steps:
- task: AzureResourceGroupDeployment@2
  displayName: 'Basic solution-template-validation'
  inputs:
    azureSubscription: 'subscription name(XXXX-XXXX-XXXX-XXXX-XXXX)'
    resourceGroupName: '$(rgname)'
    location: 'East US'
    csmFile: 'main-template.json'
    csmParametersFile: 'main-template.parameters.json'
    overrideParameters: '-solutionType "Project-Edison" -deploymenttype Standard -geo-paired-region "EastUS2" -signalRlocation eastus2 -acrDeploymentLocation "CanadaCentral" -omsWorkspaceRegion "southeastasia" -appInsightsLocation "eastus" -appInsightsLocationDr "southcentralus" -tenantId $(tid) -botAdClientId "XXXXXXXXXX" -adObjectId "XXXX-XXXX-XXX-XXX-XXXX" -adClientSecret "$(adsp)" -azureAccountName "abc@gmail.com" -azurePassword "abcacacc123@" -adminName "adminuser" -sessionId "3791180c-24c5-4290-8459-a454feee90ab" -vmUsername "adminuser" -vmPassword "Pass@1212" -aksServicePrincipalClientId "XXX-XXXXa63c-XXXX" -aksServicePrincipalClientSecret $(ksp) -signalrCapacity 1 -dockerVM Yes -githuburl "$giturl" -azureAdPreviewModuleUri "https://github.com/raw/dev/code/Azu.zip" -cosmosdbModuleUri "https://github.com/raw/dev/code.zip" -siteName "test"'
    deploymentMode: Validation
  condition: always()

steps:
- task: AzureCLI@1
  displayName: 'Deleting-validation-RG'
  inputs:
    azureSubscription: 'Subscription name (XXXX-XXXX-XXXX-XXXX-XXX)'
    scriptLocation: inlineScript
    inlineScript: 'az group delete -n $(rgname) --yes'

steps:
- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: ARM_templates'
  inputs:
    PathtoPublish: /home/arm
    ArtifactName: 'ARM_templates'

# Release(CD) pipeline is to deploy a resources like vms,aks cluster,webapps into Azure cloud.
variables:
  basic_rg: 's_basic1'
  location: 'westus'

steps:
- task: AzureResourceGroupDeployment@2
  displayName: 'Basic Solution   Deployment'
  inputs:
    azureSubscription: 'CICD (XXXX-XXXX-XXXX)'
    resourceGroupName: '$(basic_rg)'
    location: '$(location)'
    csmFile: '$(System.DefaultWorkingDirectory)/_SnS_Build_CI/ARM_drop/maintemplate.json'
    overrideParameters: '-solutionType Basic-Solution -deploymentPrefix tere -cognitiveServicesLocation "eastus" -omsLocation "eastus" -appInsightsLocation "westus2" -locationDr westcentralus -trafficManagerName "NA" -b2cApplicationId XXXX-XXXX-XXX -b2cApplicationIdDR "NA" -b2cPolicy B2C_1_SignUp-In -b2cTenant onmicrosoft.com -b2cScope https://onmicrosoft.com/ -b2cScopeDR "NA" -videoIndexerKey "XXXX-XXXX-XXXX" -keyVaultName "NA" -keyVaultwebAppSecretName "NA" -keyVaultResourceGroup "NA" -webAppCertificatethumbPrint "NA"'
    deploymentOutputs: 'output_variables'
variables:
  basic_rg: 'basic1'

steps:
- task: AzureCLI@1
  displayName: 'Azure CLI '
  inputs:
    azureSubscription: 'CICD (XXXX-XXXX-XXXX-XXXX-XXXX)'
    scriptLocation: inlineScript
    inlineScript: 'az group delete -n $(basic_rg) --yes'