Leveraging ARM templates for deploying Azure CosmosDB

The other day I stumbled upon the brand new (but long awaited) ARM support for CosmosDB databases and collections. CosmosDB ARM support used to be limited to just provisioning the database account. Everything inside it, such as databases and collections, had to be provisioned using some other mechanism, such as PowerShell or Azure CLI – or heaven forbid, the portal.

But that’s over: support for ARM is finally here! It’s not all perfect yet, though. I guess Rome wasn’t build in a day either, so let’s count our blessings – and find workarounds for what’s still missing.

One of those workarounds has to do with provisioning and updating throughput (i.e. RU/s) on either a database or a collection. Provisioning throughput upon creating a new database can be done by setting an options object with a throughput property in the database resource for example, like so:


{
"type": "Microsoft.DocumentDB/databaseAccounts/apis/databases",
"name": "[concat(variables('databaseAccountName'), '/sql/', variables('databaseName'))]",
"apiVersion": "2016-03-31",
"dependsOn": [ "[resourceId('Microsoft.DocumentDB/databaseAccounts/', variables('databaseAccountName'))]" ],
"properties": {
"resource": {
"id": "[variables('databaseName')]"
},
"options": {
"throughput": "400"
}
}
}

But, changing that value after initial creation is not allowed. Updates to that value can instead be passed through a nested settings resource, like this:


{
"type": "Microsoft.DocumentDB/databaseAccounts/apis/databases/settings",
"name": "[concat(variables('databaseAccountName'), '/sql/', variables('databaseName'), '/throughput')]",
"apiVersion": "2016-03-31",
"dependsOn": [
"[resourceId('Microsoft.DocumentDB/databaseAccounts/apis/databases', variables('databaseAccountName'), 'sql', variables('databaseName'))]"
],
"properties": {
"resource": {
"throughput": "500"
}
}
}

And this settings resource, in turn, is only valid as a child to a parent resource that is itself already provisioned with throughput – so a template only holding a settings resource with throughput in it (e.g. without the options object) fails upon creating. So, it seems that updating throughput requires a different template than the one that was used for the initial creation. That’s not how I like my ARM templates…

So I set out to devise a workaround, where the end goal is to have a deployment pipeline that uses ARM wherever possible, and that can be run multiple times yielding the same result, i.e. is idempotent. I found that, while you need to use the options object to initially provision the throughput, the throughput is allowed to be set to null upon subsequent deployments. This can be leveraged by some piece of simple logic, that conditionally sets this value to either the specified throughput, or null, depending on whether it’s an update or a create:


{
"type": "Microsoft.DocumentDB/databaseAccounts/apis/databases",
"name": "[concat(variables('databaseAccountName'), '/sql/', variables('databaseName'))]",
"apiVersion": "2016-03-31",
"dependsOn": [ "[resourceId('Microsoft.DocumentDB/databaseAccounts/', variables('databaseAccountName'))]" ],
"properties": {
"resource": {
"id": "[variables('databaseName')]"
},
"options": {
"throughput": "[if(parameters('isUpdate'), json('null'), parameters('throughput'))]"
}
}
}

Now, I just need to pass that flag to the template from the outside. For that, I can use an Azure CLI or PowerShell task to determine whether the database already exists, and pass the result of that into the ARM template as an input parameter. I won’t go into the details here, but this should be easy to implement.

Obviously, this is not ideal. I’m sacrificing the decoupling of my ARM template from other tasks in my pipeline: the template depends on another task to provide input instead of just a parameter file, and I no longer have the option to let the ARM template figure out the database name based on naming conventions or whatever, because I need to know that name up front in order to be able to pass it to the CLI / PowerShell task. But, putting it all together, I do have a single template used for creates and updates, and a single place (i.e. the parameter file) to keep track of the throughput I provisioned. To me, that’s something worth a sacrifice.

The full ARM template looks like this:


{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"isUpdate": {
"type": "bool"
},
"throughput": {
"type": "int",
"minValue": 400,
"maxValue": 1000000
}
},
"variables": {
"databaseAccountName": "docsdbaccount",
"databaseName": "docsdb",
"docsCollectionName": "docs"
},
"resources": [
{
"apiVersion": "2015-04-08",
"kind": "GlobalDocumentDB",
"location": "[resourceGroup().location]",
"name": "[variables('databaseAccountName')]",
"properties": {
"name": "[variables('databaseAccountName')]",
"databaseAccountOfferType": "Standard",
"locations": [
{
"failoverPriority": 0,
"locationName": "[resourceGroup().location]"
}
]
},
"tags": {
"defaultExperience": "DocumentDB"
},
"type": "Microsoft.DocumentDB/databaseAccounts"
},
{
"type": "Microsoft.DocumentDB/databaseAccounts/apis/databases",
"name": "[concat(variables('databaseAccountName'), '/sql/', variables('databaseName'))]",
"apiVersion": "2016-03-31",
"dependsOn": [ "[resourceId('Microsoft.DocumentDB/databaseAccounts/', variables('databaseAccountName'))]" ],
"properties": {
"resource": {
"id": "[variables('databaseName')]"
},
"options": {
"throughput": "[if(parameters('isUpdate'), json('null'), parameters('throughput'))]"
}
},
"resources": [
{
"type": "Microsoft.DocumentDB/databaseAccounts/apis/databases/settings",
"name": "[concat(variables('databaseAccountName'), '/sql/', variables('databaseName'), '/throughput')]",
"apiVersion": "2016-03-31",
"dependsOn": [
"[resourceId('Microsoft.DocumentDB/databaseAccounts/apis/databases', variables('databaseAccountName'), 'sql', variables('databaseName'))]"
],
"properties": {
"resource": {
"throughput": "[parameters('throughput')]"
}
}
},
{
"type": "Microsoft.DocumentDb/databaseAccounts/apis/databases/containers",
"name": "[concat(variables('databaseAccountName'), '/sql/', variables('databaseName'), '/', variables('docsCollectionName'))]",
"apiVersion": "2016-03-31",
"dependsOn": [
"[resourceId('Microsoft.DocumentDB/databaseAccounts/apis/databases', variables('databaseAccountName'), 'sql', variables('databaseName'))]"
],
"properties": {
"resource": {
"id": "[variables('docsCollectionName')]"
}
}
}
]
}
],
"outputs": { }
}

Hope this helps!

Advertisement

Enabling Azure App Service VNet Integration ‘v2’ from CI/CD

If you’re anything like me, you want to automate everything from deploying your basic Azure infrastructure all the way to the application code. And the bar is set exceedingly high for deviations to this rule.

Sometimes – and especially with new and/or preview features – that requires some extra work, because support for such features in your deployment technology of choice may not yet be available.

Take the case of the new VNet Integration feature for Azure Web Apps. All very cool that it can be set through the portal, but that’s about the last thing I want to do when creating robust deployments. For me, ARM templates are the technology of choice when it comes to deploying Azure resources, even though a case can be made for alternatives such as Azure CLI. But in the case of VNet integration, both ARM and the Azure CLI are not an option yet at the time of writing. There aren’t even proper Azure Powershell commands to get this done.

In situations like this, I head over to the Azure Resource Explorer, and see if I can reverse-engineer what happens in the Azure Resource Manager API when I change a setting through the portal. Armed with that knowledge, I’m able to craft an API call that gets the job done. Wrapped in a fairly simple Powershell script, it may end up like this:


Param(
[Parameter(Mandatory=$true)]
[string]$subnetResourceId, ## Something along the lines of '/subscriptions/[subscriptionid]/resourceGroups/[resourceGroupName]/providers/Microsoft.Network/virtualNetworks/[vnetname]/subnets/[subnetname]'
[Parameter(Mandatory=$true)]
[string]$webappName
)
function Get-AccessToken {
$context = Get-AzureRmContext
$tokenCache = $context.TokenCache
$cachedTokens = $tokenCache.ReadItems() `
| Sort-Object Property ExpiresOn Descending
$accessToken = $cachedTokens[0].AccessToken
$accessToken
}
function Set-VNetIntegration {
$app = Get-AzureRMWebApp Name $webappName
$resourceGroup = $app.ResourceGroup
$location = $app.Location
$body = "{
""location"": ""$location"",
""properties"": {
""subnetResourceId"": ""$subnetResourceId""
}
}"
$url = "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroup/providers/Microsoft.Web/sites/$webappName/config/virtualNetwork?api-version=2018-02-01"
$accessToken = Get-AccessToken
Invoke-RestMethod Uri $url Method PUT Headers @{Authorization = "Bearer $accessToken"} ContentType 'application/json' Body $body
}
$subscriptionId = (Get-AzureRMContext).Subscription.Id
if ($subscriptionId -eq $null) {
throw "Not logged in. Please login using Connect-AzureRmAccount and select the correct subscription using Select-AzureRmSubscription; then try again"
}
Set-VNetIntegration

Some remarks here are in order, the most important of which is that the subnet with which the app is integrated, must be preconfigured with a delegation to Microsoft.Web/serverFarms. This is done automatically when you enable VNet Integration through the portal, but not when making the API call yourself like in the script above. Fortunately, this actually is settable through ARM, so no need to include it in the script.

Second, the script works with the ‘old’ AzureRM modules for all Azure interactions apart from the actual API call. For me, this works best for interoperability with Azure DevOps hosted agents, which didn’t support the new Az modules yet at the time I created this script. But of course it can quite easily be adapted to the new modules.

And lastly, this is not really pretty and production-ready code (but then again, is Powershell ever pretty?). It can certainly be improved upon, but for me this is a piece of dispensable code in the sense that I take it out of my pipeline and discard it, the minute ARM or Azure CLI support becomes available. It’s just that I don’t feel like waiting for that to happen before I can automate deployments that use this feature.

Hope this helps anyone looking for a way to automate the new VNet Integration feature, and just maybe it can also serve as a bit of inspiration as to how you could create your own temporary solutions to features not available in your primary deployment technology of choice.