Configure Azure Diagnostic Settings with Azure Policies

Configure Azure Diagnostic Settings with Azure Policies

Configure Azure Diagnostic Settings with Azure Policies

Table of Contents

1 Objective
2 Diagnostic Settings in Azure Portal
3 Policy
3.1 Policy Rule
3.2 Policy Set Definitions
4 Permissions
5 Deployment
6 Policy in Azure Portal

1 Objective

Azure resources had to be configured with diagnostic settings. To apply these settings the general approach with Azure Policies had the most advantages:

  • Management with ARM Templates
  • Check for Compliance
  • CI/CD Integration

Azure Diagnostic Settings can be configured in several ways:

2 Diagnostic Settings in Azure Portal

The screenshots below display the diagnostic settings (logs and metrics) for a Cosmos DB account.

Diagnostic Settings

Diagnostic Settings

The Azure diagnostic setting ARM template has to be integrated into the Policy ARM Template.

3 Policy

A policy contains different elements.

3.1 Policy Rule

Each resource type needs a Policy Rule.

I my case, I configured Policies for following resource types:

  • Microsoft.KeyVault/vaults
  • Microsoft.EventGrid/topics
  • Microsoft.EventGrid/eventSubscriptions
  • Microsoft.Web/sites
  • Microsoft.ServiceBus/namespaces
  • Microsoft.Network/networkSecurityGroups
  • Microsoft.Network/applicationGateways

3.1.1 Example policy rule type

Extract from policyRule:

          "policyRule": {
  "if": {
    "field": "type",
    "equals": "Microsoft.DocumentDB/databaseAccounts"
  },
  "then": {
    "effect": "[[parameters('effect')]",
    "details": {
      "type": "Microsoft.Insights/diagnosticSettings",
      "name": "[[parameters('profileName')]",
      "existenceCondition": {
        "allOf": [
          {
            "field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
            "equals": "[[parameters('logsEnabled')]"
          },
          {
            "field": "Microsoft.Insights/diagnosticSettings/metrics.enabled",
            "equals": "[[parameters('metricsEnabled')]"
          },
          {
            "field": "Microsoft.Insights/diagnosticSettings/storageAccountId",
            "equals": "[[parameters('storageExists')]"
          }
        ]
      },
      "roleDefinitionIds": [
        "/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa",
        "/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293"
      ]
    }
  }
}
3.1.1.1 existenceCondition

Based on the parameter values, the diagnostic settings for logging, metric and storage logging will be checked.

3.1.1.2 roleDefinitionIds

Azure built-in roles are required to set proper permissions:

3.1.2

The deployment property contains the logs and metrics.

Extract from policyRule:

{
  "policyRule": {
    "if": {
    },
    "then": {
      "deployment": {
        "properties": {
          "mode": "incremental",
          "template": {
            "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
            "contentVersion": "1.0.0.0",
            "parameters": {
              "resourceName": {
                "type": "string"
              },
              "location": {
                "type": "string"
              },
              "logAnalytics": {
                "type": "string"
              },
              "archiveStorage": {
                "type": "string"
              },
              "metricsEnabled": {
                "type": "string"
              },
              "logsEnabled": {
                "type": "string"
              },
              "storageExists": {
                "type": "string"
              },
              "profileName": {
                "type": "string"
              }
            },
            "variables": {},
            "resources": [
              {
                "type": "Microsoft.DocumentDB/databaseAccounts/providers/diagnosticSettings",
                "apiVersion": "2017-05-01-preview",
                "name": "[[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileName'))]",
                "location": "[[parameters('location')]",
                "dependsOn": [],
                "properties": {
                  "workspaceId": "[[parameters('logAnalytics')]",
                  "storageAccountId": "[[parameters('archiveStorage')]",
                  "metrics": [
                    {
                      "category": "Requests",
                      "enabled": true,
                      "retentionPolicy": {
                        "days": 0,
                        "enabled": false
                      },
                      "timeGrain": null
                    }
                  ],
                  "logs": [
                    {
                      "category": "DataPlaneRequests",
                      "enabled": true
                    },
                    {
                      "category": "QueryRuntimeStatistics",
                      "enabled": true
                    },
                    {
                      "category": "PartitionKeyStatistics",
                      "enabled": true
                    },
                    {
                      "category": "PartitionKeyRUConsumption",
                      "enabled": true
                    },
                    {
                      "category": "ControlPlaneRequests",
                      "enabled": true
                    },
                    {
                      "category": "GremlinRequests",
                      "enabled": true
                    }
                  ]
                }
              }
            ],
            "outputs": {}
          },
          "parameters": {
            "location": {
              "value": "[[field('location')]"
            },
            "resourceName": {
              "value": "[[field('name')]"
            },
            "logAnalytics": {
              "value": "[[parameters('logAnalytics')]"
            },
            "archiveStorage": {
              "value": "[[parameters('archiveStorage')]"
            },
            "metricsEnabled": {
              "value": "[[parameters('metricsEnabled')]"
            },
            "logsEnabled": {
              "value": "[[parameters('logsEnabled')]"
            },
            "storageExists": {
              "value": "[[parameters('storageExists')]"
            },
            "profileName": {
              "value": "[[parameters('profileName')]"
            }
          }
        }
      }
    }
  }
}

3.2 Policy Set Definitions

An Azure Policy Initiative is a set of policies.

Initiative - Policies

Extract with the Policy Set Definition:

{
  "type": "Microsoft.Authorization/policySetDefinitions",
  "name": "[variables('diagnosticSettingsInitiativeName')]",
  "apiVersion": "2019-09-01",
  "dependsOn": [
  ],
  "properties": {
    "displayName": "MarkusMeyer - Apply diagnostic settings for applicable resources - Log Analytics",
    "description": "This initiative configures application Azure resources to forward diagnostic logs and metrics to an Azure Log Analytics workspace.",
    "metadata": {
      "category": "Monitoring"
    },
    "parameters": {
      "logAnalytics": {
        "type": "string",
        "metadata": {
          "displayName": "Log Analytics workspace",
          "description": "Select the Log Analytics workspace from dropdown list",
          "strongType": "omsWorkspace"
        }
      }
    },
    "policyDefinitions": [
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('kvLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      },
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('sbLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      },
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('cosmosLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      },
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('funcLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      },
      {
        "policyDefinitionId": "[resourceId('Microsoft.Authorization/policyDefinitions/', variables('evgTopicLogPolicyName'))]",
        "parameters": {
          "logAnalytics": {
            "value": "[[parameters('logAnalytics')]"
          }
        }
      }
    ]
  }
}

The complete template can be found in my GitHub repository.

4 Permissions

Following permissions are required to deploy the policies:

ActionsDescription
*/readRead resources of all types, except secrets.
Microsoft.Authorization/policyassignments/*Create and manage policy assignments
Microsoft.Authorization/policydefinitions/*Create and manage policy definitions
Microsoft.Authorization/policyexemptions/*
Microsoft.Authorization/policysetdefinitions/*Create and manage policy sets
Microsoft.PolicyInsights/*
Microsoft.Support/*Create and update a support ticket

5 Deployment

Ths policies will be deployed on subscription level

Policies can also be deployed to:

5.1 Azure CLI

az deployment sub create  --location westeurope --template-file .\Policy.json --parameters "@Policy.parameters.json"

5.2 Powershell

New-AzDeployment -Name "diagPolicies" -location "West Europe" -TemplateFile .\Policy.json  -verbose -TemplateParameterFile .\Policy.parameters.json

Deploy To Azure

6 Policy in Azure Portal

The deployed policies will be display in Azure Portal.

Deploy To Azure