Azure Blob Storage with private endpoint throws System.Xml.XmlException

When accessing Azure Blob Storage from intranet using private endpoint it may throw confusing System.Xml.XmlException with message “The ‘meta’ start tag on line 4 position 2 does not match the end tag of ‘head’. Line 118, position 3.”. This exceptions occurs when e. g. blob is uploaded to storage in following context:

  • Azure Blob Storage is accessed using private endpoint
  • Intranet network requires to use dedicated proxy when communicating with external network resources
  • NuGet packages Azure.Storage.Blobs and Azure.Identity are used to access Azure Blob Storage from .NET Core app

First step to resolve this issue is to enable logging adding code:

using var listener = AzureEventSourceListener.CreateConsoleLogger(EventLevel.Verbose);

In the detailed logs we can see that it is possible to access Azure AD endpoint https://login.microsoftonline.com, but try to access Azure Blob Storage endpoint returns error 503 Service Unavailable because application proxy is not set correctly. Body of HTTP response can’t be parsed as valid XML which causes exception. To resolve this issue the requests to Azure AD login.microsoftonline.com have to use proxy, but requests to Azure Blob Storage have to bypass proxy. We can achieve this adding code:

HttpClient.DefaultProxy = new WebProxy("myproxy", true, new[] { "mystorage.blob.core.windows.net" });

Now is possible to access Azure Blob Storage from .NET Core app.


Udemy course: Migrate Windows service to Azure

Deploy Windows service using Terraform

Terraform can be used for deployment of Windows service by combination of provisioner together with PowerShell cmdlets.


Udemy course: Migrate Windows service to Azure

Following Terraform code placed in main.tf file can be used to install and uninstall of Windows service:

variable "binary_path" {
  type = string
  description = "Binary path"
}

variable "service_name" {
  type = string
  description = "Service name"
}

resource "null_resource" "win_service" {
  triggers = {
    service_name = var.service_name
  }
  provisioner "local-exec" {
    when = create
    command = "New-Service -BinaryPathName ${var.binary_path} -Name ${var.service_name} -StartupType Automatic; Start-Service -Name ${var.service_name}"
    interpreter = ["PowerShell", "-Command"]
  }
  provisioner "local-exec" {
    when = destroy
    command = "Stop-Service -Name ${self.triggers.service_name}; (Get-WmiObject -Class Win32_Service -Filter \"Name='${self.triggers.service_name}'\").Delete()"
    interpreter = ["PowerShell", "-Command"]
  }
}

Install Windows service using:

terraform apply -var 'binary_path={binary_path}' -var 'service_name={service_name}' -auto-approve

Uninstall Windows service using:

terraform destroy -var 'binary_path={binary_path}' -var 'service_name={service_name}' -auto-approve

Before running this commands replace variables {binary_path} and {service_name} with required values.

Deploy Azure WebJob using Terraform

In my previous post I wrote about creating infrastructure for Azure WebJob using Terraform. Using that solution is created empty infrastructure ready for deployment of Azure WebJob, but the deployment of Azure WebJob have to be done separately.


Udemy course: Migrate Windows service to Azure

Terraform provides also options to deploy Azure WebJob immediately after required Azure resources are created. It is posible by combination of Terraform provisioner together with Azure CLI command az webapp deployment as demonstrated in following code snippet:

resource "null_resource" "webjob" {
  provisioner "local-exec" {
    when = create
    command = "az webapp deployment source config-zip -g ${var.resource_group_name} -n ${var.resource_group_name} --src ${var.deployment_package}"
  }
  depends_on = [ azurerm_app_service.as ]
}

Terraform provisioner is executed locally at creation of resource and is placed in null_resource which has dependency on azurerm_app_service.as resource. Complete main.tf file which is responsible for creating Azure resources and then deployment of Azure WebJob is:

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = ">= 2.0"
    }
  }
}

provider "azurerm" {
  features {}
}

variable "resource_group_name" {
  type = string
  description = "Resource group name"
  default = "WinServiceToAzureTest"
}

variable "location_name" {
  type = string
  description = "Resource location"
  default = "westeurope"
}

variable "deployment_package" {
  type = string
  description = "Deployment package"
  default = "Publish.zip"
}

resource "azurerm_resource_group" "rg" {
  name = var.resource_group_name
  location = var.location_name
}

resource "azurerm_app_service_plan" "asp" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  sku {
    tier = "Basic"
    size = "B1"
  }
}

resource "azurerm_app_service" "as" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.asp.id
  client_affinity_enabled = false
  site_config {
    use_32_bit_worker_process = false
    always_on = true
  }
  app_settings = {
    "APPINSIGHTS_INSTRUMENTATIONKEY" = azurerm_application_insights.ai.instrumentation_key
  }
  connection_string {
    name = "AzureWebJobsDashboard"
    type = "Custom"
    value = "DefaultEndpointsProtocol=https;AccountName=${lower(var.resource_group_name)};AccountKey=${azurerm_storage_account.sa.primary_access_key};EndpointSuffix=core.windows.net"
  }
  connection_string {
    name = "AzureWebJobsStorage"
    type = "Custom"
    value = "DefaultEndpointsProtocol=https;AccountName=${lower(var.resource_group_name)};AccountKey=${azurerm_storage_account.sa.primary_access_key};EndpointSuffix=core.windows.net"
  }  
}

resource "azurerm_storage_account" "sa" {
  name = lower(var.resource_group_name)
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name  
  account_tier = "Standard"
  account_replication_type = "LRS"
}

resource "azurerm_application_insights" "ai" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  application_type = "web"  
}

resource "null_resource" "webjob" {
  provisioner "local-exec" {
    when = create
    command = "az webapp deployment source config-zip -g ${var.resource_group_name} -n ${var.resource_group_name} --src ${var.deployment_package}"
  }
  depends_on = [ azurerm_app_service.as ]
}

If you are interested in Azure WebJobs and how to use them to migrage Windows service to Azure, take my Udemy course Migrate Windows service to Azure.

Create infrastructure for Azure WebJob using Terraform

In one of my post I wrote about creating infrastructure for Azure WebJob using PowerShell. It is imperative approach using PowerShell cmdlets. More DevOps friendly alternative to this is to use declarative approach together with infrastructure as code approach using Terraform. Infrastructure for Azure WebJob consists of following resources:

  • App Service Plan – scalable cluster of web servers
  • Web App – hosting environment running on App Service plan
  • Storage Account – stores data about Azure WebJob execution
  • Application Insights – monitoring


Udemy course: Migrate Windows service to Azure

To create required Azure resources create file named main.tf and place following content into it:

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = ">= 2.0"
    }
  }
}

provider "azurerm" {
  features {}
}

variable "resource_group_name" {
  type = string
  description = "Resource group name"
  default = "WinServiceToAzureTest"
}

variable "location_name" {
  type = string
  description = "Resource location"
  default = "westeurope"
}

resource "azurerm_resource_group" "rg" {
  name = var.resource_group_name
  location = var.location_name
}

resource "azurerm_app_service_plan" "asp" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  sku {
    tier = "Basic"
    size = "B1"
  }
}

resource "azurerm_app_service" "as" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.asp.id
}

resource "azurerm_storage_account" "sa" {
  name = lower(var.resource_group_name)
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name  
  account_tier = "Standard"
  account_replication_type = "LRS"
}

resource "azurerm_application_insights" "ai" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  application_type = "web"  
}

Then run terraform init command to download providers and initialize state. To see planned infrastructure changes run the terraform plan command. As a final step the infrastructure will be created using terraform apply command.
Azure WebJob requires some additional settings which can be done manually using Azure Portal, but it is possible to implement them in code. Next code snippet shows changes in azurerm_app_service resource where is disabled client affinity, added site config, instrumentation key for Application Insights and connection strings to Storage Account:

resource "azurerm_app_service" "as" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.asp.id
  client_affinity_enabled = false
  site_config {
    use_32_bit_worker_process = false
    always_on = true
  }
  app_settings = {
    "APPINSIGHTS_INSTRUMENTATIONKEY" = azurerm_application_insights.ai.instrumentation_key
  }
  connection_string {
    name = "AzureWebJobsDashboard"
    type = "Custom"
    value = "DefaultEndpointsProtocol=https;AccountName=${lower(var.resource_group_name)};AccountKey=${azurerm_storage_account.sa.primary_access_key};EndpointSuffix=core.windows.net"
  }
  connection_string {
    name = "AzureWebJobsStorage"
    type = "Custom"
    value = "DefaultEndpointsProtocol=https;AccountName=${lower(var.resource_group_name)};AccountKey=${azurerm_storage_account.sa.primary_access_key};EndpointSuffix=core.windows.net"
  }  
}

With this changes are Azure resources prepared for deployment of Azure WebJob.

If you are interested in Azure WebJobs and how to use them to migrage Windows service to Azure, take my Udemy course Migrate Windows service to Azure.