Azure Blob Storage with private endpoint throws System.Xml.XmlException

When accessing Azure Blob Storage from intranet using private endpoint it may throw confusing System.Xml.XmlException with message “The ‘meta’ start tag on line 4 position 2 does not match the end tag of ‘head’. Line 118, position 3.”. This exceptions occurs when e. g. blob is uploaded to storage in following context:

  • Azure Blob Storage is accessed using private endpoint
  • Intranet network requires to use dedicated proxy when communicating with external network resources
  • NuGet packages Azure.Storage.Blobs and Azure.Identity are used to access Azure Blob Storage from .NET Core app

First step to resolve this issue is to enable logging adding code:

using var listener = AzureEventSourceListener.CreateConsoleLogger(EventLevel.Verbose);

In the detailed logs we can see that it is possible to access Azure AD endpoint https://login.microsoftonline.com, but try to access Azure Blob Storage endpoint returns error 503 Service Unavailable because application proxy is not set correctly. Body of HTTP response can’t be parsed as valid XML which causes exception. To resolve this issue the requests to Azure AD login.microsoftonline.com have to use proxy, but requests to Azure Blob Storage have to bypass proxy. We can achieve this adding code:

HttpClient.DefaultProxy = new WebProxy("myproxy", true, new[] { "mystorage.blob.core.windows.net" });

Now is possible to access Azure Blob Storage from .NET Core app.


Udemy course: Migrate Windows service to Azure

Deploy Azure WebJob using Terraform

In my previous post I wrote about creating infrastructure for Azure WebJob using Terraform. Using that solution is created empty infrastructure ready for deployment of Azure WebJob, but the deployment of Azure WebJob have to be done separately.


Udemy course: Migrate Windows service to Azure

Terraform provides also options to deploy Azure WebJob immediately after required Azure resources are created. It is posible by combination of Terraform provisioner together with Azure CLI command az webapp deployment as demonstrated in following code snippet:

resource "null_resource" "webjob" {
  provisioner "local-exec" {
    when = create
    command = "az webapp deployment source config-zip -g ${var.resource_group_name} -n ${var.resource_group_name} --src ${var.deployment_package}"
  }
  depends_on = [ azurerm_app_service.as ]
}

Terraform provisioner is executed locally at creation of resource and is placed in null_resource which has dependency on azurerm_app_service.as resource. Complete main.tf file which is responsible for creating Azure resources and then deployment of Azure WebJob is:

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = ">= 2.0"
    }
  }
}

provider "azurerm" {
  features {}
}

variable "resource_group_name" {
  type = string
  description = "Resource group name"
  default = "WinServiceToAzureTest"
}

variable "location_name" {
  type = string
  description = "Resource location"
  default = "westeurope"
}

variable "deployment_package" {
  type = string
  description = "Deployment package"
  default = "Publish.zip"
}

resource "azurerm_resource_group" "rg" {
  name = var.resource_group_name
  location = var.location_name
}

resource "azurerm_app_service_plan" "asp" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  sku {
    tier = "Basic"
    size = "B1"
  }
}

resource "azurerm_app_service" "as" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.asp.id
  client_affinity_enabled = false
  site_config {
    use_32_bit_worker_process = false
    always_on = true
  }
  app_settings = {
    "APPINSIGHTS_INSTRUMENTATIONKEY" = azurerm_application_insights.ai.instrumentation_key
  }
  connection_string {
    name = "AzureWebJobsDashboard"
    type = "Custom"
    value = "DefaultEndpointsProtocol=https;AccountName=${lower(var.resource_group_name)};AccountKey=${azurerm_storage_account.sa.primary_access_key};EndpointSuffix=core.windows.net"
  }
  connection_string {
    name = "AzureWebJobsStorage"
    type = "Custom"
    value = "DefaultEndpointsProtocol=https;AccountName=${lower(var.resource_group_name)};AccountKey=${azurerm_storage_account.sa.primary_access_key};EndpointSuffix=core.windows.net"
  }  
}

resource "azurerm_storage_account" "sa" {
  name = lower(var.resource_group_name)
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name  
  account_tier = "Standard"
  account_replication_type = "LRS"
}

resource "azurerm_application_insights" "ai" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  application_type = "web"  
}

resource "null_resource" "webjob" {
  provisioner "local-exec" {
    when = create
    command = "az webapp deployment source config-zip -g ${var.resource_group_name} -n ${var.resource_group_name} --src ${var.deployment_package}"
  }
  depends_on = [ azurerm_app_service.as ]
}

If you are interested in Azure WebJobs and how to use them to migrage Windows service to Azure, take my Udemy course Migrate Windows service to Azure.

Create infrastructure for Azure WebJob using Terraform

In one of my post I wrote about creating infrastructure for Azure WebJob using PowerShell. It is imperative approach using PowerShell cmdlets. More DevOps friendly alternative to this is to use declarative approach together with infrastructure as code approach using Terraform. Infrastructure for Azure WebJob consists of following resources:

  • App Service Plan – scalable cluster of web servers
  • Web App – hosting environment running on App Service plan
  • Storage Account – stores data about Azure WebJob execution
  • Application Insights – monitoring


Udemy course: Migrate Windows service to Azure

To create required Azure resources create file named main.tf and place following content into it:

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = ">= 2.0"
    }
  }
}

provider "azurerm" {
  features {}
}

variable "resource_group_name" {
  type = string
  description = "Resource group name"
  default = "WinServiceToAzureTest"
}

variable "location_name" {
  type = string
  description = "Resource location"
  default = "westeurope"
}

resource "azurerm_resource_group" "rg" {
  name = var.resource_group_name
  location = var.location_name
}

resource "azurerm_app_service_plan" "asp" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  sku {
    tier = "Basic"
    size = "B1"
  }
}

resource "azurerm_app_service" "as" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.asp.id
}

resource "azurerm_storage_account" "sa" {
  name = lower(var.resource_group_name)
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name  
  account_tier = "Standard"
  account_replication_type = "LRS"
}

resource "azurerm_application_insights" "ai" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  application_type = "web"  
}

Then run terraform init command to download providers and initialize state. To see planned infrastructure changes run the terraform plan command. As a final step the infrastructure will be created using terraform apply command.
Azure WebJob requires some additional settings which can be done manually using Azure Portal, but it is possible to implement them in code. Next code snippet shows changes in azurerm_app_service resource where is disabled client affinity, added site config, instrumentation key for Application Insights and connection strings to Storage Account:

resource "azurerm_app_service" "as" {
  name = var.resource_group_name
  location = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  app_service_plan_id = azurerm_app_service_plan.asp.id
  client_affinity_enabled = false
  site_config {
    use_32_bit_worker_process = false
    always_on = true
  }
  app_settings = {
    "APPINSIGHTS_INSTRUMENTATIONKEY" = azurerm_application_insights.ai.instrumentation_key
  }
  connection_string {
    name = "AzureWebJobsDashboard"
    type = "Custom"
    value = "DefaultEndpointsProtocol=https;AccountName=${lower(var.resource_group_name)};AccountKey=${azurerm_storage_account.sa.primary_access_key};EndpointSuffix=core.windows.net"
  }
  connection_string {
    name = "AzureWebJobsStorage"
    type = "Custom"
    value = "DefaultEndpointsProtocol=https;AccountName=${lower(var.resource_group_name)};AccountKey=${azurerm_storage_account.sa.primary_access_key};EndpointSuffix=core.windows.net"
  }  
}

With this changes are Azure resources prepared for deployment of Azure WebJob.

If you are interested in Azure WebJobs and how to use them to migrage Windows service to Azure, take my Udemy course Migrate Windows service to Azure.

Convert Windows service to Azure Function

In the past I wrote a post how to migrate Windows service to Azure using Azure WebJobs. Another option for migration of Windows service to Azure is to use Azure Functions.


Udemy course: Migrate Windows service to Azure

Windows service I want to migrate to Azure acts as runtime for execution of scheduled jobs using Timer, which run every 60 seconds to execute scheduled jobs. JobSchedulerService execute JobExecutor.Execute() method every 60 seconds as visible in source code of Windows service:

public partial class JobSchedulerService : ServiceBase
{
    private Timer timer;

    public JobSchedulerService()
    {
        InitializeComponent();
    }

    protected override void OnStart(string[] args)
    {
        timer = new Timer();
        timer.Elapsed += OnTimer;
        timer.Interval = 60 * 1000;
        timer.Start();
    }

    protected override void OnStop()
    {
        timer.Stop();
    }

    private void OnTimer(object sender, ElapsedEventArgs e)
    {
        JobExecutor.Execute();
    }
}

To convert Windows service to Azure Function, add new project to Visual Studio solution:

Visual Studio - Add a new project

Select Azure Functions v1 (.NET Framework) as version, because Windows service is build on .NET Framework 4.7.2. Then select Timer trigger, none Storage Account and enter schedule “0 * * * * *”, which is CRON expression for triggering run every 60 seconds.

Visual Studio - Create a new Azure Functions Application

Code changes in function are simple, only call of JobExecutor.Execute() is added into Run method:

public static class JobSchedulerFunction
{
    [FunctionName("JobSchedulerFunction")]
    public static void Run([TimerTrigger("0 * * * * *")]TimerInfo timer, TraceWriter log)
    {
        JobExecutor.Execute();
    }
}

Next step is to create Azure resources to run Azure Function using Azure CLI:

# Before running this script check if Application Insights extension is installed:
# az extension show --name application-insights
# If not installed, install it by:
# az extension add --name application-insights

az group create --name WinServiceToAzureTest --location westeurope

az storage account create --resource-group WinServiceToAzureTest --name winservicetoazuretest --sku Standard_LRS --kind StorageV2
az monitor app-insights component create --resource-group WinServiceToAzureTest --app WinServiceToAzureTest --location westeurope --kind web

az functionapp create --name WinServiceToAzureTest --resource-group WinServiceToAzureTest --storage-account winservicetoazuretest --app-insights WinServiceToAzureTest --consumption-plan-location westeurope --runtime dotnet
az functionapp config appsettings set --name WinServiceToAzureTest --resource-group WinServiceToAzureTest --settings "FUNCTIONS_EXTENSION_VERSION=~1"

After finishing of run is created new Resource Group, Storage Account, Application Insights and finally Function App. Important part is here to set Azure Functions runtime to version 1 using application setting FUNCTIONS_EXTENSION_VERSION.
Deployment of Azure Functions to Azure can be done by combination of PowerShell and Azure CLI:

Compress-Archive -Path .\bin\Release\net472\* -DestinationPath WindowsServiceToAzure.Job.Function.zip
az functionapp deployment source config-zip --resource-group WinServiceToAzureTest --name WinServiceToAzureTest --src .\WindowsServiceToAzure.Job.Function.zip

After deployment is Azure Function running, which is visible on Function App dashboard:

Azure Portal - Function App

Azure Function integration diagram displays function input and outputs:

Azure Portal - Function Integration

Detailed logs about the activity of Azure Function is visible in monitor:

Azure Portal - Function Monitor

Cleanup of used Azure resources can be done using Azure CLI:

az group delete --name WinServiceToAzureTest --yes -y

If you are interested in Windows services and how to migrate them to Azure, take my Udemy course Migrate Windows service to Azure.

Create infrastructure for Azure WebJob using PowerShell

Azure infrastructure for Azure WebJob consists from following resources:

  • App Service Plan – scalable cluster of web servers
  • Web App – hosting environment running on App Service plan
  • Storage Account – stores data about Azure WebJob execution
  • Application Insights – monitoring

All there resources can be created using following PowerShell script:

param
(
    $resourceGroup = $(throw "Resource group is required"),
    $location = "westeurope"
)

$resourceGroupLower = $resourceGroup.ToLower()

New-AzResourceGroup -Name $resourceGroup -Location $location -ErrorAction Stop

New-AzAppServicePlan -Name $resourceGroup -ResourceGroupName $resourceGroup `
    -Location $location -Tier Basic -WorkerSize Small -NumberofWorkers 1 -ErrorAction Stop

New-AzWebApp -Name $resourceGroupLower -ResourceGroupName $resourceGroup `
    -Location $location -AppServicePlan $resourceGroup -ErrorAction Stop

New-AzStorageAccount -Name $resourceGroupLower -ResourceGroupName $resourceGroup `
    -Location $location -SkuName Standard_LRS -Kind StorageV2 -ErrorAction Stop

New-AzApplicationInsights -Name $resourceGroup -ResourceGroupName $resourceGroup `
    -Location $location -Kind web -ErrorAction Stop

PowerShell script has following parameters:

  • $resourceGroup – resource group name
  • $location – location name with West Europe as default value


Udemy course: Improve your productivity with PowerShell