[QUERY]Custom metrics not published when running in Container Instances?

[marc-lauper]

Library name and version

OpenTelemetry 1.13.1

Query/Question

I have a dotnet application which publishes custom metrics, as per the following example.

When running locally on my computer, the custom metrics are properly created in Application Insights. However, when running in Container Instances, Application Insights seems to not receive anything. The APPLICATIONINSIGHTS_CONNECTION_STRING is exactly the same in both cases. All calls to the SDK are successful, with no exceptions thrown whatsoever.

What I'm wondering is, is there anything specific that needs to be done to enable the usage of custom metrics from a Container Instance? Or are there specific steps I could take to pinpoint what's preventing the metrics to reaching Application Insights?

Environment

When running locally

> dotnet --info
.NET SDK:
 Version:           9.0.203   
 Commit:            dc7acfa194
 Workload version:  9.0.200-manifests.12d79ccf
 MSBuild version:   17.13.20+a4ef1e90f

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19045
 OS Platform: Windows
 RID:         win-x64
 Base Path:   C:\Program Files\dotnet\sdk\9.0.203\

IDE is Visual Studio code version 1.105.1

When running in the container instances

Docker file: based on mcr.microsoft.com/dotnet/aspnet:9.0
ARM deployment template, with hopefully the relevant information:

  "resources": [
    {
      "name": "[toLower(parameters('containerGroupName'))]",
      "type": "Microsoft.ContainerInstance/containerGroups",
      "apiVersion": "2021-03-01",
      "location": "[parameters('location')]",
      "properties": {
        "containers": [
          {
            "name": "[toLower(parameters('containerName'))]",
            "properties": {
              "environmentVariables": [
                // business parameters have been removed
                {
                  "name": "DOTNET_EnableDiagnostics",
                  "value": "1"
                },
                {
                  "name": "APPLICATIONINSIGHTS_CONNECTION_STRING",
                  "secureValue": "[parameters('applicationInsightsConnectionString')]"
                }
              ],
              "image": "[parameters('image')]",
              "ports": [
                {
                  "port": "[parameters('port')]",
                  "protocol": "TCP"
                }
              ],
              "resources": {
                "requests": {
                  "cpu": "1.0",
                  "memoryInGB": "1"
                }
              }
            }
          }
        ],
        "osType": "Linux",
        "ipAddress": {
          "type": "Private",
          "ports": [
            {
              "port": "[parameters('port')]",
              "protocol": "TCP"
            }
          ]
        },
        "networkProfile": {
          "id": "[resourceId('Microsoft.Network/networkProfiles', parameters('networkProfileName'))]"
        },
        "restartPolicy": "Always",
        "diagnostics": {
          "logAnalytics": {
            "logType": "ContainerInstanceLogs",
            "workspaceId": "[reference(resourceId('Microsoft.OperationalInsights/workspaces', variables('names').logAnalytics),'2022-10-01').customerId]",
            "workspaceKey": "[listKeys(resourceId('Microsoft.OperationalInsights/workspaces', variables('names').logAnalytics), '2022-10-01').primarySharedKey]",
            "workspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces', variables('names').logAnalytics)]"
          }
        }
      }
    }
  ]

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @harsimar @mattsains-msft @rajkumar-rangaraj @xiang17.