Agent Instrumentation Overrides Spans Created With SDK API

[rsteppac]

Expected behavior

In a SpringBoot application that uses either Micrometer API with the Otel-Bridge or the Otel SpringBoot starter to initialize OpenTelemtry and the OpenTelemetry SDK to create custom spans, those spans should be "honored" by agent instrumentation if the application is run with the agent and instrumentation is enabled. Logs produced by code within a custom span should be exported to AppInsights.

Actual behavior

The custom spans are created inside a Spring Scheduled function. If instrumentation of Spring Scheduled functions is enabled in the agent configuration, then the custom spans get overridden by instrumentation and the standard approach of one span per scheduled execution is applied instead. All logs get shipped as expected.
If instrumentation of Spring Scheduled functions is disabled in the agent configuration then the local log file contains the custom spans, but the logs do not get exported to AppInsights.

The behavior is the same for both Micrometer with Otel-Bridge and direct use of the Otel SDK.

To Reproduce

See here for the scenario of starting a custom span within a Kotlin flow, started by a scheduled function.

This is how we create and close spans.

This is how we transfer the context from the flow to co-routines we spin off the flow.

System information

Please provide the following information:

• SDK version: Otel SDK v1.49 (SpringBoot v3.5.4 managed dependency) or v1.52 (Otel BOM v2.18.1)
• OS type and version: Linux, Ubuntu & Centos
• Application Server type and version (if applicable): none
• Using spring-boot? Yes. v.3.5.4
• Additional relevant libraries (with version, if applicable):
• Kotlin v2.2.0
• Kotlin coroutines v1.10.2

Logs

The agent logs show no obvious errors. The complete agent logs have been submitted in the context of a Microsoft support ticket.