[NA] [BE] Add extension to add workspace and SDK version to Otel metrics #3812

thiagohora · 2025-10-27T08:58:06Z

Details

This PR adds an Otel extension to the Otel agent so that it adds the workspace header to our HTTP-related metrics. This is required to avoid losing the filter granularity we have today.

Change checklist

User facing
Documentation update

Issues

Resolves #
NA

Testing

NA

Documentation

NA

- WebhookSubscriber: Fixed 5 async/non-blocking issues * Replaced Mono.fromRunnable with Mono.fromCallable for proper exception handling * Wrapped blocking deserialization with Mono.fromCallable + subscribeOn * Moved attributes building inside Mono chain * Pre-built metrics counters to avoid repeated instantiation * Added timeout and validated HTTP client usage - BaseRedisSubscriber: Added backpressure monitoring * Added pre-built backpressure drop counter * Logs when backpressure causes events to be dropped * Helps identify system overload conditions - WebhookHttpClient: Major async/resource management improvements * Async JSON serialization on boundedElastic scheduler * Try-with-resources for automatic response cleanup * Optional for null-safe string handling * Retry attempt logging for observability * Returns response body or 'ok' instead of exposing Response object * No duplicate error logging - AlertService: Updated to work with new Mono<String> return type * Handles response body directly * Defaults to 200 status code on success All changes follow reactive best practices and enterprise-grade patterns.

…move unnecessary wrappers, consolidate schedulers, fix resource management, and clarify logging messages

… async callback

…ound JSON work, remove misleading comment, simplify validation with Mono.fromRunnable()

…th for future virtual threads

…le for virtual threads

…hreads support (enableVirtualThreads flag) for future efficiency

…ture with proper error handling

Copilot

Pull Request Overview

This PR adds a custom OpenTelemetry extension to the Otel agent to preserve workspace and SDK version information in HTTP-related metrics. The extension is built as a separate Maven module and integrated into the backend's Docker image and startup script to maintain metric filtering granularity.

Key changes:

Custom Otel extension module added to enhance HTTP metrics with workspace context
Updated OpenTelemetry version from 2.20.0 to 2.21.0
Modified Docker build and entrypoint to conditionally load the extension

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
apps/opik-backend/pom.xml	Bumped OpenTelemetry version to 2.21.0
apps/opik-backend/entrypoint.sh	Added conditional logic to attach telemetry extension to Otel agent
apps/opik-backend/Dockerfile	Integrated telemetry extension build and deployment into Docker image

Copilot · 2025-10-27T08:59:09Z

apps/opik-backend/Dockerfile

 COPY --from=build /opt/opik-backend/target/openapi.yaml redoc/
 COPY --from=build /opt/opik-backend/target/*.jar ./
+# Copy telemetry extension JAR (built as part of the build process)
+COPY --from=build /opt/opik-backend/opik-telemetry-extension/target/opik-telemetry-extension-*.jar ./opik-telemetry-extension.jar


The wildcard pattern opik-telemetry-extension-*.jar may match multiple JAR files if version suffixes or classifiers are present. This could result in copy failures or unpredictable behavior. Consider using a more specific pattern or renaming the artifact during the build phase to ensure exactly one file is copied.

I think copilot has a point here

Probably not a problem, we're the ones generating the .jar, it's fixed as 1.0.0-SNAPSHOT

github-actions · 2025-10-27T08:59:19Z

SDK E2E Tests Results

0 tests 0 ✅ 0s ⏱️
0 suites 0 💤
0 files 0 ❌

Results for commit d40a987.

♻️ This comment has been updated with latest results.

github-actions · 2025-10-27T09:28:53Z

Backend Tests Results

5 205 tests 5 198 ✅ 49m 27s ⏱️
258 suites 7 💤
258 files 0 ❌

Results for commit a30439d.

♻️ This comment has been updated with latest results.

idoberko2

Looks good. I Had some comments I believe we should attend before merging

idoberko2 · 2025-10-27T10:58:57Z

...telemetry-extension/src/main/java/com/comet/opik/telemetry/OpikMetricProviderCustomizer.java

+
+    @Override
+    public void customize(AutoConfigurationCustomizer customizer) {
+        System.out.println("[OpikTelemetryExtension] Applying Opik-specific OpenTelemetry metric provider customizations");


Suggested change

System.out.println("[OpikTelemetryExtension] Applying Opik-specific OpenTelemetry metric provider customizations");

log.info("[OpikTelemetryExtension] Applying Opik-specific OpenTelemetry metric provider customizations");

replied here #3812 (comment)

idoberko2 · 2025-10-27T11:01:10Z

...metry-extension/src/test/java/com/comet/opik/telemetry/OpikMetricProviderCustomizerTest.java

+        assertNotNull(result);
+        assertEquals(meterProviderBuilder, result);


Use assertJ assertions instead of JUnit

idoberko2 · 2025-10-27T11:01:24Z

...metry-extension/src/test/java/com/comet/opik/telemetry/OpikMetricProviderCustomizerTest.java

+        int order = customizer.order();
+
+        // Then
+        assertEquals(100, order);


Same here, assertJ

idoberko2 · 2025-10-27T11:03:14Z

apps/opik-backend/Dockerfile

 COPY --from=build /opt/opik-backend/target/openapi.yaml redoc/
 COPY --from=build /opt/opik-backend/target/*.jar ./
+# Copy telemetry extension JAR (built as part of the build process)
+COPY --from=build /opt/opik-backend/opik-telemetry-extension/target/opik-telemetry-extension-*.jar ./opik-telemetry-extension.jar


I think copilot has a point here

...telemetry-extension/src/main/java/com/comet/opik/telemetry/OpikMetricProviderCustomizer.java

ldaugusto · 2025-10-27T11:06:51Z

...telemetry-extension/src/main/java/com/comet/opik/telemetry/OpikMetricProviderCustomizer.java

+
+        // Create a single view that includes workspace context attributes
+        // This view will be applied to all instruments without changing their names
+        View workspaceView = View.builder()


sorry i cant understand how this is injecting workspaceId and the sdk version inside the metrics. What am I missing?

The header is added in the propagation context via OTEL_INSTRUMENTATION_HTTP_SERVER_CAPTURE_REQUEST_HEADERS. However, it removes all attributes not specified by the instrumentation by default. Then I had to create this extension following this: https://github.com/open-telemetry/opentelemetry-java/tree/main/sdk-extensions/incubator#file-configuration

…_and_sdk_version_to_otel_metrics

ldaugusto

LGTM

thiagohora added 12 commits October 18, 2025 18:25

Revision 2: Address all 8 GitHub PR comments - fix reactive chain, re…

9ae722e

…move unnecessary wrappers, consolidate schedulers, fix resource management, and clarify logging messages

Revision 3: Use try-with-resources for cleaner Resource management in…

7a7ae72

… async callback

Revision 4: Simplify try-with-resources syntax using Java 9+ style

8d8ea37

Revision 5: Address 4 new Copilot comments - use parallel() for CPU-b…

7dafe81

…ound JSON work, remove misleading comment, simplify validation with Mono.fromRunnable()

Revision 6: Keep boundedElastic() for JSON work - better migration pa…

24f5caa

…th for future virtual threads

Revision 7: Use parallel() for short CPU-bound JSON work - not suitab…

2dd47e0

…le for virtual threads

Revision 8: Use boundedElastic() consistently - project has virtual t…

2602e71

…hreads support (enableVirtualThreads flag) for future efficiency

[NA] [BE] Fix webhook async handling - restore original working struc…

6061ed6

…ture with proper error handling

Merge branch 'main' of https://github.com/comet-ml/opik

11a88d0

Merge branch 'main' of https://github.com/comet-ml/opik

4063c1b

[NA] [BE] Add extension to add workspace and sdk version to otel metrics

47b421c

github-actions bot assigned thiagohora Oct 27, 2025

thiagohora marked this pull request as ready for review October 27, 2025 08:58

thiagohora requested a review from a team as a code owner October 27, 2025 08:58

Copilot AI review requested due to automatic review settings October 27, 2025 08:58

Copilot AI reviewed Oct 27, 2025

View reviewed changes

Fix

213328f

thiagohora force-pushed the thiagohora/NA_add_extension_to_add_workspace_and_sdk_version_to_otel_metrics branch from d40a987 to 213328f Compare October 27, 2025 08:59

BorisTkachenko previously approved these changes Oct 27, 2025

View reviewed changes

idoberko2 reviewed Oct 27, 2025

View reviewed changes

ldaugusto reviewed Oct 27, 2025

View reviewed changes

Merge branch 'main' into thiagohora/NA_add_extension_to_add_workspace…

a30439d

…_and_sdk_version_to_otel_metrics

thiagohora dismissed BorisTkachenko’s stale review via a30439d October 27, 2025 12:50

thiagohora requested a review from ldaugusto October 27, 2025 14:35

ldaugusto approved these changes Oct 27, 2025

View reviewed changes

thiagohora merged commit 37dd25e into main Oct 27, 2025
14 checks passed

thiagohora deleted the thiagohora/NA_add_extension_to_add_workspace_and_sdk_version_to_otel_metrics branch October 27, 2025 16:41

	System.out.println("[OpikTelemetryExtension] Applying Opik-specific OpenTelemetry metric provider customizations");
	log.info("[OpikTelemetryExtension] Applying Opik-specific OpenTelemetry metric provider customizations");

		assertNotNull(result);
		assertEquals(meterProviderBuilder, result);

[NA] [BE] Add extension to add workspace and SDK version to Otel metrics #3812

[NA] [BE] Add extension to add workspace and SDK version to Otel metrics #3812

Uh oh!

Conversation

thiagohora commented Oct 27, 2025

Details

Change checklist

Issues

Testing

Documentation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SDK E2E Tests Results

Uh oh!

github-actions bot commented Oct 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backend Tests Results

Uh oh!

idoberko2 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ldaugusto left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions bot commented Oct 27, 2025 •

edited

Loading

github-actions bot commented Oct 27, 2025 •

edited

Loading