Reduce CI Vis initialization that happens when not required #7811

andrewlock · 2025-11-13T17:59:18Z

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

github-actions · 2025-11-13T18:00:12Z

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

1 occurrences of :

-                "string_value": "unknown_service:dotnet"
+                "string_value": "Samples.OpenTelemetrySdk"

…app startup when not in CI Visibility

datadog-official · 2025-11-14T08:17:39Z

⚠️ Tests

⚠️ Warnings

🧪 7573 Tests failed

AzureFunctionsTests+InProcessRuntimeV4.SubmitsTraces from Datadog.Trace.ClrProfiler.IntegrationTests (Datadog)
Expected collection to contain at least 21 item(s) because we want to ensure that we don't timeout while waiting for spans from the mock tracer agent, but found 0: {empty}.
AzureFunctionsTests+IsolatedRuntimeV4AspNetCore.SubmitsTraces from Datadog.Trace.ClrProfiler.IntegrationTests (Datadog)
Expected collection to contain at least 26 item(s) because we want to ensure that we don't timeout while waiting for spans from the mock tracer agent, but found 0: {empty}.
AzureFunctionsTests+IsolatedRuntimeV4AspNetCore.SubmitsTraces from Datadog.Trace.ClrProfiler.IntegrationTests (Datadog)
Expected collection to contain at least 26 item(s) because we want to ensure that we don't timeout while waiting for spans from the mock tracer agent, but found 0: {empty}.
View all

ℹ️ Info

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 3e1d0d0 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

dd-trace-dotnet-ci-bot · 2025-11-14T08:40:01Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (7811) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	75.17 ± (75.01 - 75.89) ms	74.24 ± (73.98 - 74.74) ms	-1.2%	✅
.NET Framework 4.8 - Bailout
duration	79.36 ± (79.15 - 79.86) ms	80.44 ± (79.87 - 80.72) ms	+1.4%	✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1107.51 ± (1109.26 - 1119.18) ms	2964.49 ± (2964.49 - 2964.49) ms	+167.7%	✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	22.89 ± (22.81 - 22.96) ms	22.79 ± (22.72 - 22.86) ms	-0.4%	✅
process.time_to_main_ms	86.14 ± (85.84 - 86.43) ms	86.69 ± (86.35 - 87.03) ms	+0.6%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.92 ± (10.91 - 10.92) MB	10.92 ± (10.91 - 10.92) MB	-0.0%	✅
runtime.dotnet.threads.count	12 ± (12 - 12)	12 ± (12 - 12)	+0.0%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	22.69 ± (22.63 - 22.76) ms	22.83 ± (22.77 - 22.90) ms	+0.6%	✅⬆️
process.time_to_main_ms	86.74 ± (86.42 - 87.07) ms	87.76 ± (87.42 - 88.11) ms	+1.2%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.95 ± (10.95 - 10.96) MB	10.97 ± (10.96 - 10.97) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	13 ± (13 - 13)	13 ± (13 - 13)	+0.0%	✅
.NET 6 - Baseline
process.internal_duration_ms	21.77 ± (21.70 - 21.84) ms	21.64 ± (21.58 - 21.71) ms	-0.6%	✅
process.time_to_main_ms	75.66 ± (75.35 - 75.98) ms	75.95 ± (75.60 - 76.30) ms	+0.4%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.60 ± (10.59 - 10.60) MB	10.63 ± (10.63 - 10.63) MB	+0.3%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 6 - Bailout
process.internal_duration_ms	21.82 ± (21.76 - 21.89) ms	21.64 ± (21.58 - 21.71) ms	-0.8%	✅
process.time_to_main_ms	77.31 ± (76.95 - 77.67) ms	77.30 ± (76.93 - 77.66) ms	-0.0%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.65 ± (10.64 - 10.65) MB	10.66 ± (10.66 - 10.67) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 8 - Baseline
process.internal_duration_ms	19.88 ± (19.81 - 19.95) ms	19.98 ± (19.91 - 20.05) ms	+0.5%	✅⬆️
process.time_to_main_ms	75.03 ± (74.64 - 75.43) ms	75.12 ± (74.79 - 75.46) ms	+0.1%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.66 ± (7.65 - 7.67) MB	7.67 ± (7.66 - 7.67) MB	+0.0%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 8 - Bailout
process.internal_duration_ms	19.92 ± (19.87 - 19.97) ms	19.70 ± (19.64 - 19.75) ms	-1.1%	✅
process.time_to_main_ms	76.03 ± (75.76 - 76.29) ms	75.55 ± (75.25 - 75.85) ms	-0.6%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.71 ± (7.70 - 7.72) MB	7.71 ± (7.70 - 7.71) MB	-0.0%	✅
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	192.31 ± (192.25 - 192.99) ms	191.99 ± (191.94 - 192.65) ms	-0.2%	✅
.NET Framework 4.8 - Bailout
duration	195.35 ± (195.10 - 195.84) ms	196.76 ± (196.46 - 197.23) ms	+0.7%	✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1162.82 ± (1161.78 - 1170.04) ms	1057.57 ± (1057.57 - 1057.57) ms	-9.1%	✅
.NET Core 3.1 - Baseline
process.internal_duration_ms	187.88 ± (187.46 - 188.29) ms	188.36 ± (188.00 - 188.71) ms	+0.3%	✅⬆️
process.time_to_main_ms	80.29 ± (80.12 - 80.46) ms	80.57 ± (80.33 - 80.82) ms	+0.3%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.06 ± (16.03 - 16.09) MB	16.12 ± (16.09 - 16.15) MB	+0.4%	✅⬆️
runtime.dotnet.threads.count	20 ± (20 - 20)	20 ± (19 - 20)	-0.3%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	187.32 ± (187.01 - 187.63) ms	189.33 ± (189.00 - 189.65) ms	+1.1%	✅⬆️
process.time_to_main_ms	81.70 ± (81.55 - 81.86) ms	82.20 ± (82.06 - 82.34) ms	+0.6%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.11 ± (16.04 - 16.19) MB	16.15 ± (16.13 - 16.18) MB	+0.3%	✅⬆️
runtime.dotnet.threads.count	21 ± (20 - 21)	21 ± (20 - 21)	-0.2%	✅
.NET 6 - Baseline
process.internal_duration_ms	191.96 ± (191.59 - 192.34) ms	192.91 ± (192.48 - 193.33) ms	+0.5%	✅⬆️
process.time_to_main_ms	69.88 ± (69.67 - 70.10) ms	69.90 ± (69.72 - 70.08) ms	+0.0%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.31 ± (16.24 - 16.38) MB	16.24 ± (16.15 - 16.33) MB	-0.5%	✅
runtime.dotnet.threads.count	19 ± (19 - 19)	19 ± (19 - 19)	+0.5%	✅⬆️
.NET 6 - Bailout
process.internal_duration_ms	191.54 ± (191.16 - 191.92) ms	191.31 ± (191.03 - 191.58) ms	-0.1%	✅
process.time_to_main_ms	70.76 ± (70.65 - 70.87) ms	70.70 ± (70.61 - 70.80) ms	-0.1%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.13 ± (15.98 - 16.27) MB	16.33 ± (16.22 - 16.44) MB	+1.3%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 20)	20 ± (20 - 20)	+2.0%	✅⬆️
.NET 8 - Baseline
process.internal_duration_ms	189.18 ± (188.91 - 189.46) ms	189.99 ± (189.66 - 190.32) ms	+0.4%	✅⬆️
process.time_to_main_ms	68.83 ± (68.66 - 68.99) ms	69.05 ± (68.88 - 69.22) ms	+0.3%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.72 ± (11.66 - 11.78) MB	11.71 ± (11.69 - 11.74) MB	-0.1%	✅
runtime.dotnet.threads.count	18 ± (17 - 18)	18 ± (18 - 18)	+2.7%	✅⬆️
.NET 8 - Bailout
process.internal_duration_ms	188.25 ± (187.97 - 188.52) ms	189.26 ± (189.01 - 189.50) ms	+0.5%	✅⬆️
process.time_to_main_ms	69.71 ± (69.61 - 69.80) ms	70.05 ± (69.97 - 70.12) ms	+0.5%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.70 ± (11.61 - 11.80) MB	11.71 ± (11.65 - 11.77) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	19 ± (18 - 19)	19 ± (19 - 19)	+2.3%	✅⬆️

Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts

FakeDbCommand (.NET Framework 4.8)

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7811) - mean (74ms)  : 68, 80
    master - mean (75ms)  : 69, 82

    section Bailout
    This PR (7811) - mean (80ms)  : 74, 87
    master - mean (80ms)  : 74, 85

    section CallTarget+Inlining+NGEN
    This PR (7811) - mean (2,964ms)  : 2964, 2964
    master - mean (1,114ms)  : 1043, 1185

FakeDbCommand (.NET Core 3.1)

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7811) - mean (117ms)  : 111, 123
    master - mean (116ms)  : 110, 123

    section Bailout
    This PR (7811) - mean (118ms)  : 111, 125
    master - mean (117ms)  : 112, 122

    section CallTarget+Inlining+NGEN
    This PR (7811) - mean (3,605ms)  : 3605, 3605
    master - mean (796ms)  : 759, 833

FakeDbCommand (.NET 6)

gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7811) - mean (104ms)  : 98, 110
    master - mean (104ms)  : 97, 112

    section Bailout
    This PR (7811) - mean (106ms)  : 100, 112
    master - mean (106ms)  : 98, 114

    section CallTarget+Inlining+NGEN
    This PR (7811) - mean (5,136ms)  : 5136, 5136
    master - mean (760ms)  : 717, 804

FakeDbCommand (.NET 8)

gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7811) - mean (103ms)  : 98, 109
    master - mean (103ms)  : 96, 110

    section Bailout
    This PR (7811) - mean (103ms)  : 97, 109
    master - mean (104ms)  : 100, 109

    section CallTarget+Inlining+NGEN
    This PR (7811) - mean (874ms)  : 874, 874
    master - mean (716ms)  : 686, 746

HttpMessageHandler (.NET Framework 4.8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7811) - mean (192ms)  : 189, 196
    master - mean (193ms)  : 189, 196

    section Bailout
    This PR (7811) - mean (197ms)  : 193, 201
    master - mean (195ms)  : 192, 199

    section CallTarget+Inlining+NGEN
    This PR (7811) - mean (1,058ms)  : 1058, 1058
    master - mean (1,166ms)  : 1106, 1226

HttpMessageHandler (.NET Core 3.1)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7811) - mean (277ms)  : 272, 282
    master - mean (277ms)  : 272, 282

    section Bailout
    This PR (7811) - mean (279ms)  : 275, 284
    master - mean (277ms)  : 273, 281

    section CallTarget+Inlining+NGEN
    This PR (7811) - mean (3,495ms)  : 3495, 3495
    master - mean (949ms)  : 899, 999

HttpMessageHandler (.NET 6)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7811) - mean (271ms)  : 265, 277
    master - mean (271ms)  : 265, 276

    section Bailout
    This PR (7811) - mean (270ms)  : 266, 274
    master - mean (270ms)  : 266, 275

    section CallTarget+Inlining+NGEN
    This PR (7811) - mean (4,048ms)  : 4048, 4048
    master - mean (926ms)  : 883, 970

HttpMessageHandler (.NET 8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7811) - mean (268ms)  : 264, 273
    master - mean (268ms)  : 264, 272

    section Bailout
    This PR (7811) - mean (269ms)  : 266, 272
    master - mean (267ms)  : 264, 270

    section CallTarget+Inlining+NGEN
    This PR (7811) - mean (5,175ms)  : 5175, 5175
    master - mean (853ms)  : 836, 871

andrewlock · 2025-11-18T10:11:34Z

Superseded by #7821

andrewlock changed the base branch from master to andrew/settings/6a-stop-using-tracer-instance November 13, 2025 17:59

andrewlock force-pushed the andrew/settings/switch-up-ci-vis-initialization branch from 09d63e4 to f05a867 Compare November 13, 2025 18:03

Try reduce amount of TestOptimization initialization that happens on …

3e1d0d0

…app startup when not in CI Visibility

andrewlock force-pushed the andrew/settings/switch-up-ci-vis-initialization branch from f05a867 to 3e1d0d0 Compare November 14, 2025 07:41

andrewlock force-pushed the andrew/settings/6a-stop-using-tracer-instance branch from 71c8119 to aad788e Compare November 14, 2025 07:41

andrewlock closed this Nov 18, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Reduce CI Vis initialization that happens when not required #7811

Reduce CI Vis initialization that happens when not required #7811

andrewlock commented Nov 13, 2025

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

datadog-official bot commented Nov 14, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading

Uh oh!

dd-trace-dotnet-ci-bot bot commented Nov 14, 2025

FakeDbCommand

HttpMessageHandler

Uh oh!

andrewlock commented Nov 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Reduce CI Vis initialization that happens when not required #7811

Reduce CI Vis initialization that happens when not required #7811

Conversation

andrewlock commented Nov 13, 2025

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

Uh oh!

github-actions bot commented Nov 13, 2025

Snapshots difference summary

Uh oh!

datadog-official bot commented Nov 14, 2025 • edited by datadog-datadog-prod-us1 bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

dd-trace-dotnet-ci-bot bot commented Nov 14, 2025

Execution-Time Benchmarks Report ⏱️

FakeDbCommand

HttpMessageHandler

Uh oh!

andrewlock commented Nov 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

datadog-official bot commented Nov 14, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading