Structured Logs: Collect stdout/stderr

[denrase]

📜 Description

• Capture stdout/stderr and log with structured logs
• Redirect SentrySDK print to original stdout file handle

💡 Motivation and Context

Closes #6411

💚 How did you test it?

📝 Checklist

You have to check all boxes before merging:

• I added tests to verify the changes.
• No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled.
• I updated the docs if needed.
• I updated the wizard if needed.
• Review from the native team if needed.
• No breaking change or entry added to the changelog.
• No breaking change for hybrid SDKs or communicated to hybrid SDKs.

[codecov]

❌ 5 Tests Failed:

Tests completed	Failed	Passed	Skipped
3880	5	3875	23

View the top 3 failed test(s) by shortest run time

SentryTests.SentryStdOutLogIntegrationTests::testInstallWithLogsEnabled

Stack Traces | 0s run time

.../Integrations/StdOutLog/SentryStdOutLogIntegrationTests.swift:55 - XCTAssertTrue failed - Integration should install when logs are enabled

SentryTests.SentryStdOutLogIntegrationTests::testSentryLogsAreIgnored

Stack Traces | 0s run time

.../Integrations/StdOutLog/SentryStdOutLogIntegrationTests.swift:152 - Asynchronous wait failed: Exceeded timeout of 1 seconds, with unfulfilled expectations: "Wait for second normal log capture".

SentryTests.SentryStdOutLogIntegrationTests::testStderrCapture

Stack Traces | 0s run time

.../Integrations/StdOutLog/SentryStdOutLogIntegrationTests.swift:152 - Asynchronous wait failed: Exceeded timeout of 1 seconds, with unfulfilled expectations: "Wait for stderr capture to trigger async dispatch".

SentryTests.SentryStdOutLogIntegrationTests::testStdoutCapture

Stack Traces | 0s run time

.../Integrations/StdOutLog/SentryStdOutLogIntegrationTests.swift:152 - Asynchronous wait failed: Exceeded timeout of 1 seconds, with unfulfilled expectations: "Wait for stdout capture to trigger async dispatch".

SentryTests.SentryStdOutLogIntegrationTests::testUninstall

Stack Traces | 0s run time

.../Integrations/StdOutLog/SentryStdOutLogIntegrationTests.swift:74 - XCTAssertTrue failed - Integration should install first

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[github-actions]

Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	1194.12 ms	1215.43 ms	21.31 ms
Size	23.75 KiB	1012.51 KiB	988.76 KiB

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
b045d0a	1227.48 ms	1252.22 ms	24.75 ms
21efa18	1227.73 ms	1250.04 ms	22.31 ms
50a9ff7	1221.43 ms	1238.07 ms	16.64 ms
0ee162c	1226.90 ms	1261.72 ms	34.83 ms
c63e0fe	1230.58 ms	1253.94 ms	23.35 ms
41c0aa8	1222.75 ms	1262.40 ms	39.65 ms
32584c4	1225.58 ms	1256.02 ms	30.44 ms
3133d0e	1237.86 ms	1262.87 ms	25.01 ms
2de3f92	1207.56 ms	1234.96 ms	27.40 ms
891fd1d	1220.02 ms	1227.60 ms	7.57 ms

App size

Revision	Plain	With Sentry	Diff
b045d0a	23.75 KiB	880.21 KiB	856.46 KiB
21efa18	23.75 KiB	919.70 KiB	895.95 KiB
50a9ff7	23.75 KiB	913.63 KiB	889.89 KiB
0ee162c	23.75 KiB	933.33 KiB	909.58 KiB
c63e0fe	23.74 KiB	874.08 KiB	850.33 KiB
41c0aa8	23.75 KiB	986.80 KiB	963.05 KiB
32584c4	23.75 KiB	920.74 KiB	896.99 KiB
3133d0e	23.74 KiB	976.79 KiB	953.04 KiB
2de3f92	23.75 KiB	919.69 KiB	895.94 KiB
891fd1d	23.75 KiB	919.92 KiB	896.17 KiB

[itaybre]

I worry about having a feedback loop between loop and envelopes generating a never ending loop of mora data being generated, most of which will be garbage.

I see you removed SentrySDKLog.log call in SentryLogger, but I wonder if we should remove all or some Sentry logs from stdout. Have you checked that?

[denrase]

@itaybre Yeah, that worries me also. One "misplaced" stdout call by our SDK, and we run the risk of producing a never ending loop. Like you said, we'd need to change how the internal SDK logging to stdout works, but that would be something we'd need to discuss more.

[denrase]

@itaybre @philipphofmann @noahsmartin

Ok, so the logging we do internally is always prefixed with [Sentry] ., so we could detect those logs and not route it to the logger.

However, i'm still worrying that some where down the code path of using the logger -> hub -> client -> batcher -> transport, we could trigger some non-sentry API that will print to stdout, then we'd still get trapped in a loop.

We could think about offering this as an optional integration to users, but still we'd have to be sure we have the loop issue solved IMHO.

[denrase]

I still don't have a good solution to prevent loops being produced here, maybe I'm missing something obvious.

The gist is we use a Pipe to capture all output to stdout/stderr. When somewhere a call to those is provoked by our SDK, we are stuck in a loop.

SentrySDKLog

This was easier, as we can bypass the pipe when this is enabled and just write to directly to original stdout. So this is solvable.

Users calling print in SentrySDK.loggerBeforeSend

options.beforeSendLog = { log in
  print("foo")
  DispatchQueue.main.async {
      print("bar")
  }
  return log
}

While we can easily detect a loop if it's on the same thread, it becomes more difficult if a user for some reasons decides to do something dispatched to another thread or with a delay that invokes a print call.

OSLog from System Frameworks

When some Apple internal framework is logging an error, probably produced as a side-effect of our SDK, we capture the log and are probably in a loop already. Tried this with our sample app and there were a log of logs produce, without stopping. Debugging is difficult, as we do not have a stack trace for the origin of the OSLog.

So we'd need some kind of circuit breaker for these cases, where we cannot detect if loops are being produced as a side-effect of either us or users using the SDK.

[denrase]

The OSLog outputs from above were produced from failing network requests using Spotlight in debug. The result is the same, if the system is logging to stderr through OSLog because of a failing request our SDK produces while sending logs, we are trapped in a loop.

The data we get through listening to stderr looks like this:

OSLOG-0FECB58A-2CE1-4784-A7E3-8EDE627374F8 7 80 L 12 {t:1763471132.125205,offset:0xd87338}	nw_endpoint_flow_failed_with_error [C1.1.1 ::1.8969 in_progress socket-flow (satisfied (Path is satisfied), interface: lo0)] already failing, returning

OSLOG-0FECB58A-2CE1-4784-A7E3-8EDE627374F8 7 80 L 13 {t:1763471132.128491,tid:0x173a37,subsystem:"com.apple.CFNetwork",category:"Default",offset:0x83a4,imgUUID:"E8F01728-C4AF-3B4F-B340-AF39876BF31D",imgPath:"/Library/Developer/CoreSimulator/Volumes/iOS_23B86/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 26.1.simruntime/Contents/Resources/RuntimeRoot/System/Library/Frameworks/CFNetwork.framework/CFNetwork",lines:3}	Task <BCCACB59-A546-4906-B914-A1B8376C7166>.<1> finished with error [-1001] Error Domain=NSURLErrorDomain Code=-1001 "The request timed out." UserInfo={_kCFStreamErrorCodeKey=-2102, NSUnderlyingError=0x600000c10690 {Error Domain=kCFErrorDomainCFNetwork Code=-1001 "(null)" UserInfo={_kCFStreamErrorCodeKey=-2102, _kCFStreamErrorDomainKey=4}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <BCCACB59-A546-4906-B914-A1B8376C7166>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <BCCACB59-A546-4906-B914-A1B8376C7166>.<1>"
), NSLocalizedDescription=The request timed out., NSErrorFailingURLStringKey=http://localhost:8969/stream, NSErrorFailingURLKey=http://localhost:8969/stream, _kCFStreamErrorDomainKey=4}