cli-test(fix): wait for cli run start trace instead of activity output

[zimeg]

Summary

This PR uses SLACK_TRACE_PLATFORM_RUN_START variable as a signal of an active running app instead of the exact output:

Connected, awaiting events

Fixes an issue where bolt apps were not detected as running.

Requirements

• I've read and understood the Contributing Guidelines and have done my best effort to follow them.
• I've read and agree to the Code of Conduct.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.77%. Comparing base (06a3dcf) to head (6f90edf).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2355   +/-   ##
=======================================
  Coverage   92.77%   92.77%           
=======================================
  Files          39       39           
  Lines       10693    10693           
  Branches      692      692           
=======================================
  Hits         9920     9920           
  Misses        761      761           
  Partials       12       12           

Flag	Coverage Δ
cli-hooks	95.23% <ø> (ø)
cli-test	94.80% <100.00%> (ø)
oauth	77.39% <ø> (ø)
socket-mode	61.87% <ø> (ø)
web-api	97.99% <ø> (ø)
webhook	96.66% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[zimeg]

👾 Rambling thoughts on this change but I am open to follow up or updates otherwise!

[zimeg]

🗣️ note: This is preferred over SLACK_TRACE_PLATFORM_RUN_READY since the "ready" variable appears once activity logs stream for CLI managed connections. It does not appear for bolt apps at this time.

👁️‍🗨️ thought: I'm unsure if we should add that trace to the CLI outputs in a follow up PR? For now I think using a trace is still an improvement but I'm somewhat worried about tests skipping to the next step before run is settled for some apps... 🦕

[mwbrooks]

I'm unsure if we should add that trace to the CLI outputs in a follow up PR?

Just checking if I understand correctly - are you thinking that the CLI would print the SLACK_TRACE_PLATFORM_RUN_READY trace for sdk-managed socket connections?

[ewanek1]

LGTM! Like the more reliable detection of running Slack apps 👏 👏

[mwbrooks]

✅ Very healthy fix for Bolt tests! Sorry that this review took 3 weeks! :shame:

[mwbrooks]

I'm unsure if we should add that trace to the CLI outputs in a follow up PR?

Just checking if I understand correctly - are you thinking that the CLI would print the SLACK_TRACE_PLATFORM_RUN_READY trace for sdk-managed socket connections?

[zimeg]

@ewanek1 @mwbrooks Thanks both for the kind reviews. And never worries on a rushed review for this! 👾

Just checking if I understand correctly - are you thinking that the CLI would print the SLACK_TRACE_PLATFORM_RUN_READY trace for sdk-managed socket connections?

Exactly this! I'm not finding such printed at this time, but I'm also not sure if that'd be possible to know for SDK managed connections...

Somewhat following that, I'm hesitant to release this change since it might disrupt our E2E tests if workflows are started after "PLATFORM_RUN_START" and before "PLATFORM_RUN_READY" but we can perhaps wait for the second trace in an explicit step?