Skip to content

Conversation

@LucaDai
Copy link

@LucaDai LucaDai commented Sep 22, 2025

Fix Flakiness of CommentNotificationEventListenerTest caused by Iteration Order Nondeterminism

Context

This test was failing nondeterministically due to iteration order differences in user lists.
It passed in some runs but failed in others, depending on whether "john" or "mary" appeared first.
Error Message:

[ERROR] Failures: 
[ERROR]   CommentNotificationEventListenerTest.testNotificationOnCommentAddedWithRawBody:184 expected:<...ment to [[UserImpl:'[mary'], [UserImpl:'john]']] with body simple...> but was:<...ment to [[UserImpl:'[john'], [UserImpl:'mary]']] with body simple...>
[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

Fix

Replaced strict equality (isEqualTo) with a regex match (matches), allowing the order of "john" and "mary" to vary.

[UserImpl:'(mary|john)'], \\[UserImpl:'(john|mary)']]

This change makes the test verify the correctness of the notification content with nondeterministic ordering.

Verification

  • Re-ran the suite multiple times with randomized test order → 0 failures observed.
  • Confirmed both possible user orderings are accepted by the regex.

Checklist

  • Root cause explained in PR
  • Fix applied deterministically
  • Random-order reruns are green

@LucaDai LucaDai closed this Sep 22, 2025
@LucaDai LucaDai deleted the FixFlakinessCommentNotificationEventListenerTest branch September 30, 2025 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant