Skip to content

Conversation

@XiaoyangCai360
Copy link

Purpose

This PR fixes the flaky test:

  • org.jbpm.process.workitem.rest.TransformerJSONandXMLTest.testJSONTransformer

This test fails nondeterministically due to the non-deterministic field ordering in JSON serialization performed by Jackson’s ObjectMapper.


Why the test fails (root cause)

The test serializes a Person object into JSON using the expression:

new com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(employee)

and asserts string equality against an expected JSON string:

{"name":"john","age":34}

However, Jackson does not guarantee deterministic property ordering when serializing POJOs unless explicitly configured.
Depending on JVM and internal HashMap ordering, the serialized output can vary between:

{"name":"john","age":34}

and

{"age":34,"name":"john"}

Although these JSON objects are semantically equivalent, the test performs a raw string comparison, making it order-sensitive and thus flaky under randomized iteration (e.g., when using [NonDex](https://github.com/TestingResearchIllinois/NonDex)).


How to reproduce the test failure

Running the test multiple times with NonDex randomization:

mvn -pl jbpm-workitems/jbpm-workitems-rest -am \
  edu.illinois:nondex-maven-plugin:2.1.7:nondex \
  -Dtest=org.jbpm.process.workitem.rest.TransformerJSONandXMLTest#testJSONTransformer \
  -DnondexRuns=10 \
  -DfailIfNoTests=false

Example flaky output:

expected:<{"[name":"john","age":34]}> but was:<{"[age":34,"name":"john"]}>

Description of the fix

The fix replaces raw string comparison with semantic JSON comparison using Jackson’s ObjectMapper:

ObjectMapper mapper = new ObjectMapper();
Map<String, Object> expectedMap = mapper.readValue(expectedJson, new TypeReference<Map<String, Object>>() {});
Map<String, Object> resultMap = mapper.readValue((String) result, new TypeReference<Map<String, Object>>() {});
assertEquals(expectedMap, resultMap);

This ensures that the test compares the contents of the JSON objects rather than their textual field order.
Both maps are deserialized into standard Java maps whose equality check is order-insensitive.


Verification of the fix

  • All NonDex runs (nondexRuns=10) passed consistently.
  • Regular test runs passed deterministically.
  • No production code or logic was modified.

Key points

  • Only modifies test logic in TransformerJSONandXMLTest.
  • Preserves the test’s intent and expected behavior.
  • Eliminates nondeterminism caused by JSON field ordering.
  • No changes to runtime or production behavior.
  • Verified stable under NonDex randomized execution.

@XiaoyangCai360 XiaoyangCai360 changed the title fix(jbpm-workitems-rest):fix flaky test testJSONTransformer fix(jbpm-workitems-rest): make test TransformerJSONandXMLTest.testJSONTransformer deterministic Oct 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant