Skip to content

Commit 942ad48

Browse files
authored
[CI] Fix flaky Ray Datasets integration test. (#952)
Datasets partition ordering is now nondeterministic, so any tests that assert equality with some cross-partition ordering will need to sort or freeze the data before comparison. This PR applies the fix from #917 (freezing) to the tensor test.
1 parent 2a7805c commit 942ad48

File tree

1 file changed

+8
-7
lines changed

1 file changed

+8
-7
lines changed

tests/ray/test_datasets.py

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -187,13 +187,14 @@ def test_from_ray_dataset_tensor(n_partitions: int):
187187
ds = ds.map(lambda i: {"int": i, "np": np.ones((3, 3))}).repartition(n_partitions)
188188

189189
df = daft.from_ray_dataset(ds)
190-
np.testing.assert_equal(
191-
df.to_pydict(),
192-
{
193-
"int": list(range(8)),
194-
"np": [np.ones((3, 3)) for i in range(8)],
195-
},
196-
)
190+
out = df.to_pydict()
191+
out["np"] = [arr.tolist() for arr in out["np"]]
192+
expected = {
193+
"int": list(range(8)),
194+
"np": [np.ones((3, 3)) for i in range(8)],
195+
}
196+
expected["np"] = [arr.tolist() for arr in expected["np"]]
197+
assert freeze(out) == freeze(expected)
197198

198199

199200
@pytest.mark.skipif(get_context().runner_config.name != "ray", reason="Needs to run on Ray runner")

0 commit comments

Comments
 (0)