-
-
Notifications
You must be signed in to change notification settings - Fork 73
Description
I'm using stream_data to generate a rather large data structure representing an SQL query. I was having problems shrinking the generated examples - it seemed like only minor details like integer values were being shrunk while the query structure as a whole remained the same. I managed to reduce it to the following property (should work after pasting into stream_data tests):
property "shrinks towards earlier generators" do
check all size <- positive_integer(),
seed <- {integer(), integer(), integer()} do
{:error, result} =
one_of([:foo, {:bar, integer()}])
|> check_all(
[max_shrinking_steps: 100, initial_size: size, initial_seed: seed],
fn example -> {:error, example} end
)
assert result.nodes_visited < 100
assert :foo = result.shrunk_failure
end
endError:
1) property one_of/1 shrinks towards earlier generators (StreamDataTest)
test/stream_data_test.exs:199
Failed with generated values (after 5 successful runs):
* Clause: size <- positive_integer()
Generated: 4
* Clause: seed <- {integer(), integer(), integer()}
Generated: {3, 0, 0}
match (=) failed
code: assert :foo = result.shrunk_failure()
right: {:bar, 0}
Curiously - it does seem to be dependent on the size, even though integer/1 which is used by one_of seems to operate independently of size. If I just set size <- constant(1), then the property passes.
I did some tracing and it does seem like for small initial_size the initial failure found is {:bar, _} and after some shrinking it gets reduced to :foo but that doesn't seem to happen for a larger initial_size. It doesn't seem to happen when the generator tested is just one_of([:foo, :bar]).
This also affects frequency/1 as it has almost the same implementation with the minor difference of how the option is chosen from the list and at the very least tree/2, which is how I originally found it. I assume it also affects anything else that uses frequency/1.
I'd be happy to make some more attempts at fixing this, but I haven't yet understood the code dealing with shrinking fully, so I wanted to make sure I'm on the right track here and also ask for any advice you might have.