Short test covering most of Riak for development #606
Description
This is an exploration along the lines of Kelly's proposal: https://gist.github.com/kellymclaughlin/f28b75ac295b3d397ef0
The main goal is to have a single test running in a reasonable amount of time that covers as much of Riak's functionality as possible in that time. Developers can use this before checking in code, since running all existing tests in their current form takes more than a day an destroys your laptop in the process. In order to do this, tests should be refactored to be more modular and reusable, so they can be run in small or large setups. It would be great if they can perform steps in a way that allows different functionality to be mixed. For example, mixing preparing data for 2i test in a bucket with preparing data for a CRDT test, then interleave verification for each. Run the same steps before/after adding/removing nodes, destroying partitions, etc.
The idea is to give up on side effect free runs and favor verifying functionality in the context of other functionality running. This is usually not what you want with unit tests, but the crux of integration testing.
A sketch of possible phases of this test:
- The test creates the smallest cluster possible, with as few vnodes as possible and configured to perform most operations much faster than a real cluster.
- It runs as many functionality verification steps as can run without a setup that makes them incompatible to test with other functionality.
- Perform cluster operations/events such as adding/removing nodes, losing partitions, resizing the ring, etc
- Run verification steps for as much functionality as possible
This is highly experimental and I expect lots of challenges. But the more we can cram into a short running tests, even if not perfect, the more useful this would be day to day.