[docs] add language about flaky tests / issues for them (#6837)

Flaky tests happen and when they are encountered in CI, we should have tickets for them. This adds some language that to effect to the contrib and dev docs.

Published updated sections
* https://www.pantsbuild.org/staging/nhoward-test-site/howto_contribute.html#iterating
* https://www.pantsbuild.org/staging/nhoward-test-site/howto_develop.html#dev_run_all_tests

Author: baroquebobcat

src/docs/howto_contribute.md

@@ -192,6 +192,16 @@ If at any point you need to make changes that will fundamentally overhaul a revi
 consider temporarily removing the `reviewable` label in order to let reviewers know
 to hold off until the code is ready.
 
+#### CI Failing?
+
+If your pull request fails in travis, you can look at the shard logs to see the
+failure cause. From that log, you can figure out what tests you need to run to reproduce the failure
+locally. If you cannot reproduce the failure locally and it looks unrelated to your change, please
+open an issue for it with the label
+(`flaky-test`)[https://github.com/pantsbuild/pants/labels/flaky-test]. You can also ping slack to
+ask for someone to restart the failing shard.
+(<a pantsref="dev_run_all_tests">More on checking CI test runs here</a>)
+
 ### Committing a Change
 
 At this point you've made a change, had it reviewed (and received one or more Ship Its!) and

src/docs/howto_develop.md

@@ -154,7 +154,8 @@ a pull request, and allow travis to run them. You can reproduce failures by runn
     $ ./build-support/bin/ci.sh
 
 with whatever relevant flags reproduce the failure (`./build-support/bin/ci.sh -h` will list the
-available flags).
+available flags). The relevant flags are in the log for the shard on a line that looks something
+like `Executing ./build-support/bin/ci.sh "-c3 -i 0/6" ...`.
 
 To run just Pants' *unit* tests (skipping the can-be-slow integration tests), filter out
 the python tests tagged with 'integration':
@@ -181,6 +182,13 @@ not your fork. If you are posting a review request, put the pull request number
 field. Then, when you close the request, you can navigate from the bug number to easily close
 the pull request.
 
+If your CI-build failed in Travis-CI, and the failure looks like it's not due to
+your change, please open an issue with the part of the CI log containing the test failure and label
+the issue with `flaky-test`. If an issue already exists, add a comment to it noting that you
+encountered it too. After you've done that, you can ask in slack for someone to restart the shard.
+That will cause the shard to re-run its tests.
+
+
 Debugging
 ---------