diff options
| author | Tim Pepper <tpepper@vmware.com> | 2018-09-19 17:08:46 -0700 |
|---|---|---|
| committer | Tim Pepper <tpepper@vmware.com> | 2018-09-19 17:12:32 -0700 |
| commit | 3a13fb1487badd1c746978024fec964442d6e3b3 (patch) | |
| tree | 47effa00464db04d9aa38349d082ef2e13ffaa79 | |
| parent | 9b0e0b53745d3786407ec3ce4cb65cbaac06f5e1 (diff) | |
devel guide: note where to track flaky tests
We have a good document on investigating flaky tests, but it can
be hard to correlate personal experience (my PR wont pass presubmits!?!)
with broader CI health (did AWS go down?). For that portion of
triage it's useful to look at some aggregated statistics and we have
two nice pages for that.
I'm always forgetting these two links though and while some kind
folks have reminded me repeatedly, I take that as a sign it's time
to put them somewhere more visible.
Signed-off-by: Tim Pepper <tpepper@vmware.com>
| -rw-r--r-- | contributors/devel/flaky-tests.md | 13 |
1 files changed, 13 insertions, 0 deletions
diff --git a/contributors/devel/flaky-tests.md b/contributors/devel/flaky-tests.md index f06bb078..14f8da9e 100644 --- a/contributors/devel/flaky-tests.md +++ b/contributors/devel/flaky-tests.md @@ -15,6 +15,19 @@ what caused the failure. Note that flakes can occur in unit tests, integration tests, or end-to-end tests, but probably occur most commonly in end-to-end tests. +## Hunting Flakes + +You may notice lots of your PRs or ones you watch are having a common +pre-submit failure, but less frequent issues that are still of concern take +more analysis over time. There are metrics recorded and viewable in: +- [TestGrid](https://k8s-testgrid.appspot.com/presubmits-kubernetes-blocking#Summary) +- [Velodrome](http://velodrome.k8s.io/dashboard/db/bigquery-metrics?orgId=1) + +It is worth noting tests are going to fail in presubmit a lot due +to unbuildable code, but that wont happen as much on the same commit unless +there's a true issue in the code or a broader problem like a dep failed to +pull in. + ## Filing issues for flaky tests Because flakes may be rare, it's very important that all relevant logs be |