diff options
| author | Marie Shaw <mnshaw@google.com> | 2016-08-16 15:53:26 -0700 |
|---|---|---|
| committer | Marie Shaw <mnshaw@google.com> | 2016-08-19 10:05:33 -0700 |
| commit | e774267211edc2b5e9aa65a981464f24bbccfd76 (patch) | |
| tree | 93bb9dc760a2cc961d7a2a48e10fe11b86e1b05b | |
| parent | da2d4622d875a71b0bf084bb12ac12af3958c1cb (diff) | |
Add gubernator.md and image files
| -rw-r--r-- | gubernator-images/filterpage.png | bin | 0 -> 408077 bytes | |||
| -rw-r--r-- | gubernator-images/filterpage1.png | bin | 0 -> 375248 bytes | |||
| -rw-r--r-- | gubernator-images/filterpage2.png | bin | 0 -> 372828 bytes | |||
| -rw-r--r-- | gubernator-images/filterpage3.png | bin | 0 -> 362554 bytes | |||
| -rw-r--r-- | gubernator-images/skipping1.png | bin | 0 -> 67007 bytes | |||
| -rw-r--r-- | gubernator-images/skipping2.png | bin | 0 -> 114503 bytes | |||
| -rw-r--r-- | gubernator-images/testfailures.png | bin | 0 -> 189178 bytes | |||
| -rw-r--r-- | gubernator.md | 171 |
8 files changed, 171 insertions, 0 deletions
diff --git a/gubernator-images/filterpage.png b/gubernator-images/filterpage.png Binary files differnew file mode 100644 index 00000000..2d08bd8e --- /dev/null +++ b/gubernator-images/filterpage.png diff --git a/gubernator-images/filterpage1.png b/gubernator-images/filterpage1.png Binary files differnew file mode 100644 index 00000000..838cb0fa --- /dev/null +++ b/gubernator-images/filterpage1.png diff --git a/gubernator-images/filterpage2.png b/gubernator-images/filterpage2.png Binary files differnew file mode 100644 index 00000000..63da782e --- /dev/null +++ b/gubernator-images/filterpage2.png diff --git a/gubernator-images/filterpage3.png b/gubernator-images/filterpage3.png Binary files differnew file mode 100644 index 00000000..33066d78 --- /dev/null +++ b/gubernator-images/filterpage3.png diff --git a/gubernator-images/skipping1.png b/gubernator-images/skipping1.png Binary files differnew file mode 100644 index 00000000..a5dea440 --- /dev/null +++ b/gubernator-images/skipping1.png diff --git a/gubernator-images/skipping2.png b/gubernator-images/skipping2.png Binary files differnew file mode 100644 index 00000000..b133347e --- /dev/null +++ b/gubernator-images/skipping2.png diff --git a/gubernator-images/testfailures.png b/gubernator-images/testfailures.png Binary files differnew file mode 100644 index 00000000..1b331248 --- /dev/null +++ b/gubernator-images/testfailures.png diff --git a/gubernator.md b/gubernator.md new file mode 100644 index 00000000..f7ac0318 --- /dev/null +++ b/gubernator.md @@ -0,0 +1,171 @@ +<!-- BEGIN MUNGE: UNVERSIONED_WARNING --> + +<!-- BEGIN STRIP_FOR_RELEASE --> + +<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" + width="25" height="25"> +<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" + width="25" height="25"> +<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" + width="25" height="25"> +<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" + width="25" height="25"> +<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" + width="25" height="25"> + +<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2> + +If you are using a released version of Kubernetes, you should +refer to the docs that go with that version. + +Documentation for other releases can be found at +[releases.k8s.io](http://releases.k8s.io). +</strong> +-- + +<!-- END STRIP_FOR_RELEASE --> + +<!-- END MUNGE: UNVERSIONED_WARNING --> + +# Gubernator + +*This document is oriented at developers who want to use Gubernator to debug while developing for Kubernetes.* + +<!-- BEGIN MUNGE: GENERATED_TOC --> + +- [Gubernator](#gubernator) + - [What is Gubernator?](#what-is-gubernator) + - [Gubernator Features](#gubernator-features) + - [Test Failures list](#test-failures-list) + - [Log Filtering](#log-filtering) + - [Gubernator for Local Tests](#gubernator-for-local-tests) + - [Future Work](#future-work) + +<!-- END MUNGE: GENERATED_TOC --> + +## What is Gubernator? + +[Gubernator](https://k8s-gubernator.appspot.com/) is a webpage for viewing and filtering Kubernetes +test results. + +Gubernator simplifies the debugging proccess and makes it easier to track down failures by automating many +steps commonly taken in searching through logs, and by offering tools to filter through logs to find relevant lines. +Gubernator automates the steps of finding the failed tests, displaying relevant logs, and determining the +failed pods and the corresponing pod UID, namespace, and container ID. +It also allows for filtering of the log files to display relevant lines based on selected keywords, and +allows for multiple logs to be woven together by timestamp. + +Gubernator runs on Google App Engine and fetches logs stored on Google Cloud Storage. + +## Gubernator Features + +### Test Failures list + +Issues made by k8s-merge-robot will post a link to a page listing the failed tests. +Each failed test comes with the corresponding error log from a junit file and a link +to filter logs for that test. + +Based on the message logged in the junit file, the pod name may be displayed. + + + +[Test Failures List Example](https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-e2e-gke/11721) + +### Log Filtering + +The log filtering page comes with checkboxes and textboxes to aid in filtering. Filtered keywords will be bolded +and lines including keywords will be highlighted. Up to four lines around the line of interest will also be displayed. + + + +If less than 100 lines are skipped, the "... skipping xx lines ..." message can be clicked to expand and show +the hidden lines. + +Before expansion: + +After expansion: + + +If the pod name was displayed in the Test Failures list, it will automatically be included in the filters. +If it is not found in the error message, it can be manually entered into the textbox. Once a pod name +is entered, the Pod UID, Namespace, and ContainerID may be automatically filled in as well. These can be +altered as well. To apply the filter, check off the options corresponding to the filter. + + + +To add a filter, type the term to be filtered into the textbox labeled "Add filter:" and press enter. +Additional filters will be displayed as checkboxes under the textbox. + + + +To choose which logs to view check off the checkboxes corresponding to the logs of interest. If multiple logs are +included, the "Weave by timestamp" option can weave the selected logs together based on the timestamp in each line. + + + +[Log Filtering Example 1](https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubelet-gce-e2e-ci/5535/nodelog?pod=pod-configmaps-b5b876cb-3e1e-11e6-8956-42010af0001d&junit=junit_03.xml&wrap=on&logfiles=%2Fkubernetes-jenkins%2Flogs%2Fkubelet-gce-e2e-ci%2F5535%2Fartifacts%2Ftmp-node-e2e-7a5a3b40-e2e-node-coreos-stable20160622-image%2Fkube-apiserver.log&logfiles=%2Fkubernetes-jenkins%2Flogs%2Fkubelet-gce-e2e-ci%2F5535%2Fartifacts%2Ftmp-node-e2e-7a5a3b40-e2e-node-coreos-stable20160622-image%2Fkubelet.log&UID=on&poduid=b5b8a59e-3e1e-11e6-b358-42010af0001d&ns=e2e-tests-configmap-oi12h&cID=tmp-node-e2e-7a5a3b40-e2e-node-coreos-stable20160622-image) + +[Log Filtering Example 2](https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-e2e-gke/11721/nodelog?pod=client-containers-a53f813c-503e-11e6-88dd-0242ac110003&junit=junit_19.xml&wrap=on) + + +### Gubernator for Local Tests + +*Currently Gubernator can only be used with remote node e2e tests.* + +**NOTE: Using Gubernator with local tests will publically upload your test logs to Google Cloud Storage** + +To use Gubernator to view logs from local test runs, set the GUBERNATOR tag to true. +A URL link to view the test results will be printed to the console. +Please note that running with the Gubernator tag will bypass the user confirmation for uploading to GCS. + +```console + +$ make test-e2e-node REMOTE=true GUBERNATOR=true +... +================================================================ +Running gubernator.sh + +Gubernator linked below: +k8s-gubernator.appspot.com/build/yourusername-g8r-logs/logs/e2e-node/timestamp +``` + +The gubernator.sh script can be run after running a remote node e2e test for the same effect. + +```console +$ ./test/e2e_node/gubernator.sh +Do you want to run gubernator.sh and upload logs publicly to GCS? [y/n]y +... +Gubernator linked below: +k8s-gubernator.appspot.com/build/yourusername-g8r-logs/logs/e2e-node/timestamp +``` + +## Future Work + +Gubernator provides a framework for debugging failures and introduces useful features. +There is still a lot of room for more features and growth to make the debugging process more efficient. + +How to contribute (see https://github.com/kubernetes/test-infra/blob/master/gubernator/README.md) + +* Extend GUBERNATOR flag to all local tests + +* More accurate identification of pod name, container ID, etc. + * Change content of logged strings for failures to include more information + * Better regex in Gubernator + +* Automate discovery of more keywords + * Volume Name + * Disk Name + * Pod IP + +* Clickable API objects in the displayed lines in order to add them as filters + +* Construct story of pod's lifetime + * Have concise view of what a pod went through from when pod was started to failure + +* Improve UI + * Have separate folders of logs in rows instead of in one long column + * Improve interface for adding additional features (maybe instead of textbox and checkbox, have chips) + +<!-- BEGIN MUNGE: GENERATED_ANALYTICS --> +[]() +<!-- END MUNGE: GENERATED_ANALYTICS --> |
