Merge pull request #354 from fejta/e2e

Add -- flag in hack/e2e.go documentation

2 files changed, 43 insertions, 27 deletions

diff --git a/contributors/devel/e2e-tests.md b/contributors/devel/e2e-tests.md
index 78176f44..6f787673 100644
--- a/contributors/devel/e2e-tests.md
+++ b/contributors/devel/e2e-tests.md
@@ -79,26 +79,26 @@ changing the `KUBERNETES_PROVIDER` environment variable to something other than
 To build Kubernetes, up a cluster, run tests, and tear everything down, use:
 
 ```sh
-go run hack/e2e.go -v --build --up --test --down
+go run hack/e2e.go -- -v --build --up --test --down
 ```
 
 If you'd like to just perform one of these steps, here are some examples:
 
 ```sh
 # Build binaries for testing
-go run hack/e2e.go -v --build
+go run hack/e2e.go -- -v --build
 
 # Create a fresh cluster.  Deletes a cluster first, if it exists
-go run hack/e2e.go -v --up
+go run hack/e2e.go -- -v --up
 
 # Run all tests
-go run hack/e2e.go -v --test
+go run hack/e2e.go -- -v --test
 
 # Run tests matching the regex "\[Feature:Performance\]"
-go run hack/e2e.go -v --test --test_args="--ginkgo.focus=\[Feature:Performance\]"
+go run hack/e2e.go -- -v --test --test_args="--ginkgo.focus=\[Feature:Performance\]"
 
 # Conversely, exclude tests that match the regex "Pods.*env"
-go run hack/e2e.go -v --test --test_args="--ginkgo.skip=Pods.*env"
+go run hack/e2e.go -- -v --test --test_args="--ginkgo.skip=Pods.*env"
 
 # Run tests in parallel, skip any that must be run serially
 GINKGO_PARALLEL=y go run hack/e2e.go --v --test --test_args="--ginkgo.skip=\[Serial\]"
@@ -112,13 +112,13 @@ GINKGO_PARALLEL=y go run hack/e2e.go --v --test --test_args="--ginkgo.skip=\[Ser
 # You can also specify an alternative provider, such as 'aws'
 #
 # e.g.:
-KUBERNETES_PROVIDER=aws go run hack/e2e.go -v --build --up --test --down
+KUBERNETES_PROVIDER=aws go run hack/e2e.go -- -v --build --up --test --down
 
 # -ctl can be used to quickly call kubectl against your e2e cluster. Useful for
 # cleaning up after a failed test or viewing logs. Use -v to avoid suppressing
 # kubectl output.
-go run hack/e2e.go -v -ctl='get events'
-go run hack/e2e.go -v -ctl='delete pod foobar'
+go run hack/e2e.go -- -v -ctl='get events'
+go run hack/e2e.go -- -v -ctl='delete pod foobar'
 ```
 
 The tests are built into a single binary which can be run used to deploy a
@@ -133,11 +133,27 @@ something goes wrong and you still have some VMs running you can force a cleanup
 with this command:
 
 ```sh
-go run hack/e2e.go -v --down
+go run hack/e2e.go -- -v --down
 ```
 
 ## Advanced testing
 
+### Installing/updating kubetest
+
+The logic in `e2e.go` moved out of the main kubernetes repo to test-infra.
+The remaining code in `hack/e2e.go` installs `kubetest` and sends it flags.
+It now lives in [kubernetes/test-infra/kubetest](https://github.com/kubernetes/test-infra/tree/master/kubetest).
+By default `hack/e2e.go` updates and installs `kubetest` once per day.
+Control the updater behavior with the `--get` and `--old` flags:
+The `--` flag separates updater and kubetest flags (kubetest flags on the right).
+
+```sh
+go run hack/e2e.go --get=true --old=1h -- # Update every hour
+go run hack/e2e.go --get=false -- # Never attempt to install/update.
+go install k8s.io/test-infra/kubetest  # Manually install
+go get -u k8s.io/test-infra/kubetest  # Manually update installation
+```
+
 ### Bringing up a cluster for testing
 
 If you want, you may bring up a cluster in some other manner and run tests
@@ -265,7 +281,7 @@ Next, specify the docker repository where your ci images will be pushed.
 * Compile the binaries and build container images:
 
   ```sh
-  $ KUBE_RELEASE_RUN_TESTS=n KUBE_FASTBUILD=true go run hack/e2e.go -v -build
+  $ KUBE_RELEASE_RUN_TESTS=n KUBE_FASTBUILD=true go run hack/e2e.go -- -v -build
   ```
 
 * Push the federation container images
@@ -280,7 +296,7 @@ The following command will create the underlying Kubernetes clusters in each of
 federation control plane in the cluster occupying the last zone in the `E2E_ZONES` list.
 
 ```sh
-$ go run hack/e2e.go -v --up
+$ go run hack/e2e.go -- -v --up
 ```
 
 #### Run the Tests
@@ -288,13 +304,13 @@ $ go run hack/e2e.go -v --up
 This will run only the `Feature:Federation` e2e tests. You can omit the `ginkgo.focus` argument to run the entire e2e suite.
 
 ```sh
-$ go run hack/e2e.go -v --test --test_args="--ginkgo.focus=\[Feature:Federation\]"
+$ go run hack/e2e.go -- -v --test --test_args="--ginkgo.focus=\[Feature:Federation\]"
 ```
 
 #### Teardown
 
 ```sh
-$ go run hack/e2e.go -v --down
+$ go run hack/e2e.go -- -v --down
 ```
 
 #### Shortcuts for test developers
@@ -364,13 +380,13 @@ at a custom host directly:
 export KUBECONFIG=/path/to/kubeconfig
 export KUBE_MASTER_IP="http://127.0.0.1:<PORT>"
 export KUBE_MASTER=local
-go run hack/e2e.go -v --test
+go run hack/e2e.go -- -v --test
 ```
 
 To control the tests that are run:
 
 ```sh
-go run hack/e2e.go -v --test --test_args="--ginkgo.focus=\"Secrets\""
+go run hack/e2e.go -- -v --test --test_args="--ginkgo.focus=\"Secrets\""
 ```
 
 ### Version-skewed and upgrade testing
@@ -403,7 +419,7 @@ export CLUSTER_API_VERSION=${OLD_VERSION}
 
 # Deploy a cluster at the old version; see above for more details
 cd ./kubernetes_old
-go run ./hack/e2e.go -v --up
+go run ./hack/e2e.go -- -v --up
 
 # Upgrade the cluster to the new version
 #
@@ -411,11 +427,11 @@ go run ./hack/e2e.go -v --up
 #
 # You can target Feature:MasterUpgrade or Feature:ClusterUpgrade
 cd ../kubernetes
-go run ./hack/e2e.go -v --test --check_version_skew=false --test_args="--ginkgo.focus=\[Feature:MasterUpgrade\]"
+go run ./hack/e2e.go -- -v --test --check_version_skew=false --test_args="--ginkgo.focus=\[Feature:MasterUpgrade\]"
 
 # Run old tests with new kubectl
 cd ../kubernetes_old
-go run ./hack/e2e.go -v --test --test_args="--kubectl-path=$(pwd)/../kubernetes/cluster/kubectl.sh"
+go run ./hack/e2e.go -- -v --test --test_args="--kubectl-path=$(pwd)/../kubernetes/cluster/kubectl.sh"
 ```
 
 If you are just testing version-skew, you may want to just deploy at one
@@ -427,14 +443,14 @@ upgrade process:
 
 # Deploy a cluster at the new version
 cd ./kubernetes
-go run ./hack/e2e.go -v --up
+go run ./hack/e2e.go -- -v --up
 
 # Run new tests with old kubectl
-go run ./hack/e2e.go -v --test --test_args="--kubectl-path=$(pwd)/../kubernetes_old/cluster/kubectl.sh"
+go run ./hack/e2e.go -- -v --test --test_args="--kubectl-path=$(pwd)/../kubernetes_old/cluster/kubectl.sh"
 
 # Run old tests with new kubectl
 cd ../kubernetes_old
-go run ./hack/e2e.go -v --test --test_args="--kubectl-path=$(pwd)/../kubernetes/cluster/kubectl.sh"
+go run ./hack/e2e.go -- -v --test --test_args="--kubectl-path=$(pwd)/../kubernetes/cluster/kubectl.sh"
 ```
 
 ## Kinds of tests
@@ -536,13 +552,13 @@ export KUBERNETES_CONFORMANCE_TEST=y
 export KUBERNETES_PROVIDER=skeleton
 
 # run all conformance tests
-go run hack/e2e.go -v --test --test_args="--ginkgo.focus=\[Conformance\]"
+go run hack/e2e.go -- -v --test --test_args="--ginkgo.focus=\[Conformance\]"
 
 # run all parallel-safe conformance tests in parallel
-GINKGO_PARALLEL=y go run hack/e2e.go -v --test --test_args="--ginkgo.focus=\[Conformance\] --ginkgo.skip=\[Serial\]"
+GINKGO_PARALLEL=y go run hack/e2e.go -- -v --test --test_args="--ginkgo.focus=\[Conformance\] --ginkgo.skip=\[Serial\]"
 
 # ... and finish up with remaining tests in serial
-go run hack/e2e.go -v --test --test_args="--ginkgo.focus=\[Serial\].*\[Conformance\]"
+go run hack/e2e.go -- -v --test --test_args="--ginkgo.focus=\[Serial\].*\[Conformance\]"
 ```
 
 ### Defining Conformance Subset
diff --git a/contributors/devel/node-performance-testing.md b/contributors/devel/node-performance-testing.md
index d6bb657f..60368e20 100644
--- a/contributors/devel/node-performance-testing.md
+++ b/contributors/devel/node-performance-testing.md
@@ -59,9 +59,9 @@ sampling.
 There is an end-to-end test for collecting overall resource usage of node
 components: [kubelet_perf.go](../../test/e2e/kubelet_perf.go). To
 run the test, simply make sure you have an e2e cluster running (`go run
-hack/e2e.go -up`) and [set up](#cluster-set-up) correctly.
+hack/e2e.go -- -up`) and [set up](#cluster-set-up) correctly.
 
-Run the test with `go run hack/e2e.go -v -test
+Run the test with `go run hack/e2e.go -- -v -test
 --test_args="--ginkgo.focus=resource\susage\stracking"`. You may also wish to
 customise the number of pods or other parameters of the test (remember to rerun
 `make WHAT=test/e2e/e2e.test` after you do).