diff options
| author | Kubernetes Prow Robot <k8s-ci-robot@users.noreply.github.com> | 2021-07-26 12:54:33 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2021-07-26 12:54:33 -0700 |
| commit | ebf05b5f5a2239f2e9de00835ac1083eee4eeb9b (patch) | |
| tree | c35a64f0ac1ad882239d7d3469dac80e6a7fb712 | |
| parent | 89026dd65e2083b2937272b81b9d51a594810450 (diff) | |
| parent | 51b267d7a4fd545dce354e195325dd35fd3085d9 (diff) | |
Merge pull request #5909 from SataQiu/remove-bazel-build
Remove bazel build
| -rw-r--r-- | contributors/devel/README.md | 2 | ||||
| -rw-r--r-- | contributors/devel/development.md | 7 | ||||
| -rw-r--r-- | contributors/devel/sig-architecture/conformance-tests.md | 12 | ||||
| -rw-r--r-- | contributors/devel/sig-testing/bazel.md | 183 |
4 files changed, 0 insertions, 204 deletions
diff --git a/contributors/devel/README.md b/contributors/devel/README.md index fb2129c0..845a0eb4 100644 --- a/contributors/devel/README.md +++ b/contributors/devel/README.md @@ -161,8 +161,6 @@ See the [kubernetes/release](https://github.com/kubernetes/release) repository f * **Flaky tests** [flaky-tests.md](sig-testing/flaky-tests.md) -* **Build and test with Bazel** [bazel.md](sig-testing/bazel.md) - ### SIG Node * **CRI: the Container Runtime Interface** [container-runtime-interface.md](sig-node/container-runtime-interface.md) diff --git a/contributors/devel/development.md b/contributors/devel/development.md index 09987a40..6dbbbd8b 100644 --- a/contributors/devel/development.md +++ b/contributors/devel/development.md @@ -481,12 +481,6 @@ Developers who need to manage dependencies in the `vendor/` tree should read the docs on [using go modules to manage dependencies](/contributors/devel/sig-architecture/vendor.md). -## Build with Bazel/Gazel - -Building with Bazel is currently experimental. For more information, -see [Build with Bazel]. - - ## GitHub workflow To check out code to work on, please refer to [this guide](/contributors/guide/github-workflow.md). @@ -503,4 +497,3 @@ To check out code to work on, please refer to [this guide](/contributors/guide/g [kubectl user guide]: https://kubernetes.io/docs/user-guide/kubectl [kubernetes.io]: https://kubernetes.io [mercurial]: http://mercurial.selenic.com/wiki/Download -[Build with Bazel]: sig-testing/bazel.md diff --git a/contributors/devel/sig-architecture/conformance-tests.md b/contributors/devel/sig-architecture/conformance-tests.md index 9f293863..e0614cb1 100644 --- a/contributors/devel/sig-architecture/conformance-tests.md +++ b/contributors/devel/sig-architecture/conformance-tests.md @@ -188,11 +188,7 @@ appropriate endpoint and credentials. 2. Create your kind node image: ```sh -# Without bazel kind build node-image - -# Or with bazel -kind build node-image --type bazel ``` 3. Create your kind e2e cluster config kind-config-yaml: @@ -225,21 +221,13 @@ kind create cluster --config kind-config.yaml --image kindest/node:latest -v4 6. Create your e2e Kubernetes binary (from your Kubernetes src code): ```sh -# Without bazel make WHAT="test/e2e/e2e.test" - -# Or with bazel -bazel build //test/e2e:e2e.test ``` 7. Execute your tests: ```sh -# Without bazel ./_output/bin/e2e.test -context kind-kind -ginkgo.focus="\[sig-network\].*Conformance" -num-nodes 2 - -# Or with bazel -./bazel-bin/test/e2e/e2e.test -context kind-kind -ginkgo.focus="\[sig-network\].*Conformance" -num-nodes 2 ``` ### Running Conformance Tests With kubetest diff --git a/contributors/devel/sig-testing/bazel.md b/contributors/devel/sig-testing/bazel.md deleted file mode 100644 index 8f0f827b..00000000 --- a/contributors/devel/sig-testing/bazel.md +++ /dev/null @@ -1,183 +0,0 @@ -# Build and test with Bazel - -Building and testing Kubernetes with Bazel is supported but not yet default. - -Bazel is used to run all Kubernetes PRs on [Prow], -as remote caching enables significantly reduced build and test times. - -Some repositories (such as kubernetes/test-infra) have switched to using Bazel -exclusively for all build, test, and release workflows. - -Go rules are managed by the [`gazelle`][gazelle] -tool, with some additional rules managed by the [`kazel`][kazel] tool. -These tools are called via the `hack/update-bazel.sh` script. - -Kubernetes requires [a specific version of Bazel](https://github.com/kubernetes/kubernetes/blob/master/build/root/.bazelversion). -[Consider using Bazelisk](https://docs.bazel.build/versions/master/install-bazelisk.html) -which automatically installs the version of Bazel that Kubernetes requires: - - go get github.com/bazelbuild/bazelisk - bazelisk test //pkg/kubectl/... - -If you `alias bazel=bazelisk`, then you can use several convenience `make` rules which have been created for common operations: - -* `make bazel-build`: builds all binaries in tree (`bazel build -- //... - -//vendor/...`) -* `make bazel-test`: runs all unit tests (`bazel test --config=unit -- //... - //hack:verify-all -//build/... -//vendor/...`) -* `make bazel-test-integration`: runs all integration tests (`bazel test - --config integration //test/integration/...`) -* `make bazel-release`: builds release tarballs, Docker images (for server - components), and Debian images (`bazel build //build/release-tars`) - -You can also interact with Bazel directly; for example, to run all `kubectl` unit -tests, run - -```console -$ bazel test //pkg/kubectl/... -``` - -[Prow]: https://prow.k8s.io -[bazel-install]: https://www.bazel.io/versions/master/docs/install.html -[kazel]: https://git.k8s.io/repo-infra/kazel -[gazelle]: https://github.com/bazelbuild/rules_go/tree/master/go/tools/gazelle -[bazel-python-faq]: https://github.com/bazelbuild/bazel/issues/7899 - -## Continuous Integration - -There are several bazel CI jobs: -* [ci-kubernetes-bazel-build](http://k8s-testgrid.appspot.com/google-unit#bazel-build): builds everything - with Bazel -* [ci-kubernetes-bazel-test](http://k8s-testgrid.appspot.com/google-unit#bazel-test): runs unit tests in - with Bazel - -Similar jobs are run on all PRs; additionally, several of the e2e jobs use -Bazel-built binaries when launching and testing Kubernetes clusters. - -## Updating `BUILD` files - -To update `BUILD` files, run: - -```console -$ ./hack/update-bazel.sh -``` - -To prevent Go rules from being updated, consult the [gazelle -documentation](https://github.com/bazelbuild/rules_go/tree/master/go/tools/gazelle). - -Note that much like Go files and `gofmt`, `BUILD` files have standardized, -opinionated style rules, and running `hack/update-bazel.sh` will format them for you. - -If you want to auto-format `BUILD` files in your editor, use of -[Buildifier](https://github.com/bazelbuild/buildtools/blob/master/buildifier/README.md) -is recommended. - -Updating the `BUILD` file for a package will be required when: -* Files are added to or removed from a package -* Import dependencies change for a package -* A `BUILD` file has been updated and needs to be reformatted -* A new `BUILD` file has been added (parent `BUILD` files will be updated) - -## Known issues and limitations - -### [Cross-compilation of cgo is not currently natively supported](https://github.com/bazelbuild/rules_go/issues/1020) -All binaries are currently built for the host OS and architecture running Bazel. -(For example, you can't currently target linux/amd64 from macOS or linux/s390x -from an amd64 machine.) - -The Go rules support cross-compilation of pure Go code using the `--platforms` -flag, and this is being used successfully in the kubernetes/test-infra repo. - -It may already be possible to cross-compile cgo code if a custom CC toolchain is -set up, possibly reusing the kube-cross Docker image, but this area needs -further exploration. - -### The CC toolchain is not fully hermetic -Bazel requires several tools and development packages to be installed in the system, including `gcc`, `g++`, `glibc and libstdc++ development headers` and `glibc static development libraries`. Please check your distribution for exact names of the packages. Examples for some commonly used distributions are below: - -| Dependency | Debian/Ubuntu | CentOS | openSUSE | -|:---------------------:|-------------------------------|--------------------------------|-----------------------------------------| -| Build essentials | `apt install build-essential` | `yum groupinstall development` | `zypper install -t pattern devel_C_C++` | -| GCC C++ | `apt install g++` | `yum install gcc-c++` | `zypper install gcc-c++` | -| GNU Libc static files | `apt install libc6-dev` | `yum install glibc-static` | `zypper install glibc-devel-static` | - -If any of these packages change, they may also cause spurious build failures -as described in [this issue](https://github.com/bazelbuild/bazel/issues/4907). - -An example error might look something like -``` -ERROR: undeclared inclusion(s) in rule '//vendor/golang.org/x/text/cases:go_default_library.cgo_c_lib': -this rule is missing dependency declarations for the following files included by 'vendor/golang.org/x/text/cases/linux_amd64_stripped/go_default_library.cgo_codegen~/_cgo_export.c': - '/usr/lib/gcc/x86_64-linux-gnu/7/include/stddef.h' -``` - -The only way to recover from this error is to force Bazel to regenerate its -automatically-generated CC toolchain configuration by running `bazel clean ---expunge`. - -Improving cgo cross-compilation may help with all of this. - -### Changes to Go imports requires updating BUILD files -The Go rules in `BUILD` and `BUILD.bazel` files must be updated any time files -are added or removed or Go imports are changed. These rules are automatically -maintained by `gazelle`, which is run via `hack/update-bazel.sh`, but this is -still a source of friction. - -[Autogazelle](https://github.com/bazelbuild/bazel-gazelle/tree/master/cmd/autogazelle) -is a new experimental tool which may reduce or remove the need for developers -to run `hack/update-bazel.sh`, but no work has yet been done to support it in -kubernetes/kubernetes. - -### Code coverage support is incomplete for Go -Bazel and the Go rules have limited support for code coverage. Running something -like `bazel coverage -- //... -//vendor/...` will run tests in coverage mode, -but no report summary is currently generated. It may be possible to combine -`bazel coverage` with -[Gopherage](https://github.com/kubernetes/test-infra/tree/master/gopherage), -however. - -### Kubernetes code generators are not fully supported -The make-based build system in kubernetes/kubernetes runs several code -generators at build time: -* [conversion-gen](https://github.com/kubernetes/code-generator/tree/master/cmd/conversion-gen) -* [deepcopy-gen](https://github.com/kubernetes/code-generator/tree/master/cmd/deepcopy-gen) -* [defaulter-gen](https://github.com/kubernetes/code-generator/tree/master/cmd/defaulter-gen) -* [openapi-gen](https://github.com/kubernetes/kube-openapi/tree/master/cmd/openapi-gen) -* [go-bindata](https://github.com/jteeuwen/go-bindata/tree/master/go-bindata) - -Of these, only `openapi-gen` and `go-bindata` are currently supported when -building Kubernetes with Bazel. - -The `go-bindata` generated code is produced by hand-written genrules. - -The other code generators use special build tags of the form `// -+k8s:generator-name=arg`; for example, input files to the openapi-gen tool are -specified with `// +k8s:openapi-gen=true`. - -`kazel` is used to find all packages that require OpenAPI generation, and then a -handwritten genrule consumes this list of packages to run `openapi-gen`. - -For `openapi-gen`, a single output file is produced in a single Go package, which -makes this fairly compatible with Bazel. -All other Kubernetes code generators generally produce one output file per input -package, which is less compatible with the Bazel workflow. - -The make-based build system batches up all input packages into one call to the -code generator binary, but this is inefficient for Bazel's incrementality, as a -change in one package may result in unnecessarily recompiling many other -packages. -On the other hand, calling the code generator binary multiple times is less -efficient than calling it once, since many of the generators parse the tree for -Go type information and other metadata. - -One additional challenge is that many of the code generators add additional -Go imports which `gazelle` (and `autogazelle`) cannot infer, and so they must be -explicitly added as dependencies in the `BUILD` files. - -Kubernetes has even more code generators than this limited list, but the rest -are generally run as `hack/update-*.sh` scripts and checked into the repository, -and so are not immediately needed for Bazel parity. - -## Contacts -For help or discussion, join the [#bazel](https://kubernetes.slack.com/messages/bazel) -channel on Kubernetes Slack. |
