diff options
| author | eduartua <eduartua@gmail.com> | 2019-01-28 16:21:43 -0600 |
|---|---|---|
| committer | eduartua <eduartua@gmail.com> | 2019-01-28 16:21:43 -0600 |
| commit | f735461a6ece087943cc6ef7c428f7f8235ec1de (patch) | |
| tree | adf584abdfc87aac952aa0405078f648e79bbac0 | |
| parent | b64df87fb19b768962598c606266e84fccbfa9bf (diff) | |
file cri-testing-policy.md was moved to the new sig-node folder - URLs in k/community were updated
| -rw-r--r-- | communication/meeting-notes-archive/q1-2_2018_community_meeting_minutes.md | 2 | ||||
| -rw-r--r-- | contributors/devel/cri-testing-policy.md | 119 | ||||
| -rw-r--r-- | contributors/devel/sig-node/cri-testing-policy.md | 118 | ||||
| -rw-r--r-- | sig-node/charter.md | 2 |
4 files changed, 122 insertions, 119 deletions
diff --git a/communication/meeting-notes-archive/q1-2_2018_community_meeting_minutes.md b/communication/meeting-notes-archive/q1-2_2018_community_meeting_minutes.md index fd7a076f..252ce630 100644 --- a/communication/meeting-notes-archive/q1-2_2018_community_meeting_minutes.md +++ b/communication/meeting-notes-archive/q1-2_2018_community_meeting_minutes.md @@ -1499,7 +1499,7 @@ * Many fixes in the 1.10 release * More great things in the pipeline * SIG Node [Derek Carr] (confirmed) - * New [CRI testing policy](https://github.com/kubernetes/community/blob/master/contributors/devel/cri-testing-policy.md) + * New [CRI testing policy](/contributors/devel/sig-node/cri-testing-policy.md) * Feature going into Beta - Local storage capacity isolation * Feature going into alpha - debug container, supports pod pid limits, cri container log rotation * wg-resource-mgmt : graduated device plugins, hugepages, cpu pinning (beta) diff --git a/contributors/devel/cri-testing-policy.md b/contributors/devel/cri-testing-policy.md index 73b48c5e..975f4884 100644 --- a/contributors/devel/cri-testing-policy.md +++ b/contributors/devel/cri-testing-policy.md @@ -1,118 +1,3 @@ -# Container Runtime Interface: Testing Policy +This file has moved to https://git.k8s.io/community/contributors/devel/sig-node/cri-testing-policy.md. -**Owner: SIG-Node** - -This document describes testing policy and process for runtimes implementing the -[Container Runtime Interface (CRI)](/contributors/devel/container-runtime-interface.md) -to publish test results in a federated dashboard. The objective is to provide -the Kubernetes community an easy way to track the conformance, stability, and -supported features of a CRI runtime. - -This document focuses on Kubernetes node/cluster end-to-end (E2E) testing -because many features require integration of runtime, OS, or even the cloud -provider. A higher-level integration tests provider better signals on vertical -stack compatibility to the Kubernetes community. On the other hand, runtime -developers are strongly encouraged to run low-level -[CRI validation test suite](https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/validation.md) -for validation as part of their development process. - -## Required and optional tests - -Runtime maintainers are **required** to submit the tests listed below. - 1. Node conformance test suite - 2. Node feature test suite - -Node E2E tests qualify an OS image with a pre-installed CRI runtime. The -runtime maintainers are free to choose any OS distribution, packaging, and -deployment mechanism. Please see the -[tutorial](https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-node-tests.md) -to know more about the Node E2E test framework and tests for validating a -compatible OS image. - -The conformance suite is a set of platform-agnostic (e.g., OS, runtime, and -cloud provider) tests that validate the conformance of the OS image. The feature -suite allows the runtime to demonstrate what features are supported with the OS -distribution. - -In addition to the required tests, the runtime maintainers are *strongly -recommended to run and submit results from the Kubernetes conformance test -suite*. This cluster-level E2E test suite provides extra test signal for areas -such as Networking, which cannot be covered by CRI, or Node-level -tests. Because networking requires deep integration between the runtime, the -cloud provider, and/or other cluster components, runtime maintainers are -recommended to reach out to other relevant SIGs (e.g., SIG-GCP or SIG-AWS) for -guidance and/or sponsorship. - -## Process for publishing test results - -To publish tests results, please submit a proposal in the -[Kubernetes community repository](https://github.com/kubernetes/community) -briefly explaining your runtime, providing at least two maintainers, and -assigning the proposal to the leads of SIG-Node. - -These test results should be published under the `sig-node` tab, organized -as follows. - -``` -sig-node -> sig-node-cri-{Kubernetes-version} -> [page containing the required jobs] -``` - -Only the last three most recent Kubernetes versions and the master branch are -kept at any time. This is consistent with the Kubernetes release schedule and -policy. - -## Test job maintenance - -Tests are required to run at least nightly. - -The runtime maintainers are responsible for keeping the tests healthy. If the -tests are deemed not actively maintained, SIG-Node may remove the tests from -the test grid at their discretion. - -## Process for adding pre-submit testing - -If the tests are in good standing (i.e., consistently passing for more than 2 -weeks), the runtime maintainers may request that the tests to be included in the -pre-submit Pull Request (PR) tests. Please note that the pre-submit tests -require significantly higher testing capacity, and are held at a higher standard -since they directly affect the development velocity. - -If the tests are flaky or failing, and the maintainers are unable to respond and -fix the issues in a timely manner, the SIG leads may remove the runtime from -the presubmit tests until the issues are resolved. - -As of now, SIG-Node only accepts promotion of Node conformance tests to -pre-submit because Kubernetes conformance tests involve a wider scope and may -need co-sponsorships from other SIGs. - -## FAQ - - *1. Can runtime maintainers publish results from other E2E tests?* - -Yes, runtime maintainers can publish additional Node E2E tests results. These -test jobs will be displayed in the `sig-node-{runtime-name}` page. The same -policy for test maintenance applies. - -As for additional Cluster E2E tests, SIG-Node may agree to host the -results. However, runtime maintainers are strongly encouraged to seek for a more -appropriate SIG to sponsor or host the results. - - *2. Can these runtime-specific test jobs be considered release blocking?* - -This is beyond the authority of SIG-Node, and requires agreement and consensus -across multiple SIGs (e.g., Release, the relevant cloud provider SIG, etc). - - *3. How to run the aforementioned tests?* - -It is hard to keep instructions are even links to them up-to-date in one -document. Please contact the relevant SIGs for assistance. - - *4. How can I change the test-grid to publish the test results?* - -Please contact SIG-Node for the detailed instructions. - - *5. How does this policy apply to Windows containers?* - -Windows containers are still in the early development phase and the features -they support change rapidly. Therefore, it is suggested to treat it as a -feature with select, whitelisted tests to run. +This file is a placeholder to preserve links. Please remove by April 28, 2019 or the release of kubernetes 1.13, whichever comes first.
\ No newline at end of file diff --git a/contributors/devel/sig-node/cri-testing-policy.md b/contributors/devel/sig-node/cri-testing-policy.md new file mode 100644 index 00000000..73b48c5e --- /dev/null +++ b/contributors/devel/sig-node/cri-testing-policy.md @@ -0,0 +1,118 @@ +# Container Runtime Interface: Testing Policy + +**Owner: SIG-Node** + +This document describes testing policy and process for runtimes implementing the +[Container Runtime Interface (CRI)](/contributors/devel/container-runtime-interface.md) +to publish test results in a federated dashboard. The objective is to provide +the Kubernetes community an easy way to track the conformance, stability, and +supported features of a CRI runtime. + +This document focuses on Kubernetes node/cluster end-to-end (E2E) testing +because many features require integration of runtime, OS, or even the cloud +provider. A higher-level integration tests provider better signals on vertical +stack compatibility to the Kubernetes community. On the other hand, runtime +developers are strongly encouraged to run low-level +[CRI validation test suite](https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/validation.md) +for validation as part of their development process. + +## Required and optional tests + +Runtime maintainers are **required** to submit the tests listed below. + 1. Node conformance test suite + 2. Node feature test suite + +Node E2E tests qualify an OS image with a pre-installed CRI runtime. The +runtime maintainers are free to choose any OS distribution, packaging, and +deployment mechanism. Please see the +[tutorial](https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-node-tests.md) +to know more about the Node E2E test framework and tests for validating a +compatible OS image. + +The conformance suite is a set of platform-agnostic (e.g., OS, runtime, and +cloud provider) tests that validate the conformance of the OS image. The feature +suite allows the runtime to demonstrate what features are supported with the OS +distribution. + +In addition to the required tests, the runtime maintainers are *strongly +recommended to run and submit results from the Kubernetes conformance test +suite*. This cluster-level E2E test suite provides extra test signal for areas +such as Networking, which cannot be covered by CRI, or Node-level +tests. Because networking requires deep integration between the runtime, the +cloud provider, and/or other cluster components, runtime maintainers are +recommended to reach out to other relevant SIGs (e.g., SIG-GCP or SIG-AWS) for +guidance and/or sponsorship. + +## Process for publishing test results + +To publish tests results, please submit a proposal in the +[Kubernetes community repository](https://github.com/kubernetes/community) +briefly explaining your runtime, providing at least two maintainers, and +assigning the proposal to the leads of SIG-Node. + +These test results should be published under the `sig-node` tab, organized +as follows. + +``` +sig-node -> sig-node-cri-{Kubernetes-version} -> [page containing the required jobs] +``` + +Only the last three most recent Kubernetes versions and the master branch are +kept at any time. This is consistent with the Kubernetes release schedule and +policy. + +## Test job maintenance + +Tests are required to run at least nightly. + +The runtime maintainers are responsible for keeping the tests healthy. If the +tests are deemed not actively maintained, SIG-Node may remove the tests from +the test grid at their discretion. + +## Process for adding pre-submit testing + +If the tests are in good standing (i.e., consistently passing for more than 2 +weeks), the runtime maintainers may request that the tests to be included in the +pre-submit Pull Request (PR) tests. Please note that the pre-submit tests +require significantly higher testing capacity, and are held at a higher standard +since they directly affect the development velocity. + +If the tests are flaky or failing, and the maintainers are unable to respond and +fix the issues in a timely manner, the SIG leads may remove the runtime from +the presubmit tests until the issues are resolved. + +As of now, SIG-Node only accepts promotion of Node conformance tests to +pre-submit because Kubernetes conformance tests involve a wider scope and may +need co-sponsorships from other SIGs. + +## FAQ + + *1. Can runtime maintainers publish results from other E2E tests?* + +Yes, runtime maintainers can publish additional Node E2E tests results. These +test jobs will be displayed in the `sig-node-{runtime-name}` page. The same +policy for test maintenance applies. + +As for additional Cluster E2E tests, SIG-Node may agree to host the +results. However, runtime maintainers are strongly encouraged to seek for a more +appropriate SIG to sponsor or host the results. + + *2. Can these runtime-specific test jobs be considered release blocking?* + +This is beyond the authority of SIG-Node, and requires agreement and consensus +across multiple SIGs (e.g., Release, the relevant cloud provider SIG, etc). + + *3. How to run the aforementioned tests?* + +It is hard to keep instructions are even links to them up-to-date in one +document. Please contact the relevant SIGs for assistance. + + *4. How can I change the test-grid to publish the test results?* + +Please contact SIG-Node for the detailed instructions. + + *5. How does this policy apply to Windows containers?* + +Windows containers are still in the early development phase and the features +they support change rapidly. Therefore, it is suggested to treat it as a +feature with select, whitelisted tests to run. diff --git a/sig-node/charter.md b/sig-node/charter.md index 4150b605..dffbf770 100644 --- a/sig-node/charter.md +++ b/sig-node/charter.md @@ -67,7 +67,7 @@ SIG Technical Leads [validation]: https://github.com/kubernetes/community/blob/master/contributors/devel/cri-validation.md -[testing policy]: https://github.com/kubernetes/community/blob/master/contributors/devel/cri-testing-policy.md +[testing policy]: /contributors/devel/sig-node/cri-testing-policy.md [test grid]: https://k8s-testgrid.appspot.com/sig-node#Summary [perf dashboard]: http://node-perf-dash.k8s.io/#/builds [sig-governance]: https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance.md |
