diff options
| author | Kubernetes Prow Robot <k8s-ci-robot@users.noreply.github.com> | 2019-05-08 15:00:50 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2019-05-08 15:00:50 -0700 |
| commit | 5d4d2caf3a0090001f07e224e8814993591198db (patch) | |
| tree | d93fc9bfb3e8bf5077635248b5ed56a7b073b2b1 /contributors | |
| parent | fc89fc347615892504f93a13e631ba400b7f60fe (diff) | |
| parent | d3a393ebe7cdf29c6e64d907679b2506ccfa6484 (diff) | |
Merge pull request #3689 from timothysc/conformance_guidance
Update recommendations for tests that don't meet criteria
Diffstat (limited to 'contributors')
| -rw-r--r-- | contributors/devel/sig-architecture/conformance-tests.md | 10 |
1 files changed, 9 insertions, 1 deletions
diff --git a/contributors/devel/sig-architecture/conformance-tests.md b/contributors/devel/sig-architecture/conformance-tests.md index 90358934..c3069402 100644 --- a/contributors/devel/sig-architecture/conformance-tests.md +++ b/contributors/devel/sig-architecture/conformance-tests.md @@ -63,12 +63,20 @@ Examples of features which are not currently eligible for conformance tests: - cloud-provider-specific features, eg: GCE monitoring, S3 Bucketing, etc. - anything that requires a non-default admission plugin -Examples of tests which are not eligible for promotion to conformance: +Conformance tests are intended to be stable and backwards compatible according to +the standard API deprecation policies. Therefore any test that relies on specific +output that is not subject to the deprecation policy cannot be promoted to conformance. +Examples of tests which are not eligible to conformance: - anything that checks specific Events are generated, as we make no guarantees about the contents of events, nor their delivery + - If a test depends on events it is recommended to change the test to + use an informer pattern and watch specific resource changes instead. - anything that checks optional Condition fields, such as Reason or Message, as these may change over time (however it is reasonable to verify these fields exist or are non-empty) + - If the test is checking for specific conditions or reasons, it is considered + overly specific and it is recommended to simply look for pass/failure criteria + where possible, and output the condition/reason for debugging purposes only. Examples of areas we may want to relax these requirements once we have a sufficient corpus of tests that define out of the box functionality in all |