diff options
| author | Timothy St. Clair <stclairt@vmware.com> | 2019-05-07 17:18:14 -0500 |
|---|---|---|
| committer | Timothy St. Clair <stclairt@vmware.com> | 2019-05-08 09:22:59 -0500 |
| commit | d3a393ebe7cdf29c6e64d907679b2506ccfa6484 (patch) | |
| tree | dd21e2508486614f3ee2a7ca01abc3c12980962e | |
| parent | 8b7b640ace8bf4b19d516357fd2a2555edaf1f34 (diff) | |
Update recommendations for tests that don't meet
conformance criteria.
| -rw-r--r-- | contributors/devel/sig-architecture/conformance-tests.md | 10 |
1 files changed, 9 insertions, 1 deletions
diff --git a/contributors/devel/sig-architecture/conformance-tests.md b/contributors/devel/sig-architecture/conformance-tests.md index 90358934..c3069402 100644 --- a/contributors/devel/sig-architecture/conformance-tests.md +++ b/contributors/devel/sig-architecture/conformance-tests.md @@ -63,12 +63,20 @@ Examples of features which are not currently eligible for conformance tests: - cloud-provider-specific features, eg: GCE monitoring, S3 Bucketing, etc. - anything that requires a non-default admission plugin -Examples of tests which are not eligible for promotion to conformance: +Conformance tests are intended to be stable and backwards compatible according to +the standard API deprecation policies. Therefore any test that relies on specific +output that is not subject to the deprecation policy cannot be promoted to conformance. +Examples of tests which are not eligible to conformance: - anything that checks specific Events are generated, as we make no guarantees about the contents of events, nor their delivery + - If a test depends on events it is recommended to change the test to + use an informer pattern and watch specific resource changes instead. - anything that checks optional Condition fields, such as Reason or Message, as these may change over time (however it is reasonable to verify these fields exist or are non-empty) + - If the test is checking for specific conditions or reasons, it is considered + overly specific and it is recommended to simply look for pass/failure criteria + where possible, and output the condition/reason for debugging purposes only. Examples of areas we may want to relax these requirements once we have a sufficient corpus of tests that define out of the box functionality in all |
