blob: ee21aea2a910bd593a9dd7a837cc6a922af333bc (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
|
# Extending Kubernetes
Note Taker: Clayton Coleman ([smarterclayton](https://github.com/smarterclayton))
* Questions
* Do we have enough extension mechanisms?
* See below
* Implementing network injection that isn’t a CNI injection of some form is hard
* e.g. adding in arbitrary network devices, for example
* Are Flex Volumes enough?
* Maybe?
* Are we doing ok on kubectl extensions?
* Yes, we’re heading in the right direction with plugins
* Kubectl itself should be developed using its own mechanisms
* Extension points:
* OpenAPI metadata (operate on object w/o knowing about it)
* Subresources (generic API-level interfaces for certain facets)
* Plugins (git-style)
* Can we do custom validation in kubectl?
* Do it via admission webhooks (beta in 1.9)
* Can run validation asynchronously, and report it (put a condition in)
* Client-side validation is iffy
* Should we have phased hooks?
* Should more complex application lifecycle layer on top?
* Are we consistent enough across our hooks to enable users to build something sane?
* Can we do referential integrity
* Technically, with a webhook
* Generally not the best idea (causes issues with components that don’t expect it)
* Can maybe do with initializers on a namespace
* Generally go for eventual consistency (e.g. wait for the secret to exist before starting the pod)
* Reasons
* Surface errors to users synchronously
* Do it on the client-side
* Avoid dealing with the full matrix of failure modes
* How do we not re-invent a service mesh with auth webhooks?
* It’s an open question
* Can we do conversions on CRDs
* Maybe with a webhook (ongoing discussion)
* Why aren’t Custom API servers easy enough
* Need OpenShift certificate signing mechanisms in Kube, or similar (also exists similarly in Istio)
* Storage
* Re-use existing etcd
* Use your own etcd
* Planned backing custom API servers with CRD storage
* Why aren’t they more uniform
* Because they came at different times
* It’s hard to fix them now (need more versioning)
* Different layers have different needs
* Declarative is API
* Below Kubelet is gRPC
* gRPC for KMS, CSI
* CNI is shell execution (mainly for legacy reasons)
* Can we make custom controllers easiers?
* OperatorKit
* Need better docs (being worked on)
* Untyped vs generated typed informers and listers
* No Go generics exists
* Can use interfaces
* Can use conversions (may be harder than it needs to be)
* Can wrap in a generic type (e.g. RBAC types)
* Generating clients for CRDs (look for stts’s blog post on generators and informers)
* Existing extension mechanisms
* API Extensions
* External APIs (Aggregated APIs)
* Custom Resources
* Admission/Early-Phase ext points
* Initializers
* Can’t do in updates
* Webhooks
* Need to make them easier to implement (and a good example)
* Pod Presets (ish)
* Webhooks have to be fast, initializers are more async (normal controller style, with restraints)
* Finalizers (late-phase ext points)
* Flex Volumes
* Being used to prototype container identity work, too
* CSI (soon™)
* (grpc) CRI
* (binaries) CNI
* (webhook) Auth plugins
* Authz
* Authn
* Groups from authn hook ???
* (API server) Custom metrics API servers
* (grpc) Device plugins
* (grpc) KMS (Key Management)
* (git style) kubectl plugins
* (API usage) Any controller
* External Cloud Provider
* Pod lifecycle isn’t extensible
* Object phase
* Init containers (do exist)
* defer containers
* Lee’s debug container
* Logging plugins
* Was a proposal for logging volumes, didn’t get follow-up
* New Extension type things
* Enumerations in the API
* Conditions (new condition types)
* Loose coordination between multiple controllers
* Signal to end users
* Exists until formalized as a field
* runc hooks
* Could be a CRI concern
* Optional CRI sub-interfaces?
* Identity injection (custom CA certs in every pod, etc)
* Simplifying the creation of controllers with controller generation ie. metacontroller: [https://github.com/GoogleCloudPlatform/kube-metacontroller](https://github.com/GoogleCloudPlatform/kube-metacontroller)
* API server builder: [https://github.com/kubernetes-incubator/apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder)
* Operator pattern toolkits:
* Rook Team: [https://github.com/rook/operator-kit](https://github.com/rook/operator-kit)
* GiantSwarm: [https://github.com/giantswarm/operatorkit](https://github.com/giantswarm/operatorkit)
|
