summaryrefslogtreecommitdiff
path: root/contributors
diff options
context:
space:
mode:
authorCheng Xing <cxing@google.com>2019-02-05 17:24:47 -0800
committerCheng Xing <cxing@google.com>2019-02-19 16:51:55 -0800
commit52469a11e134ba732efce96c341c865921d369b5 (patch)
tree599d4ae26c159438ea98f6542eb35eb39258dd9e /contributors
parent284749fe70925003877b9ff7e4732f020f13de06 (diff)
Updated CSI doc with latest deployment method information
Diffstat (limited to 'contributors')
-rw-r--r--contributors/design-proposals/storage/container-storage-interface.md45
-rw-r--r--contributors/design-proposals/storage/container-storage-interface_diagram1.pngbin69333 -> 64049 bytes
2 files changed, 30 insertions, 15 deletions
diff --git a/contributors/design-proposals/storage/container-storage-interface.md b/contributors/design-proposals/storage/container-storage-interface.md
index e368b4ac..cee2a085 100644
--- a/contributors/design-proposals/storage/container-storage-interface.md
+++ b/contributors/design-proposals/storage/container-storage-interface.md
@@ -410,35 +410,39 @@ When the controller decides to detach a CSI volume, it will call the in-tree CSI
### Recommended Mechanism for Deploying CSI Drivers on Kubernetes
-Although, Kubernetes does not dictate the packaging for a CSI volume driver, it offers the following recommendations to simplify deployment of a containerized CSI volume driver on Kubernetes.
+Although Kubernetes does not dictate the packaging for a CSI volume driver, it offers the following recommendations to simplify deployment of a containerized CSI volume driver on Kubernetes.
![Recommended CSI Deployment Diagram](container-storage-interface_diagram1.png?raw=true "Recommended CSI Deployment Diagram")
To deploy a containerized third-party CSI volume driver, it is recommended that storage vendors:
* Create a “CSI volume driver” container that implements the volume plugin behavior and exposes a gRPC interface via a unix domain socket, as defined in the CSI spec (including Controller, Node, and Identity services).
- * Bundle the “CSI volume driver” container with helper containers (external-attacher, external-provisioner, Kubernetes CSI Helper) that the Kubernetes team will provide (these helper containers will assist the “CSI volume driver” container in interacting with the Kubernetes system). More specifically, create the following Kubernetes objects:
- * A `StatefulSet` (to facilitate communication with the Kubernetes controllers) that has:
- * Replica size 1
- * Guarantees that no more than 1 instance of the pod will be running at once (so we don’t have to worry about multiple instances of the `external-provisioner` or `external-attacher` in the cluster).
+ * Bundle the “CSI volume driver” container with helper containers (external-attacher, external-provisioner, node-driver-registrar, cluster-driver-registrar, external-resizer, external-snapshotter, livenessprobe) that the Kubernetes team will provide (these helper containers will assist the “CSI volume driver” container in interacting with the Kubernetes system). More specifically, create the following Kubernetes objects:
+ * To facilitate communication with the Kubernetes controllers, a `StatefulSet` or a `Deployment` (depending on the user's need; see [Cluster-Level Deployment](#cluster-level-deployment)) that has:
* The following containers
* The “CSI volume driver” container created by the storage vendor.
- * The `external-attacher` container provided by the Kubernetes team.
- * The `external-provisioner` container provided by the Kubernetes team.
+ * Containers provided by the Kubernetes team (all of which are optional):
+ * `cluster-driver-registrar` (refer to the README in `cluster-driver-registrar` repository for when the container is required)
+ * `external-provisioner` (required for provision/delete operations)
+ * `external-attacher` (required for attach/detach operations. If you wish to skip the attach step, CSISkipAttach feature must be enabled in Kubernetes in addition to omitting this container)
+ * `external-resizer` (required for resize operations)
+ * `external-snapshotter` (required for volume-level snapshot operations)
+ * `livenessprobe`
* The following volumes:
* `emptyDir` volume
- * Mounted inside all containers at `/var/lib/csi/sockets/pluginproxy/`
- * The “CSI volume driver” container should create its Unix Domain Socket in this directory to enable communication with the Kubernetes helper container(s) (`external-provisioner`, `external-attacher`).
+ * Mounted by all containers, including the “CSI volume driver”.
+ * The “CSI volume driver” container should create its Unix Domain Socket in this directory to enable communication with the Kubernetes helper container(s).
* A `DaemonSet` (to facilitate communication with every instance of kubelet) that has:
* The following containers
* The “CSI volume driver” container created by the storage vendor.
- * The “Kubernetes CSI Helper” container provided by the Kubernetes team
- * Responsible for registering the unix domain socket with kubelet and initializing NodeId.
+ * Containers provided by the Kubernetes team:
+ * `node-driver-registrar` - Responsible for registering the unix domain socket with kubelet.
+ * `livenessprobe` (optional)
* The following volumes:
* `hostpath` volume
- * Expose `/var/lib/kubelet/device-plugins/kubelet.sock` from the host.
- * Mount only in “Kubernetes CSI Helper” container at `/var/lib/csi/sockets/kubelet.sock`
- * The Kubernetes to CSI proxy container will use this unix domain socket to register the CSI driver’s unix domain socket with kubelet.
+ * Expose `/var/lib/kubelet/plugins_registry` from the host.
+ * Mount only in `node-driver-registrar` container at `/registration`
+ * `node-driver-registrar` will use this unix domain socket to register the CSI driver’s unix domain socket with kubelet.
* `hostpath` volume
* Expose `/var/lib/kubelet/` from the host.
* Mount only in “CSI volume driver” container at `/var/lib/kubelet/`
@@ -451,6 +455,17 @@ To deploy a containerized third-party CSI volume driver, it is recommended that
Alternatively, deployment could be simplified by having all components (including external-provisioner and external-attacher) in the same pod (DaemonSet). Doing so, however, would consume more resources, and require a leader election protocol (likely https://git.k8s.io/contrib/election) in the `external-provisioner` and `external-attacher` components.
+Containers provided by Kubernetes are maintained in [GitHub kubernetes-csi organization](https://github.com/kubernetes-csi).
+
+#### Cluster-Level Deployment
+Containers in the cluster-level deployment may be deployed in one of the following configurations:
+
+1. StatefulSet with single replica. Good for clusters with a single dedicated node to run the cluster-level pod. A StatefulSet guarantees that no more than 1 instance of the pod will be running at once. One downside is that if the node becomes unresponsive, the replica will never be deleted and recreated.
+1. Deployment with multiple replicas and leader election enabled (if supported by the container). Good for admins who prefer faster recovery time in case the main replica fails, at a cost of higher resource usage (especially memory).
+1. Deployment with a single replica and leader election enabled (if supported by the container). A compromise between the above two options. If the replica is detected to be failed, a new replica can be scheduled almost immediately.
+
+Note that certain cluster-level containers, such as `external-provisioner`, `external-attacher`, `external-resizer`, and `external-snapshotter`, may require credentials to the storage backend, and as such, admins may choose to run them on dedicated "infrastructure" nodes (such as master nodes) that don't run user pods.
+
#### Topology Representation in Node Objects
Topology information will be represented as labels.
@@ -780,4 +795,4 @@ We dismissed this approach because having attach/detach triggered by the creatio
[Flex Volume]: /contributors/devel/sig-storage/flexvolume.md
-[Flex Volume plugin]: /contributors/devel/sig-storage/flexvolume.md \ No newline at end of file
+[Flex Volume plugin]: /contributors/devel/sig-storage/flexvolume.md
diff --git a/contributors/design-proposals/storage/container-storage-interface_diagram1.png b/contributors/design-proposals/storage/container-storage-interface_diagram1.png
index eb42add7..93230659 100644
--- a/contributors/design-proposals/storage/container-storage-interface_diagram1.png
+++ b/contributors/design-proposals/storage/container-storage-interface_diagram1.png
Binary files differ