diff options
| author | Brian Grant <bgrant0607@users.noreply.github.com> | 2017-02-21 11:02:41 -0800 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2017-02-21 11:02:41 -0800 |
| commit | ece19783f28b4e5c11b7990c25f2cd59eb35bcc9 (patch) | |
| tree | fa1fb514e45d6222cb0279c706c114baff3c1a4b | |
| parent | 7a444fa1cc67d4f23329c277f324de7e643b5cba (diff) | |
Update architecture.md
| -rw-r--r-- | contributors/design-proposals/architecture.md | 234 |
1 files changed, 232 insertions, 2 deletions
diff --git a/contributors/design-proposals/architecture.md b/contributors/design-proposals/architecture.md index 5cadd046..75d00902 100644 --- a/contributors/design-proposals/architecture.md +++ b/contributors/design-proposals/architecture.md @@ -1,6 +1,236 @@ -# Kubernetes architecture +# Kubernetes Design and Architecture -Please see the [Kubernetes design overview](README.md). +## Overview + +Kubernetes is production-grade, open-source infrastructure for the deployment, scaling, +management, and composition of application containers across clusters of hosts, inspired +by [previous work at Google](https://research.google.com/pubs/pub44843.html). Kubernetes +is more than just a “container orchestrator”. It aims to eliminate the burden of orchestrating +physical/virtual compute, network, and storage infrastructure, and enable application operators +and developers to focus entirely on container-centric primitives for self-service operation. +Kubernetes also provides a stable, portable foundation (a platform) for building customized +workflows and higher-level automation. + +Kubernetes is primarily targeted at applications composed of multiple containers. It therefore +groups containers using *pods* and *labels* into tightly coupled and loosely coupled formations +for easy management and discovery. + +## Scope + +Kubernetes is a [platform for deploying and managing containers] +(https://kubernetes.io/docs/whatisk8s/). Kubernetes provides a container runtime, container +orchestration, container-centric infrastructure orchestration, self-healing mechanisms such as health checking and re-scheduling, and service discovery and load balancing. + +Kubernetes aspires to be an extensible, pluggable, building-block OSS +platform and toolkit. Therefore, architecturally, we want Kubernetes to be built +as a collection of pluggable components and layers, with the ability to use +alternative schedulers, controllers, storage systems, and distribution +mechanisms, and we're evolving its current code in that direction. Furthermore, +we want others to be able to extend Kubernetes functionality, such as with +higher-level PaaS functionality or multi-cluster layers, without modification of +core Kubernetes source. Therefore, its API isn't just (or even necessarily +mainly) targeted at end users, but at tool and extension developers. Its APIs +are intended to serve as the foundation for an open ecosystem of tools, +automation systems, and higher-level API layers. Consequently, there are no +"internal" inter-component APIs. All APIs are visible and available, including +the APIs used by the scheduler, the node controller, the replication-controller +manager, Kubelet's API, etc. There's no glass to break -- in order to handle +more complex use cases, one can just access the lower-level APIs in a fully +transparent, composable manner. + +## Goals + +The project is committed to the following (aspirational) [design ideals](principles.md): +* _Portable_. Kubernetes runs everywhere -- public cloud, private cloud, bare metal, laptop -- + with consistent behavior so that applications and tools are portable throughout the ecosystem + as well as between development and production environments. +* _General-purpose_. Kubernetes should run all major categories of workloads to enable you to run + all of your workloads on a single infrastructure, stateless and stateful, microservices and + monoliths, services and batch, greenfield and legacy. +* _Meet users partway_. Kubernetes doesn’t just cater to purely greenfield cloud-native + applications, nor does it meet all users where they are. It focuses on deployment and management + of microservices and cloud-native applications, but provides some mechanisms to facilitate + migration of monolithic and legacy applications. +* _Flexible_. Kubernetes functionality can be consumed a la carte and (in most cases) Kubernetes + does not prevent you from using your own solutions in lieu of built-in functionality. +* _Extensible_. Kubernetes enables you to integrate it into your environment and to add the + additional capabilities you need, by exposing the same interfaces used by built-in + functionality. +* _Automatable_. Kubernetes aims to dramatically reduce the burden of manual operations. It + supports both declarative control by specifying users’ desired intent via its API, as well as + imperative control to support higher-level orchestration and automation. The declarative + approach is key to the system’s self-healing and autonomic capabilities. +* _Advance the state of the art_. While Kubernetes intends to support non-cloud-native + applications, it also aspires to advance the cloud-native and DevOps state of the art, such as + in the [participation of applications in their own management] + (http://blog.kubernetes.io/2016/09/cloud-native-application-interfaces.html). However, in doing + so, we strive not to force applications to lock themselves into Kubernetes APIs, which is, for + example, why we prefer configuration over convention in the [downward API] + (https://kubernetes.io/docs/user-guide/downward-api/). Additionally, Kubernetes is not bound by + the lowest common denominator of systems upon which it depends, such as container runtimes and + cloud providers. An example where we pushed the envelope of what was achievable was in its [IP + per Pod networking model](https://kubernetes.io/docs/admin/networking/#kubernetes-model). + +## Architecture + +A running Kubernetes cluster contains node agents (kubelet) and a cluster control plane (AKA +*master*), with cluster state backed by a distributed storage system +([etcd](https://github.com/coreos/etcd)). + +### Cluster control plane (AKA *master*) + +The Kubernetes [control plane](https://en.wikipedia.org/wiki/Control_plane) is split +into a set of components, which can all run on a single *master* node, or can be replicated +in order to support high-availability clusters, or can even be run on Kubernetes itself (AKA +[self-hosted](self-hosted-kubernetes.md#what-is-self-hosted)). + +Kubernetes provides a REST API supporting primarily CRUD operations on (mostly) persistent resources as the nucleus of its control plane. Kubernetes’s API provides IaaS-like +container-centric primitives such as [Pods](https://kubernetes.io/docs/user-guide/pods/), +[Services](https://kubernetes.io/docs/user-guide/services/), and [Ingress] +(https://kubernetes.io/docs/user-guide/ingress/), and also lifecycle APIs to support orchestration +(self-healing, scaling, updates, termination) of common types of workloads, such as [ReplicaSet] +(https://kubernetes.io/docs/user-guide/replicasets/) (simple fungible/stateless app manager), +[Deployment](https://kubernetes.io/docs/user-guide/deployments/) (orchestrates updates of +stateless apps), [Job](https://kubernetes.io/docs/user-guide/jobs/) (batch), [CronJob] +(https://kubernetes.io/docs/user-guide/cron-jobs/) (cron), [DaemonSet] +(https://kubernetes.io/docs/admin/daemons/) (cluster services), and [StatefulSet] +(https://kubernetes.io/docs/concepts/abstractions/controllers/statefulsets/) (stateful apps). +We deliberately decoupled service naming/discovery and load balancing from application +implementation, since the latter is diverse and open-ended. + +Both user clients and components containing asynchronous controllers interact with the same API resources, which serve as coordination points, common intermediate representation, and shared state. Most resources contain metadata, including [labels](https://kubernetes.io/docs/user-guide/labels/) and [annotations](https://kubernetes.io/docs/user-guide/annotations/), fully elaborated desired state (spec), including default values, and observed state (status). + +Controllers work continuously to drive the actual state towards the desired state, while reporting back the currently observed state for users and for other controllers. + +While the controllers are [level-based] +(http://gengnosis.blogspot.com/2007/01/level-triggered-and-edge-triggered.html) to maximize fault +tolerance, they typically `watch` for changes to relevant resources in order to minimize reaction +latency and redundant work. This enables decentralized and decoupled +[choreography-like](https://en.wikipedia.org/wiki/Service_choreography) coordination without a +message bus. + +#### API Server + +The [API server](https://kubernetes.io/docs/admin/kube-apiserver/) serves up the +[Kubernetes API](https://kubernetes.io/docs/api/). It is intended to be a relatively simple +server, with most/all business logic implemented in separate components or in plug-ins. It mainly +processes REST operations, validates them, and updates the corresponding objects in `etcd` (and +perhaps eventually other stores). Note that, for a number of reasons, Kubernetes deliberately does +not support atomic transactions across multiple resources. + +Kubernetes cannot function without this basic API machinery, which includes: +* REST semantics, watch, durability and consistency guarantees, API versioning, defaulting, and + validation +* Built-in admission-control semantics, synchronous admission-control hooks, and asynchronous + resource initialization +* API registration and discovery + +Additionally, the API server acts as the gateway to the cluster. By definition, the API server +must be accessible by clients from outside the cluster, whereas the nodes, and certainly +containers, may not be. Clients authenticate the API server and also use it as a bastion and +proxy/tunnel to nodes and pods (and services). + +#### Cluster state store + +All persistent cluster state is stored in an instance of `etcd`. This provides a way to store +configuration data reliably. With `watch` support, coordinating components can be notified very +quickly of changes. + + +#### Controller-Manager Server + +Most other cluster-level functions are currently performed by a separate process, called the +[Controller Manager](https://kubernetes.io/docs/admin/kube-controller-manager/). It performs +both lifecycle functions (e.g., namespace creation and lifecycle, event garbage collection, +terminated-pod garbage collection, cascading-deletion garbage collection, node garbage collection) +and API business logic (e.g., scaling of pods controlled by a [ReplicaSet] +(https://kubernetes.io/docs/user-guide/replicasets/)). + +The application management and composition layer, providing self-healing, scaling, application lifecycle management, service discovery, routing, and service binding and provisioning. + +These functions may eventually be split into separate components to make them more easily +extended or replaced. + +#### Scheduler + + +Kubernetes enables users to ask a cluster to run a set of containers. The scheduler +component automatically chooses hosts to run those containers on. + +The scheduler watches for unscheduled pods and binds them to nodes via the `/binding` pod +subresource API, according to the availability of the requested resources, quality of service +requirements, affinity and anti-affinity specifications, and other constraints. + +Kubernetes supports user-provided schedulers and multiple concurrent cluster schedulers, +using the shared-state approach pioneered by [Omega] +(https://research.google.com/pubs/pub41684.html). In addition to the disadvantages of +pessimistic concurrency described by the Omega paper, [two-level scheduling models] +(http://mesos.berkeley.edu/mesos_tech_report.pdf) that hide information from the upper-level +schedulers need to implement all of the same features in the lower-level scheduler as required by +all upper-layer schedulers in order to ensure that their scheduling requests can be satisfied by +available desired resources. + + +### The Kubernetes Node + +The Kubernetes node has the services necessary to run application containers and +be managed from the master systems. + +Each node runs a container runtime (like docker, rkt, or Hyper). The container +runtime is responsible for downloading images and running containers. + +#### Kubelet + + +The most important and most prominent controller in Kubernetes is the Kubelet, which is the +primary implementer of the Pod and Node APIs that drive the container execution layer. Without +these APIs, Kubernetes would just be a CRUD-oriented REST application framework backed by a +key-value store (and perhaps the API machinery will eventually be spun out as an independent +project). + +Kubernetes executes isolated application containers as its default, native mode of execution, as +opposed to processes and traditional operating-system packages. Not only are application +containers isolated from each other, but they are also isolated from the hosts on which they +execute, which is critical to decoupling management of individual applications from each other and +from management of the underlying cluster physical/virtual infrastructure. + +Kubernetes provides [Pods](https://kubernetes.io/docs/user-guide/pods/) that can host multiple +containers and storage volumes as its fundamental execution primitive in order to facilitate +packaging a single application per container, decoupling deployment-time concerns from build-time +concerns, and migration from physical/virtual machines. The Pod primitive is key to glean the +[primary benefits](https://kubernetes.io/docs/whatisk8s/#why-containers) of deployment on modern +cloud platforms, such as Kubernetes. + +#### Container runtime + +TODO + +#### `kube-proxy` + +The [service](https://kubernetes.io/docs/user-guide/services/) abstraction provides a way to +group pods under a common access policy (e.g., load-balanced). The implementation of this creates +A virtual IP which clients can access and which is transparently proxied to the pods in a Service. +Each node runs a [kube-proxy](https://kubernetes.io/docs/admin/kube-proxy/) process which programs +`iptables` rules to trap access to service IPs and redirect them to the correct backends. This provides a highly-available load-balancing solution with low performance overhead by balancing +client traffic from a node on that same node. + +Service endpoints are found primarily via [DNS](https://kubernetes.io/docs/admin/dns/). + +### Add-ons and other dependencies + +A number of components, called [*add-ons*] +(https://github.com/kubernetes/kubernetes/tree/master/cluster/addons) typically run on Kubernetes +itself: +* [DNS](https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns) +* [Ingress controller](https://github.com/kubernetes/ingress/tree/master/controllers) +* [Heapster](https://github.com/kubernetes/heapster/) (resource monitoring) +* [Dashboard](https://github.com/kubernetes/dashboard/) (GUI) + +### Federation + +A single Kubernetes cluster may span multiple availability zones. + +However, for the highest availability, we recommend using [cluster federation](federation.md). <!-- BEGIN MUNGE: GENERATED_ANALYTICS --> []() |
