Kubernetes code read -apiserver basic articles

Apiserver is the core module of kubernetes, do more things, the amount of code is also larger. The market has a lot of apiserver code interpretation of the article, but the problem is that because k8s code changes quickly, want to write a long-term can not be used to do. So, I refer to the "Kubernetes authoritative guide" and Zhejiang University SEL laboratory of some articles, first I see things written down, to be useful after the view.

Kubernetes source code version 1.2.0

Code reading method

Simply talk about the entire code directory structure

Directory | description |
| ———– | ————————————- — |
Api | output interface documentation
Build | build script
| Cluster | Adapt to different I layers of clouds, such as Amazon AWS, Microsoft Azure, Google GCE cluster startup script |
| Cmd | all binary executable file entry code, such as apiserver / scheduler / kubelet |
| Docs | Documentation, including user documentation, administrator documentation, design, new features offer |
Example | use case |
| Godeps | project dependent on the use of Go third party packages, such as docker client SDK, rest, etc. |
Hacking toolbox, all kinds of compiling, building, testing, checking scripts are here
| Hooks | git script before and after committing
| Pkg | project code home directory, cmd is just an entry, here is all the concrete implementation |
Plugin, plugin, k8s think scheduler is part of plugin, so scheduler code here |
| Release | should i be used in Google |
Test – related tools
Third_party | some third party tools should not be strongly dependent? |
| Www | UI, but has been moved to the new project

You can see that the key implementation code is placed in the pkg directory. For apiserver this wide range of components, the only effective way to read is

  1. Traverse all the directories under pkg, the overview probably knows what the directory is Gansha
  2. From the entrance to the point of view apiserver code, and then a little bit from shallow to see the general realization of apiserver
  3. Sub-characteristics, to see how a particular feature is achieved, such as security, such as and etcd storage docking
  4. In the process of the above steps can look at other people's code to read the document, can effectively save time

0. What does apiserver do?

Apiserver is the k8s system in all the objects of the additions and deletions change http / restful server, which stare refers to the watch operation. The data is finally stored in a distributed consistent etcd store, and the apiserver itself is stateless, providing a range of features such as authentication access, cache, and api version adaptation for data access.

  • Restful service entry

For the http service and the use of go language to achieve the way, you can see go-restful documents and examples of this basic understanding of this document on the beginners and know what is extremely effective!

1. The data structure of the object

The ancients have words, the program is the algorithm data structure, understand the data structure, the whole process of processing to understand the half. For any apis request for apiserver, the figure above illustrates all data structure relationships.

K8s put in etcd storage object is api.Pod object (no version), from different versions of the request path to identify the operation, such as api/v1 , and finally get a different version, such as v1.Pod json text. Here are a few steps, including

  1. Http client access / api / v1 / pod / xyz, want to get this pod data
  2. Obtain the api.Pod object from etcd
  3. The api.Pod object is converted to a v1.Pod object
  4. V1.Pod objects are serialized to json or yaml text
  5. Text through the http response body, return to the http client

Which is used to deal with business data key data structure is APIGroupVersion, inside the role of several members of the variable is:

Members of the role
| ———— | ———————————— —- |
| GroupVersion | Contains a string of api/v1 that identifies this instance
Serializer and object – serializer
| Converter | This is a powerful data structure, put here is an interface, ontology /pkg/conversion/conversion.go , almost any object can be converted to another, as long as you put in advance the corresponding conversion function |
| Storage | The map of the key, for the object url, value is a rest.Storage structure for docking etcd storage, in the initial registration, the map will be opened, into a real rest service to the storage of a Dragon Service

2. Entrance and start

File | main data structure / function | use |
| —————————————- | ——– ———— | ——————- |
Kubernetes / cmd / kube – apiserver / apiserver.go |
| Kubernetes / cmd / kube-apiserver / app / options / options.go | struct APIServer | startup options |
| Kubernetes / cmd / kube-apiserver / apiserver.go | func Run | initialize some clients, start master objects |
| Kubernetes / pkg / genericapiserver / genericapiserver.go | func Run | start safe and non-secure http service |

3. API grouping, multi-version initialization registration (Rest)

K8s use ApiGroup to manage all api groups and version upgrades, and some API groups currently include

  1. Core group, REST path in /api/v1 , but this path is not fixed, v1 is the current version. Corresponding to the code inside the apiVersion field value is v1 .
  2. Extended group, the apiVersion: extensions/$VERSION (for example, the current apiVersion: extensions/v1beta1 ) in the /apis/extensions/$VERSION , corresponding code. The API objects provided here may be moved to other groups in the future.
  3. "Componentconfig" and "metrics" for these groups.

In this document which describes the realization of ApiGroup several goals, including api packet evolution, the legacy API backwards compatibility (Backwards compatibility), including the user can customize their own api and so on. Next we look at how it is initialized registration, here are the reduced version of the code, removed the other parts.


  • Api registered entrance

func New(c *Config) (*Master, error) {

– Add group information to Config in APIGroupsInfo according to Config and register with InstallAPIGroups

func (m *Master) InstallAPIs(c *Config) {
if err := m.InstallAPIGroups(apiGroupsInfo); err != nil {
glog.Fatalf("Error in registering group versions: %v", err)

  • Convert to APIGroupVersion this key data structure and then register it
  Func (s * GenericAPIServer) installAPIGroup (apiGroupInfo * APIGroupInfo) error { 
ApiGroupVersion, err: = s.getAPIGroupVersion (apiGroupInfo, groupVersion, apiPrefix)

If err: = apiGroupVersion.InstallREST (s.HandlerContainer); err! = Nil {
Return fmt.Errorf ("Unable to setup API% v:% v", apiGroupInfo, err)

  • Key data structures

  Kubernetes / pkg / apiserver / apiserver.go 

  Type APIGroupVersion struct { 
Storage map [string] rest.Storage

Root string

// GroupVersion is the external group version
GroupVersion unversioned.GroupVersion

The actual registered Storage map is as follows:


  M.v1ResourcesStorage = map [string] rest.Storage { 
"Pods": podStorage.Pod,
"Pods / attach": podStorage.Attach,
"Pods / status": podStorage.Status,
"Pods / log": podStorage.Log,
"Pods / exec": podStorage.Exec,
"Pods / portforward": podStorage.PortForward,
"Pods / proxy": podStorage.Proxy,
"Pods / binding": podStorage.Binding,
"Bindings": podStorage.Binding,

So, where the map[string]rest.Storage Finally how to become a specific API to provide services? Such as such a URL:

GET /api/v1/namespaces/{namespace}/pods/{name}

  • Restful service implementation

K8s use a third-party library github.com/emicklei/go-restful , which provides a set of core objects, see examples

Data structure | function | position within k8s
| —————— | —————————— ———- | ————————————— – |
| Restful.Container | represents a http rest service object, including a set of restful.WebService | genericapiserver.go – GenericAPIServer.HandlerContainer |
| Restful.WebService | consists of multiple restful.Route, handle all of the special MIME types under these paths | api_installer.go – NewWebService () |
| Restful.Route | path – handler function map map | api_installer.go – registerResourceHandlers () |

  • Actual registration process


func (a *APIInstaller) registerResourceHandlers(path string, storage rest.Storage, ws *restful.WebService, proxyHandler http.Handler) (*unversioned.APIResource, error) {

The final API registration process is done in this function, a rest.Storage object into the actual getter, lister and other processing functions, and the actual url associated.

4.etcd stored operation (ORM)

The above has been basically clarified from the http request -> restful.Route -> rest.Storage this line, that rest.Storage is just an interface, what Germany can, can really operate etcd?

This is also involved in a number of documents, but also relatively clear, first of all, all the objects have to delete these operations, if the Pod alone engage in a set, Controller alone set a set, that code will be very repeated, , So the key directory for storage is here:


This file defines all the operations on the etcd object, get, list, create, etc., but the specific object is what, this file does not care; etcd client address, this file does not care. This information is injected when a specific PodStorage object is created. Take Pod as an example, the file is:


Here the NewStorage method, the above information into the etcd inside to generate the PodStorage this object.

// REST implements a RESTStorage for pods against etcd
type REST struct {
proxyTransport http.RoundTripper

Because PodStorage.Pod is a REST type, and REST type using the Go language struct anonymous internal members, naturally have Get, List and other methods.


Finally, here PodStorage converted into a Getter object, and eventually registered to the ApiGroup inside to go.

Heads up! This alert needs your attention, but it's not super important.