Kubernetes Networking Setup
Kubernetes Operations (kops) currently supports 4 networking modes:
kubenetKubernetes native networking via a CNI plugin. This is the default.cniContainer Network Interface(CNI) style networking, often installed via a Daemonset.classicKubernetes native networking, done in-process.externalnetworking is done via a Daemonset. This is used in some custom implementations.
kops Default Networking
Kubernetes Operations (kops) uses kubenet networking by default. This sets up networking on AWS using VPC networking, where the master allocates a /24 CIDR to each Node, drawing from the Node network. Using kubenet mode routes for each node are then configured in the AWS VPC routing tables.
One important limitation when using kubenet networking is that an AWS routing table cannot have more than 50 entries, which sets a limit of 50 nodes per cluster. AWS support will sometimes raise the limit to 100, but their documentation notes that routing tables over 50 may take a performance hit.
Supported CNI Networking
Container Network Interface provides a specification and libraries for writing plugins to configure network interfaces in Linux containers. Kubernetes has built in support for CNI networking components.
Several different CNI providers are currently built into kops:
- Calico
- Canal (Flannel + Calico)
- flannel
- kopeio-vxlan
- kube-router
- romana
- weave
- amazon-vpc-routed-eni
Specifying network option for cluster creation
The --networking option accepts the three different values defined above: kubenet, cni, classic, and external. If --networking is left undefined kubenet is installed.
kops create cluster --networking kopeio-vxlan
Switching between networking providers
kops edit cluster and you will see a block like:
networking:
classic: {}
That means you are running with classic networking. The {} means there are no configuration options, beyond the setting classic.
To switch to kubenet, change the word classic to kubenet.
networking:
kubenet: {}
Now follow the normal update / rolling-update procedure:
kops update cluster # to preview
kops update cluster --yes # to apply
kops rolling-update cluster # to preview the rolling-update
kops rolling-update cluster --yes # to roll all your instances
Your cluster should be ready in a few minutes. It is not trivial to see that this has worked; the easiest way seems to be to SSH to the master and verify that kubelet has been run with --network-plugin=kubenet.
Switching from kubenet to a CNI network provider has not been tested at this time.