Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 44 additions & 4 deletions source/plugins/cloudstack-kubernetes-service.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ CloudStack Kubernetes Service

The Kubernetes Service plugin adds Kubernetes integration to CloudStack. The plugin is disabled by default and an admin can enable it using a Global Setting. It enables users to run containerized services using Kubernetes clusters.

With CoreOS having reached EOL, from 4.16 on the Kubernetes Service Plugin will use
the existing SystemVM Template by default for deploying kubernetes clusters. For
The Kubernetes Service Plugin will use the existing SystemVM Template by default for deploying kubernetes clusters. For
installation of Kubernetes binaries on the cluster nodes, a binaries ISO is used for each
Kubernetes version to be made available via CloudStack. This allows faster, offline
installation of Kubernetes binaries and docker images along with support for adding
Expand All @@ -26,7 +25,11 @@ multiple versions of Kubernetes for upgrades and running different clusters.
From version 4.21.0, users can choose different templates and service offerings for different types of nodes (worker, control, etcd nodes) for deploying Kubernetes clusters. The templates must be previously registered selecting the 'For CKS' option.
See :ref:`flexible-kubernetes-clusters`.

For deployment and setup of Kubernetes on cluster nodes, the plugin uses the Kubernetes tool, 'kubeadm'. kubeadm is the command-line tool for easily provisioning a secure Kubernetes cluster on top of physical or cloud servers or Instances. Under the hood, control node(s) of the cluster starts a Kubernetes cluster using kubeadm init command with a custom token, and worker nodes join this Kubernetes cluster using kubeadm join command with the same token. More about kubeadm here: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/. Weave Net CNI provider plugin is used for cluster networking. More about Weave Net provide plugin here: https://www.weave.works/docs/net/latest/kubernetes/kube-addon/.
For deployment and setup of Kubernetes on cluster nodes, the plugin uses the Kubernetes tool, 'kubeadm'. kubeadm is the command-line tool for easily provisioning a secure Kubernetes cluster on top of physical or cloud servers or Instances. Under the hood, control node(s) of the cluster starts a Kubernetes cluster using kubeadm init command with a custom token, and worker nodes join this Kubernetes cluster using kubeadm join command with the same token. More about kubeadm here: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/.

Weave Net CNI provider plugin is used for cluster networking. More about Weave Net provide plugin here: https://www.weave.works/docs/net/latest/kubernetes/kube-addon/.

Calico CNI provider plugin is also supported from ACS 4.21 onwards. More about Calico CNI plugin here: https://docs.projectcalico.org/getting-started/kubernetes/.

To access the Kubernetes dashboard securely, the plugin provides access to kubeconfig file data which uses the Kubernetes tool kubectl to run a local proxy and thereby access the dashboard. More about kubectl here: https://kubernetes.io/docs/reference/kubectl/overview/

Expand Down Expand Up @@ -76,6 +79,7 @@ Eg: To generate the latest kubernetes iso
1.3.0, CNI version, see https://github.com/containernetworking/plugins/releases
1.27.0, cri-tools version, see https://github.com/kubernetes-sigs/cri-tools/releases
1.11, weave addon for kubernetes, see https://github.com/weaveworks/weave/tree/master/prog/weave-kube
3.30.0, calico addon for kubernetes, see https://raw.githubusercontent.com/projectcalico/calico/v3.30.0/manifests/calico.yaml
2.7.0, kubernetes dashboard version, see https://github.com/kubernetes/dashboard/release

Usage:
Expand All @@ -84,12 +88,25 @@ Usage:

# ./create-kubernetes-binaries-iso.sh OUTPUT_PATH KUBERNETES_VERSION CNI_VERSION CRICTL_VERSION WEAVENET_NETWORK_YAML_CONFIG DASHBOARD_YAML_CONFIG [OPTIONAL_OUTPUT_FILENAME] [OPTIONAL_ETCD_VERSION]

Eg:
Eg: To generate the kubernetes iso with weave cni plugin

.. parsed-literal::

# ./create-kubernetes-binaries-iso.sh ./ 1.27.2 1.3.0 1.27.0 https://raw.githubusercontent.com/weaveworks/weave/master/prog/weave-kube/weave-daemonset-k8s-1.11.yaml https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml setup-v1.27.2

Eg: To generate the kubernetes iso with calico cni plugin

.. parsed-literal::

# ./create-kubernetes-binaries-iso.sh ./ 1.33.1 1.7.1 1.33.0 https://raw.githubusercontent.com/projectcalico/calico/v3.30.0/manifests/calico.yaml https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml setup-v1.33.1-calico

Eg: To generate the kubernetes iso with calico cni plugin for ARM64 architecture add aarch64 as the last parameter.

.. parsed-literal::

# ./create-kubernetes-binaries-iso.sh ./ 1.33.1 1.7.1 1.33.0 https://raw.githubusercontent.com/projectcalico/calico/v3.30.0/manifests/calico.yaml https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml aarch64 setup-v1.33.1-calico-arm64


**NOTE:**
From ACS 4.16 onwards, Kubernetes versions >= 1.20.x are only supported (https://endoflife.date/kubernetes).

Expand Down Expand Up @@ -765,6 +782,29 @@ For verification of the applied CNI Configuration, the following commands can be

Events: <none>

There could be Calico routing edge case encountered in some environments. By default, Calico uses the 192.168.0.0/16 network for its pod IP pool when you install it with the standard manifests. To avoid potential routing conflicts with existing networks in your infrastructure, it's advisable to customize the Calico IP pool to use a different subnet that doesn't overlap with your current network setup.

kubectl get ippool.crd.projectcalico.org -o yaml

.. code-block:: bash
apiVersion: crd.projectcalico.org/v1
kind: IPPool
metadata:
name: default-ipv4-ippool
spec:
cidr: 192.168.0.0/16
ipipMode: Always
natOutgoing: true
disabled: false

You can edit the IP pool to change the CIDR to a different subnet that fits your network architecture better. For example, you might choose to use 10.0.0.0/16.

kubectl edit ippool default-ipv4-ippool

and redeploy the pods

kubectl delete pod --all -A


.. |cks-add-version-form.png| image:: /_static/images/cks-add-version-form.png
:alt: Add Kubernetes Supported Version form.
Expand Down