Using Kubernetes with Juju

Kubernetes ("k8s") provides a flexible architecture for managing containerised applications at scale. See the Kubernetes documentation for more information.

The objective of this page is to give an overview of how an existing Kubernetes cluster can be integrated with Juju and what the general workflow is once that's done. Links will be provided at the end to theoretical and practical material. Finally, although this page is not about showing how to install Kubernetes itself, we do give pointers on how to do so.

Juju Kubernetes-specific workflow

Essentially, Juju is able to treat the added cluster as it does any other of its known clouds (i.e. create models and deploy charms). There are some differences to working with such a cloud and they are called out in this section.

The k8s-specific Juju commands are add-k8s, remove-k8s, and scale-application. All other concepts and commands are applied in the traditional manner.

The add-k8s command is used to add the Kubernetes cluster to Juju's list of known clouds and import its credentials. The cluster configuration file will first need to be copied to ~/.kube/config. This command makes the usual combination of add-cloud and add-credential unnecessary.

User credentials can still be added by way of the add-credential or autoload-credentials commands. Also, at any time, the k8s CLI can be used to add a new user to the cluster.

The add-k8s command can be used repeatedly to set up different clusters as long as the contents of the configuration file has been changed accordingly. The KUBECONFIG environment variable is useful here as it will be honoured by Juju when finding the file to load.

The remove-k8s command is used to remove a Kubernetes cluster from Juju's list of known clouds.

The scale-application command is used to scale a Kubernetes cluster. The add-unit and remove-unit commands do not apply to a Kubernetes model.

Charms need to be written specifically for a Juju:Kubernetes context.

Running Kubernetes workloads

First off, a Kubernetes cluster will be required. Essentially, you will use it as you would any other cloud that Juju interacts with: the cluster becomes the backing cloud.

The following steps describe the general approach for using Kubernetes with Juju:

1. Obtain a Kubernetes cluster
2. Add the cluster to Juju
3. Add a model
4. Create persistent storage (if necessary)
5. Create storage pools
6. Deploy a Kubernetes charm

Obtain a Kubernetes cluster

There are many ways to obtain a Kubernetes cluster. Here is a list of suggestions:

• Use the 'kubernetes-core' bundle, which gives a minimal two-machine cluster available in the Charm Store.
• Use the 'canonical-kubernetes' bundle. This is the Canonical Distribution of Kubernetes (CDK), which is a more sophisticated version of 'kubernetes-core'.
• Use the conjure-up installer.
• Use MicroK8s. This gives you a local, fully compliant Kubernetes deployment with dynamic persistent volume support.
• When Kubernetes is deployed via charms, special integrator charms made for specific cloud vendors can greatly assist (e.g. storage). Search the Charm Store for 'integrator'.
• Use a public Kubernetes cloud vendor such as Amazon EKS, Azure AKS, and Google GKE.

Add the cluster to Juju

Information about the cluster is needed in order to add it to Juju. This is found within the main Kubernetes configuration file that can be copied over from the Kubernetes master node (and saved as ~/.kube/config). We can then take advantage of the add-k8s command as it will parse the configuration file if copied to the above path. This allows us to quickly add the cluster.

Note that the conjure-up installer adds the cluster for you.

Add a model

Add a model in the usual way, with the add-model command. This will cause a Kubernetes namespace in the cluster to be created that will host all of the pods and other resources for that model. The namespace is the name of the Juju model. A Kubernetes Juju model also starts off with a storage pool called 'kubernetes'.

Create persistent storage

Create persistent static volumes for operator storage if your chosen backing cloud's storage is not supported natively by Kubernetes. You will need to do the same for charm storage if your charm has storage requirements. This is done with the Kubernetes tool kubectl.

Create storage pools

Create storage pools for operator storage and, if needed, charm storage. This is done in the usual way, with the create-storage-pool command.

Deploy a Kubernetes charm

A Kubernetes-specific charm is deployed in standard fashion, with the deploy command. If the charm has storage requirements you will need to specify them, as you do with a normal charm.

Configuration

The below table lists configuration keys supported by Kubernetes charms that are set at deploy time. The corresponding Kubernetes meaning can be obtained from the Kubernetes documentation for Services and Ingress.

For example:

juju deploy mariadb-k8s --config kubernetes-service-loadbalancer-ip=10.1.1.1

There are two other keys that are not Kubernetes-specific:

Keys 'juju-external-hostname' and 'juju-application-path' control how the application is exposed externally using a Kubernetes Ingress Resource in conjunction with the configured ingress controller (default: nginx).

Storage theory and practical guides

The Persistent storage and Kubernetes page provides the theory on how Juju works with Kubernetes storage.

The following practical guides are available:

• The conjure-up installer can be used to install Kubernetes. See the following resources for guidance:
  • The Ubuntu tutorial: Install Kubernetes with conjure-up
  • The upstream getting started guide: Spell Walkthrough
• Tutorial Setting up static Kubernetes storage shows how to set up statically provisioned persistent volumes with Juju by way of the 'kubernetes-core' charm.
• Tutorial Using Juju with MicroK8s provides steps for getting started with Juju and MicroK8s.
• Tutorial Using the aws-integrator charm demonstrates deploying Kubernetes with Juju on AWS with an integrator charm.