Best practices to configure a Multi-Cloud Kubernetes cluster

Best practices to configure a Multi-Cloud Kubernetes cluster Deploy • Emmanuelle Demompion • 18/03/22 • 9 min read

Scaleway participated in the last Devoxx Poland to host a conference called Introduction to Kubernetes, and a workshop on the best practices to configure a Kubernetes cluster.

This article aims to make the full content of the workshop available for everyone.

We will be using a Kubernetes Kosmos cluster, presented in this introductory article , and deep-diving into specific concepts introduced in this second article .

Before starting

Technical prerequisites to follow this hands-on workshop

Basic knowledge of Kubernetes

A valid Scaleway account

A Scaleway secret key

A valid user account on another Cloud provider of your choice. Here we chose to use an instance from Hetzner Cloud.

A valid token or secret key from the external Cloud provider you chose.

⚠️ Warning

To avoid adding more complexity to this very technical content, we will use root or default users provided by our Cloud providers.

In a production context, it is highly recommended to create specific secured users on your infrastructure's servers.

We will balance between concept explanations and operations or commands.

If this icon (🔥) is present before an image, a command, or a file, you are required to perform an action.

So remember, when 🔥 is on, so are you. Let's get started!

Cluster inventory

For the purpose of this workshop, we are going to use a Scaleway Kubernetes Kosmos cluster, which allows instances from multiple Cloud providers to be managed by a single Scaleway control plane.

Kubernetes Kosmos architecture

The Kubernetes Kosmos cluster we are going to build and work on will have the following characteristics:

Its control plane will be located in the Paris region (fr-par)

It will include a Scaleway node pool with one to ten Instances located in the Amsterdam Availability Zone (nl-ams1) (with auto-scaling and auto-healing features)

It will include a Multi-Cloud node pool containing:

One Scaleway unmanaged Instance located in the Warsaw Availability Zone (pl-waw1); meaning that this Instance will be created independently from the cluster itself and will be attached to it as if it came from another Cloud provider.

One Hetzner Instance located in Helsinki .

Creation of our cluster and nodes

Cluster creation

To start with this workshop, we need to log in to our Scaleway account and create a managed Kubernetes cluster.

🔥 Select Kubernetes Kosmos as the type of cluster you want to create. You can choose to locate the control plane of your cluster in any supported region: Paris (fr-par), Amsterdam (nl-ams), or Warsaw (pl-waw).

Here we are choosing to locate our control plane in Paris, but any region is compatible. Also, we decide to set up our cluster with the latest version of Kubernetes available at the time of writing: k8s 1.22.

🔥 Click the Create a cluster button, as we will add and configure node pools separately later on.

Our Kubernetes Kosmos control plane is being created. Once available, we are directed to its overview tab.

Kubernetes Kosmos cluster overview

Scaleway pool creation

🔥 To be functional, our cluster requires at least one node pool.

So, we need to go on the Pool listing page to add a new pool of nodes to our cluster.

Kubernetes Kosmos cluster pools list

We will start by creating the first pool of type Scaleway.

When in a Kubernetes Kosmos cluster, Scaleway pools can be created in any Availability Zone, whatever the region of the control plane.

🔥 Here, even if we previously chose our cluster control plane to be in Paris (fr-par) region, we can still decide to create a managed Scaleway pool in the Amsterdam 1 (nl-ams1) Availability Zone.

Scaleway pools have the advantage of being fully managed by the Kubernetes Kosmos control plane, meaning that the auto-scaling and auto-healing features can be activated. In this workshop, we decided to allow the auto-scaling of our Amsterdam pool to scale from one to ten Scaleway Instances.

Kubernetes Kosmos Scaleway pool creation interface

Once created, our pool is visible in our listing view, showing its zone and a few of its configuration parameters. We can also see that a node is already available in our Kubernetes cluster.

Multi-Cloud pool creation

🔥 Now, we need to add a new pool of type "Multi-Cloud". No configuration is needed as it will in any case only allow us to attach unmanaged and external nodes to our Kubernetes cluster.

For more visibility, we set the pool name to "worldwide", as we intend to have instances from all over the world in our cluster.

Kubernetes Kosmos Multi-Cloud pool creation interface

🔥 To attach instances and/or servers to a Kubernetes Kosmos Multi-Cloud pool, we will need the Pool ID, which we can get by clicking More info in the drop-down menu. Copy the ID and save it somewhere safe for later.

Kubernetes Kosmos Multi-Cloud pool information

Unmanaged Instances creation

Scaleway Instance creation

As we intend to create a worldwide Kubernetes cluster, we are going to create an unmanaged Scaleway Instance from the Scaleway Console.

Scaleway Instance list view

🔥 Let's create the instance in Warsaw 1 Availability Zone.

⚠️ Warning: At the time of writing, the Instance must run Ubuntu 20.04 to function correctly in a Kubernetes Kosmos cluster.

Scaleway instance creation page

Now that our Warsaw Instance is created, we can connect to it via ssh . The command to use is shown on the Instance Overview page.

Attach our external Warsaw Instance to our Kubernetes Kosmos cluster

🔥 Connect to our Instance

Execute the following command to connect to your Instance. Make sure you replace the IP address with the correct one for your server. You can check this in the "Public IP" information field or, copy-paste the complete ssh command directly from the "SSH command" information field.

ssh root@151.115.36.196

🔥 Configure our Warsaw instance

To attach our Instance to our "worldwide" Multi-Cloud pool, we need to complete a series of steps.

Remember that for this section, we need to be connected to our unmanaged Scaleway instance in Warsaw. Connection instructions were described in the previous paragraph.

As we will need some parameters to attach our Instance, we are going to set them as environment variables.

export POOL_ID=19d98397-cda3-446c-893d-b74f6e3c5d56

export POOL_REGION=fr-par

export SCW_SECRET_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

The…