Advanced Kubernetes Kapsule features you might have missed!

Advanced Kubernetes Kapsule features you might have missed! Deploy • Louis Moreau • 31/05/22 • 4 min read

The following features are used extensively by our advanced users. We thought you might be interested in going further and optimizing your use of Kubernetes Kapsule. All features described here are available via our REST API and via the Scaleway CLI .

Feature Gates & Admission Controllers

Feature Gates allow you to enable some features that are still considered Alpha in Kubernetes. They provide users with the possibility of testing new Kubernetes features without waiting for them to graduate to Beta.

Admission Controllers control every request that reaches the API-Server, and grant or deny access to the cluster workers. They can limit creation, deletion, modification, or connection rights. By setting them up on your cluster, you define the behaviour of every Kubernetes object that will run on it, and the security strategies to apply, regardless of pods or containers.

For instance, you can apply the AlwaysPullImages controller to ensure that a deprecated local version of your container images is never used for your Kubernetes objects, and that images are always pulled from their registry.

There are many admission controllers available, and they all allow you to restrict usage and secure your Kubernetes environment.

Here is the following CLI command:

scw k8s cluster update xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \ feature-gates.0=EphemeralContainers admission-plugins.0=AlwaysPullImages CopyContentIcon Copy code You can see the list of Admission Controllers and Feature Gates available on your current version by calling the version API or with the CLI:

scw k8s version get CopyContentIcon Copy code Have a look at the complete Kubernetes documentation about Admission Controllers and at the complete Kubernetes documentation about Feature Gates .

If the Feature Gate or Admission Controller is not available, talk about enabling it with our engineers on the #k8s chan in our Slack Community .

OIDC support: an experimental feature

OpenID Connect is a version of OAuth2, supported by multiple providers and services, that allows the use of the same accounts and login details. The Kubernetes documentation about authentication and OIDC explains how this authentication method works (see documentation here ). Supporting it in a managed Kubernetes engine simply means that a JWT token generated by certain supported providers (or services) can be used to manage roles, login, and access your Kubernetes cluster via kubectl .

Google, Azure, Salesforce, and Github are examples of services and providers that support this feature. You can create a group in these services, with certain restricted access, and use the same login credentials to use a Kubernetes cluster. This list is not exhaustive.

To configure this feature, you can use the following command:

scw k8s cluster update xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \ open-id-connect-config.client-id= \ open-id-connect-config.issuer-url= CopyContentIcon Copy code In order to get a more user-friendly flow, you can check out some Open Source projects like kubelogin or k8s-oidc-helper .

Persistent Volumes At-Rest Encryption

This plugin supports the at-rest encryption of volumes with Cryptsetup/LUKS, allowing, once again, to make your data storage more secure.

We have dedicated documentation for the encryption of persistent volumes within a Scaleway Kubernetes Kapsule cluster on Scaleway’s CSI Github repository where you can find out how to use this feature. Please note, however, that it cannot handle volume resizing.

XFS support

Kubernetes Kapsule supports the XFS file system on block storage. You can define a Storage Class to use it, as described below:

allowVolumeExpansion: true apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: labels: name: scw -bssd-xfs provisioner: csi.scaleway.com reclaimPolicy: Delete volumeBindingMode: Immediate parameters: csi.storage.k8s.io/fstype: xfs CopyContentIcon Copy code Different pools runtimes available

Even if Kubernetes Kapsule only officially supports Docker runtime, you can also use containerd and cri-o .

Note that, containerd ships with gvisor and kata containers for untrusted workloads.

Currently, the default runtime is Docker. However, we are planning to switch this default runtime to containerd to, on the one hand, prepare the deprecation of dockershim and, on the other hand, to benefit from containerd's simplicity and its lightweight implementation.

To redefine the runtime at pool level with the Scaleway CLI, you can use this command:

scw k8s pool create cluster-id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \ size=1 name=test node-type=DEV1-M container-runtime=containerd CopyContentIcon Copy code You will also need a runtime class:

apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: kata handler: kata CopyContentIcon Copy code or

apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: gvisor handler: runsc CopyContentIcon Copy code At pod level, you need to use the associated handler: runtimeClassName: kata

Autoscaler configuration

The AutoScaling feature provided by Kubernetes Kapsule allows you to set a maximum of parameters to manage your costs and configure your cluster with all the flexibility you need.

If you need more information about the Autoscaler, please have a look at the FAQ

You have the possibility of:

disabling the autoscaler ( scale_down_disabled )

setting the scale down delay after a scale up ( scale_down_delay_after_add )

setting the resource estimator to base the scaling rules on ( estimator )

setting the type of node group expander to be used in a scale up ( expander )

ignoring DeamonSet pods when calculating resource utilization for scaling down ( ignore_daemonsets_utilization )

detecting similar node groups and balancing the number of nodes between them ( balance_similar_node_groups )

defining a cutoff policy for pods ( expendable_pods_priority_cutoff )

setting the time a node should be unneeded before it is eligible for scale down ( scale_down_unneeded_time )

Here is how you can setup the autoscaler with the Scaleway CLI:

scw k8s cluster update xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \ autoscaler-config.scale-down-delay-after-add=5m \ autoscaler-config.scale-down-unneeded-time=5m \ autoscaler-config.scale-down-utilization-threshold=0.7 CopyContentIcon Copy code Synchronization between Scaleway tags and k8s labels & taints

Tags…