How to measure the energy consumption of your applications on Kubernetes with Scaphandre

How to measure the energy consumption of your applications on Kubernetes with Scaphandre Deploy • Damien Vergnaud & Rémi Calizzano • 29/11/23 • 6 min read

The measurement of application energy consumption is a current and critical topic, both for raising awareness and for optimization purposes. In this article, we will introduce Scaphandre, an open-source tool addressing this measurement challenge, and its application in a Kubernetes Kosmos context.

Introduction to Scaphandre

Scaphandre is an open-source project licensed under Apache 2.0 that has gained popularity with over 1.3k stars on GitHub by the end of 2023. Created and maintained by @bpetit and other contributors , it is written in Rust.

The primary goal of Scaphandre is to foster collaboration between tech companies and enthusiasts around a simple, robust, lightweight, and clear method of measuring energy consumption for informed decision-making (source).

In addition to metric collection, one of Scaphandre's strengths is its ability to expose Prometheus metrics, seamlessly integrating with observability stacks. Here is the project's example dashboard :

Scaphandre project dashboard

On this dashboard, we visualize the electrical power consumed by the processes of the machine on which Scaphandre is installed, as well as its conversion into the quantity of energy consumed over time intervals.

In summary, Scaphandre offers the following features:

Retrieval of server consumption in watts through sensors.

Distribution of this consumption among processes running on the server.

Export of metrics on a port in Prometheus format (among others).

However, Scaphandre comes with some constraints, particularly related to implemented sensors:

Compatibility of sensors with host machines.

Scope of observed components (RAM/CPU) varies depending on sensors and used machines.

Sensors require access to system files in /proc/ and /sys/ . This access is typically possible only on machines you own, especially for /sys/, unless your cloud provider shares hypervisor metrics with instances through metric propagation .

Installation on the Kubernetes Cluster

To use Scaphandre in a Kubernetes context, you need a cluster that meets the mentioned constraints. We utilized the Scaleway Kosmos solution, enabling easy creation of a Kubernetes cluster with an Elastic Metal node for Scaphandre measurements.

Prerequisites:

In order to deploy Scaphandre in a Kubernetes Scaleway environment, you must have checked these prerequisites:

Have a Kosmos cluster

How to create a Kubernetes Kosmos cluster

Have one or more Elastic Metal nodes compatible with Scaphandre

Compatibility of the node with Scaphandre

How to create an Elastic Metal server

The Elastic Metal node(s) are linked to the Kosmos cluster by a multi-cloud node group

How to add a multi-cloud pool to your Kosmos cluster

Have a local environment for operations with:

Tools : Git, Helm, Means

Connectivity to the kube-api-server (kubeconfig),

Extended rights on the cluster.

Workarounds

For our POC, we used the Scaphandre development branch and performed manual actions to address open issues on the project:

Scaphandre Helm chart does not yet allow specifying additional labels on the ServiceMonitor object, causing an issue with the kube-prometheus-stack deployed in "default" mode.

Open pull request

Workaround: Add the label release: kube-prometheus-stack manually to the deployed object or in the Chart before deployment.

ServiceMonitor object manifest with the label release: kube-prometheus-stack

The --containers option of Scaphandre does not work in our context.

Goal: This option adds labels to Prometheus metrics to identify the origin of metrics in a Kubernetes context (namespace, node name, pod name).

Workaround: Await a fix for the issue to have the functionality working.

NodeAffinity is not yet supported in the current Chart.

Workaround: Optional, detailed in a subsequent paragraph.

Deploying the Kube Prometheus Stack

First of all, we need an observability stack installed in our cluster. We use the kube-prometheus-stack which makes it easy to have Prometheus and Grafana:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo update helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack --namespace observability CopyContentIcon Copy code Deploying Scaphandre

Let's start by deploying Scaphandre in our cluster using the project's Chart:

git clone https://github.com/hubblo-org/scaphandre git switch dev cd scaphandre helm install scaphandre helm/scaphandre \ --set serviceMonitor.enabled=true \ --set serviceMonitor.namespace=observability \ --set serviceMonitor.interval=30s \ --namespace observability CopyContentIcon Copy code Deploying an application to monitor

To carry out our tests, we need an application to measure its consumption, ideally with several pods to have nice visualizations. We chose to deploy the Google online boutique demo microservice application in the cluster.

[Optional] Deploy Scaphandre Only on Certain Nodes:

You might not need to deploy Scaphandre on all nodes, for e.g. :

for nodes not meeting prerequisites,

monitoring only selected nodes.

This chapter helps you deploy Scaphandre and your applications accordingly.

Using the same principles as described here , you can label the dedicated Scaleway node in the following way:

kubectl get nodes

Identify the nodes that meet the prerequisites, then apply a label:

kubectl label nodes scw-k8s-metal-pool-metal-{ID} powercap_rapl=true

This labeling allows you to condition the deployment of Scaphandre and your monitoring applications.

For Scaphandre, depending on the current version, you might need to additionally add node selection in the DaemonSet template of the Helm chart before deploying.

Change suggestion for DaemonSet Object of Scaphandre

You will need to perform a similar type of manipulation for all the applications you want to bring under the monitoring of Scaphandre, using:

NodeSelector : This allows you to constrain which nodes your pod is eligible to be scheduled based on node labels.

NodeAffinity : This allows you to constrain which nodes your pod is eligible to be scheduled based on node affinity rules.

PodAffinity : This allows you to constrain which nodes your pod is eligible to be scheduled based on affinity to other pods. For example, if you want a pod to be scheduled on nodes where…