8 Kubernetes Security tips for cloud engineers

8 Kubernetes Security tips for cloud engineers Deploy • James Martin • 01/06/23 • 5 min read

How important is Kubernetes security?

38% of 600+ DevOps and engineers surveyed recently by Red Hat think it isn’t taken seriously enough, or that security investment is inadequate.

So before we start, let’s look into how critical Kubernetes security can get…

21% of survey respondents said a security incident led to employee termination

25% said the organization was fined

37% identified revenue/customer loss as a result of a container and Kubernetes security incident

67% said Kubernetes security concerns had slowed down deployment

90% experienced at least one security incident in the last 12 months

Suffice to say: it’s a problem. Supply chain attacks, for example, are increasing rapidly, especially after the SolarWinds attack — one of the biggest cybersecurity breaches of the 21st century — and the discovery of Log4Shell and Spring4Shell vulnerability.

We’ve already covered Kubernetes security at length here on our blog . But what are the latest tips for keeping your clusters safe? We took inspiration from Lacework’s recent white paper , and augmented its advice with some Scaleway K8s expertise. We hope it helps!

1. Encrypt your data

It’s a standard step for data protection everywhere, including with Kubernetes security! So it’s critical to check that your cloud service provider (CSP) includes this option by default, as most do:

Example of encryption options in a Kubernetes storage class (Scaleway's CSI)

How can you check? The CSP’s CSI (Container Storage Interface) “ encrypted: “true” ” line confirms its encryption of persistent volumes.

As we explained here , this is particularly critical when it comes to user data, as its “critical” and “valuable” status means it’s the first information type concerned by data regulations such as GDPR.

2. Strict access control

The next key step in Kubernetes security is determining who can do what. This can be managed in several ways.

RBAC (Role-based access control) , your K8s cluster’s built-in method for determining whether a given user is allowed to perform certain actions, such as modifying configuration or accessing data. Users both need to access the cluster ( authentication ), then have the authorization to make changes, which means two levels of security are built in in this case. Use ClusterRoles to define roles across an entire cluster, and apply the rule of least privilege , whereby users’ accounts only have the privileges they need to perform their function.

IAM, or Identity Access Management , is recommended for larger organizations, and/or for those more comfortable inside the cloud, as it allows admins to fine-tune who can access which parts of their cloud infrastructure. Find out more about IAM here .

Open source tools such as Auth0 can also be used, so developers can connect any application written in any language or stack, and define the external identity providers, as well as the integrations they want to use… whilst using authentication for deployment, of course :) Discover more open source K8s tools here …

Open ID Connect/OIDC is one of the most foolproof ways to manage cluster access. Identity providers include Google, Facebook, and GitHub, meaning users’ existing IDs can be used, providing them with identity tokens which can be cryptographically verified by an X509 certificate. Find out more about OIDC, and K8s authentication and authorization, in this blogpost .

Whatever the tools, other security basics that should always apply are, of course: deleting users that have left your company, as well as inactive users; and ensuring your infrastructure’s security and network policies are correctly configured.

3. Monitor endpoints

It’s not that uncommon that public endpoints are accidentally published or exposed; even adding an external load balancer to your Kubernetes configuration can inadvertently give authentication rights to unwanted users. Indeed, this time last year, cybersecurity firm Cyble found over 900,000 Kubernetes instances exposed across the internet , according to InfoSecurity Magazine . Whence huge global potential for Kubernetes security incidents.

Early 2022, for example, security researchers from Apiiro discovered a vulnerability in the open-source continuous delivery platform ArgoCD , that let attackers access and exfiltrate sensitive information like passwords and API keys from clusters. Similarly notorious breaches have also occurred in recent years at Weight Watchers , Honda , Tesla and Universal , to name just a few.

Best practices to avoid such breaches include restricting access to carefully-selected developers, using IAM or RBAC (cf. above); manually approving all first external connections; creating a network map; and whitelisting key IP addresses.

4. Anticipate potential IaC errors

Using a Kubernetes-specific IaC (infrastructure as code) service like HashiCorp Terraform as the basis of your infrastructure allows you to automate deployments and re-launch them as many times as needed. However, with IaC, even the slightest human mistakes can have wide-reaching consequences, as potential errors can find themselves in production on thousands of different systems, provoking security issues and unwanted dependencies.

Ways around this Kubernetes security hazard range from common sense to super smart. Obviously, developers should be responsible for constantly inspecting their own code. If your organization doesn’t already have a solid peer review process for its code, it should. And if help is needed, numerous IaC scanning tools , including automated ones, exist to do so. They should notably prevent developers from pushing non-secure code, and therefore reduce the risk of costly errors ending up in production. Finally, only those who really need to should be able to access the repository.

5. Master container configuration at conception

When building a container, it’s essential to package all components so that your application runs properly. CI/CD should be considered as an external user in its own right, not as something that simply works with your cluster . As such, its minimal access requirements are a Container Registry (where you store the containers’ images), and access to the different namespaces of your cluster. Most importantly, it should be easy to remove access keys if they are compromised.

The solution is careful code craftsmanship. So be sure to master your…