digitalocean/k8s-staticroute-operator

digitalocean/k8s-staticroute-operator

Description: Create static routes for your k8s nodes using CRDs.

Language: Python

License: Apache-2.0

Stars: 61

Forks: 12

Open issues: 21

Created: 2022-09-01T14:39:52Z

Pushed: 2025-04-06T21:25:02Z

Default branch: main

Fork: no

Archived: yes

README:

DOKS static route operator is available as a managed component (https://docs.digitalocean.com/products/kubernetes/how-to/use-routing-agent/).

2/22/2025: We will release a *managed* static route operator (called *routing agent*) for DigitalOcean Kubernetes within a couple of weeks. It will support both 0.0.0.0 default route, and multiple destinations using ECMP!! This repo will become irrelevant and replaced by a managed component. We strongly recommend to wait for the new release if you are starting out now.

Overview

The main purpose of the Static Routes Operator is to manage entries in the Linux routing table of each worker node based on CRD spec. It is deployed as a DaemonSet, hence it will run on each node of your DOKS cluster.

Below diagram illustrates the operational concept:


Table of Contents

• [Prerequisites](#prerequisites)
• [Deploying the Kubernetes Static Routes Operator](#deploying-the-kubernetes-static-routes-operator)
• [Testing the Setup](#testing-the-setup)
• [Cleaning Up](#cleaning-up)

Prerequisites

1. A working DOKS cluster you have access to. 2. Kubectl CLI installed on your local machine, and configured to point to your DOKS cluster. 3. NAT GW Droplet configured and running as detailed here.

Note:

Make sure your NAT GW Droplet is created in the same VPC as your DOKS cluster.

Deploying the Kubernetes Static Routes Operator

1. Deploy the latest release of static routes operator to your DOKS cluster using kubectl:

kubectl apply -f https://raw.githubusercontent.com/digitalocean/k8s-staticroute-operator/main/releases/v1/k8s-staticroute-operator-v1.0.0.yaml

Hint:

You can check the latest version in the [releases](releases/) path from the k8s-staticroute-operator GitHub repo.

2. Check if operator Pods are up and running:

kubectl get pods -l name=k8s-staticroute-operator -n static-routes

Output looks similar to:

NAME READY STATUS RESTARTS AGE
k8s-staticroute-operator-9vp7g 1/1 Running 0 22m
k8s-staticroute-operator-mlfff 1/1 Running 0 22m

3. Check operator logs - no exceptions should be reported:

kubectl logs -f ds/k8s-staticroute-operator -n static-routes

Output looks similar to:

[2022-08-24 14:42:13,625] kopf._core.reactor.r [DEBUG ] Starting Kopf 1.35.6.
[2022-08-24 14:42:13,625] kopf._core.engines.a [INFO ] Initial authentication has been initiated.
[2022-08-24 14:42:13,626] kopf.activities.auth [DEBUG ] Activity 'login_via_pykube' is invoked.
[2022-08-24 14:42:13,628] kopf.activities.auth [DEBUG ] Pykube is configured in cluster with service account.
[2022-08-24 14:42:13,629] kopf.activities.auth [INFO ] Activity 'login_via_pykube' succeeded.
[2022-08-24 14:42:13,629] kopf.activities.auth [DEBUG ] Activity 'login_via_client' is invoked.
[2022-08-24 14:42:13,631] kopf.activities.auth [DEBUG ] Client is configured in cluster with service account.
[2022-08-24 14:42:13,632] kopf.activities.auth [INFO ] Activity 'login_via_client' succeeded.
[2022-08-24 14:42:13,632] kopf._core.engines.a [INFO ] Initial authentication has finished.
[2022-08-24 14:42:13,789] kopf._cogs.clients.w [DEBUG ] Starting the watch-stream for customresourcedefinitions.v1.apiextensions.k8s.io cluster-wide.
[2022-08-24 14:42:13,791] kopf._cogs.clients.w [DEBUG ] Starting the watch-stream for staticroutes.v1.networking.digitalocean.com cluster-wide.
...

Testing the Setup

Each sample CRD provided creates a static route to two different websites which report back your public IP - ifconfig.me/ip, and ipinfo.io/ip.

Typical static route definition looks like below:

apiVersion: networking.digitalocean.com/v1
kind: StaticRoute
metadata:
name: static-route-ifconfig.me
spec:
destinations:
- "34.160.111.145"
gateway: "10.116.0.5"

Explanations for the above configuration:

• spec.destinations - A list of host IPs (or subnet CIDRs) to route through the gateway.
• spec.gateway - Gateway IP address used for routing the host(s)/subnet(s).

To test the setup, download a sample manifest from the [examples](examples) location:

# Example for ifconfig.me
curl -O https://raw.githubusercontent.com/digitalocean/k8s-staticroute-operator/main/examples/static-route-ifconfig.me.yaml

# Example for ipinfo.io
curl -O https://raw.githubusercontent.com/digitalocean/k8s-staticroute-operator/main/examples/static-route-ipinfo.io.yaml

After downloading the manifests, make sure to replace the <> placeholders in each manifest file. Then, apply each manifest using kubectl:

# Example for ifconfig.me
kubectl apply -f static-route-ifconfig.me.yaml

# Example for ipinfo.io
kubectl apply -f static-route-ipinfo.io.yaml

Hint:

Above command will create the static route custom resources in the default namespace. In production environments (and not only), it's best to have a dedicated namespace with RBAC set (based on what people or teams need to have access).

Next, check if the static route resources were created:

kubectl get staticroutes -o wide

The output looks similar to:

NAME DESTINATIONS GATEWAY AGE
static-route-ifconfig.me ["34.160.111.145"] 10.116.0.5 7m2s
static-route-ipinfo.io ["34.117.59.81"] 10.116.0.5 4s

Now, check if the custom static routes were applied on each worker node, after SSH-ing:

route -n

The output looks similar to (the irrelevant lines were omitted from the output for better visibility):

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 206.81.0.1 0.0.0.0 UG 0 0 0 eth0
...
34.117.59.81 10.116.0.5 255.255.255.255 UGH 0 0 0 eth1
34.160.111.145 10.116.0.5 255.255.255.255 UGH 0 0 0 eth1
...

Note:

In the above example the NAT GW private IP used for testing is 10.116.0.5.

Next, deploy the [curl-test](examples/curl-test.yaml) Pod to test the setup:

kubectl apply -f…