Scaleway Private LB for Golem.ai Security Assets

Scaleway Private LB for Golem.ai Security Assets Deploy • Kevin Baude • 17/06/24 • 5 min read

At Golem.ai , our day-to-day decisions and choices justify the trust our customers place in us.

Security is at the heart of each and every one of them.

That's why we distinguish between public and private traffic at the very heart of our applications and our Kubernetes Hosting Platform operated by Scaleway , our French Cloud provider.

Thus, Public traffic, which by definition enables our customers to consume our applications, is strictly controlled by our powerful Web Application Firewall (WAF) and AntiDDoS, while private traffic is dedicated to the administration of our applications by our Teams, and is only accessible via a ZeroTrust VPN Tunnel connected to the Private LoadBalancer.

This gives us a 360° view of network activity upstream and downstream of the platform.

In terms of watertightness between our different STAGING, PREPRODUCTION, PRODUCTION and MONITORING environments, each has its own virtual private network ( VLAN Layer 2 within a VPC ).

This prevents private flows from being opened up to the outside world, and isolates flows between environments if necessary.

Endpoint : Public / Private Flow

Operation

The operations described below correspond to the needs of Golem.ai .

Adapt to your needs !

To use a Private LoadBalancer between our application and the Golem.ai Private Network, we'll need to perform the following steps in order.

1 . Create a new Private LB with WebUI , CLI or Terraform

Click Load Balancers in the Network section of the Scaleway console side menu. If you have not already created a Load Balancer, the product creation page is displayed. Otherwise, your list of existing Load Balancers displays. Then, choose "Private Load Balancer". More info here , and below...

Informations 1. LB Name Fill in the name of the loadbalancer , respecting the nomenclature . private - lb - private - “env” - “domain without extension” eg . private - lb - prod - test for test . golem . ai 2. LB Zone Select zone where application is localised ( PARIS 1 by default ) 3. LB Model Select LB - S model . 200 Mbps of bandwidth is more than enough . 4. LB Type Select Private Load Balancer 5. Finalise Click on "Create Load Balancer" CopyContentIcon Copy code 2 . Linking your application to PrivateLB

To use Private LB, we no longer need to use Ingress.

Traffic is sent directly to the application's service in a private flow !

Scaleway Private LB ⇒ SVC K8s ⇒ Application

In our Service K8s object, we use the following annotations:

service . beta . kubernetes . io / scw - loadbalancer - externally - managed : true service . beta . kubernetes . io / scw - loadbalancer - id : zone / lb_id service . beta . kubernetes . io / scw - loadbalancer - zone : zone CopyContentIcon Copy code Please just note what CCM will or won’t manage in this case:

• Won’t create/delete the LB.

• Ignores the global configurations (such as size, private mode, IPs).

• Won’t detach private networks attached to the LB.

• won’t manage extra frontends and backends not starting with the service id.

• Will refuse to manage a LB with a name starting with the cluster id.

E.g.:

-- - apiVersion : v1 kind : Service metadata : name : test - lb - private namespace : ops - tools annotations : service . beta . kubernetes . io / scw - loadbalancer - externally - managed : "true" service . beta . kubernetes . io / scw - loadbalancer - id : "fr-par-1/0d1b714f-2767-4937-a9e5-4399cfd45338" service . beta . kubernetes . io / scw - loadbalancer - zone : fr - par - 1 labels : app . kubernetes . io / name : test - lb - private app . kubernetes . io / instance : test - lb - private spec : ports : - name : https port : 443 40D7FB7F107F0000 : error : 1608010C : STORE routines : ossl_store_handle_load_result : unsupported : . . / crypto / store / store_result . c : 151 : Unable to load certificate test . golem . ai -- -- Saving debug log to / var / log / letsencrypt / letsencrypt . log Requesting a certificate for test . golem . ai Successfully received certificate . Certificate is saved at : / etc / letsencrypt / live / test . golem . ai / fullchain . pem Key is saved at : / etc / letsencrypt / live / test . golem . ai / privkey . pem This certificate expires on 2024 - 08 - 28. These files will be updated when the certificate renews . Certbot has set up a scheduled task to automatically renew this certificate in the background . - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - If you like Certbot , please consider supporting our work by : * Donating to ISRG / Let 's Encrypt : https : / / letsencrypt . org / donate * Donating to EFF : https : / / eff . org / donate - le - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 4887 100 1178 100 3709 1593 5016 -- : -- : -- -- : -- : -- -- : -- : -- 6612 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 4442 0 4442 0 0 10888 0 -- : -- : -- -- : -- : -- -- : -- : -- 10887 b33097cf - 4e49 - 456b - 990b - 1bc8a64caef3 fd7c47b7 - 9214 - 4945 - 9bae - 67f18bfe0070 443 { "id" : "fd7c47b7-9214-4945-9bae-67f18bfe0070" , "name" : "test.golem.ai" , "inbound_port" : 443 , "backend" : { "id" : "b33097cf-4e49-456b-990b-1bc8a64caef3" , "name" : "915d146e-fef6-4873-92f9-58cbea07ac4e_tcp_30715" , "forward_protocol" : "tcp" , "forward_port" : 30715 , "forward_port_algorithm" : "roundrobin" , "sticky_sessions" : "none" , "sticky_sessions_cookie_name" : "" , "health_check" : { "port" : 30715 , "check_delay" : 5000 , "check_timeout" : 5000 , "check_max_retries" : 5 , "check_send_proxy" : false , "transient_check_delay" : null , "tcp_config" : { } } , "pool" : [ "172.16.0.54" , "172.16.0.7" , "172.16.0.25" , "172.16.0.9" , "172.16.0.21" , "172.16.0.18" , "172.16.0.15" , "172.16.0.31" , "172.16.0.14" , "172.16.0.17" , "172.16.0.22" , "172.16.0.27" ] , "lb" : { "id" : "0d1b714f-2767-4937-a9e5-4399cfd45338" , "name" : "private-lb-prod-test" , "description" : "" , "status" : "ready" , "instances" : [ ] , "organization_id" : "a7ec9296-de32-44d3-9f95-611cd8ee8e20" , "project_id" : "a7ec9296-de32-44d3-9f95-611cd8ee8e20" , "ip" : [ ] , "tags" : [ ] , "frontend_count" : 2 , "backend_count" : 2 , "type" : "lb-s" , "subscriber" : null , "ssl_compatibility_level" :…