Building a scalable video conferencing solution in a single day, using Jitsi and Scaleway

Building a scalable video conferencing solution in a single day, using Jitsi and Scaleway Deploy • Océane Franc • 23/03/20 • 7 min read

In the last 48 hours, several Scaleway teams were on the front line to launch, in less than a day, an integral videoconferencing solution: Ensemble .

Free, open-source and sovereign, Jitsi VideoConferencing powered by Scaleway will be available for the duration of the Covid-19 crisis!

You will be able to use this solution to keep in touch with your family and friends, maintain your business, interact with your customers, meet your patients or prepare your exams with other students.

How it All Started?

Monday 8am we decided to join the collective effort against #COVID19. By deploying Jitsi Meet , an open-source video conferencing solution providing secured virtual rooms with high video and audio quality, on more than one hundred Scaleway instances, we aimed to facilitate remote communication for all amid the COVID-19 pandemic.

How Does it Work?

In a nutshell, ensemble.scaleway.com aims at providing Jitsi servers. The number of people in need of a scalable videoconference solution being very high at the moment, it was our responsibility to find an alternative that was able to handle a significant load of video bridge requests.

As shown in the architecture diagram below, all Jitsi instances are constantly monitored to keep track of their capacity. This allows us to ensure that each user is provided with the least-used instance to create a virtual room and start a call.

The stateless API is composed of a front website in React and an API that will query a Prometheus (every 30 seconds) to get a list of all the Jitsi servers available and their current CPU usage.

The web application then selects the Jitsi server that has the most CPU available and returns the URL to the user. With that URL, a user can easily connect to the Jitsi server and start enjoying the call with an optimal sound and video quality.

All Jitsi servers are deployed on Scaleway Instances which can hold a large number of concurrent video bridges.

Now that we explained the general architecture and the typical user workflow of this application, let's see how it is deployed using infrastructure as code technologies.

Scaleway Terraform Module

Terraform is an infrastructure tool that manages cloud resources in a declarative paradigm. We decided to use the Scaleway Terraform Provider to manage all our infrastructure from a single versioned place. All changes applied to our infrastructure are tracked in a git repository.

To ensure consistency across concurrent Terraform execution, the terraform state is persisted in a Scaleway Database PostgreSQL managed instance. For that, we used the pg backend in Terraform .

We created all the required instances to make this application run:

The most important are the Jitsi servers. At the moment, we created more than 100 of those ( DEV1-L type). These instances run the Jitsi videoconference solution.

The instance running the Prometheus. Prometheus scraps the state of each Jitsi state and, in particular, the CPU usage of each Jitsi server.

The instances running the APIs. The API instances query the Prometheus to identify what are the CPU usage on all Jitsi servers and return them to the web application.

These constitute the infrastructure of ensemble.scaleway.com . Now we are going to complete this Terraform module by enabling those instances to serve our application.

Scaleway Jitsi Image

When creating an instance, you have to select or create an image . In each cloud deployment, instances are booted with a specific cloud image that is designed to meet the specific requirements of the instance.

First, we created a base image called base, which was the starting point for all the others. On this base image, we installed the requirements to run containers with Docker, docker-compose and a node_exporter that is used by our Prometheus monitoring system to know, among other information, the CPU usage of the machine.

From the base image, we then created a Jitsi image using the official docker compose distribution: docker-jitsi-meet . We also added an Nginx Prometheus exporter on docker-jitsi-meet docker-compose for monitoring purposes.

When a Jitsi instance boots with this image, a docker-compose will start and the Jitsi server which is running as a container will automatically start working as well.

Note that the base and Jitsi images are created with Ansible playbooks. It allows to easily recreate images when needed.

Finally, we created a front container image which gathers the web application code ( React ) and the API code ( Node.js ). This image will run inside containers that docker-compose will pull from a private Scaleway registry.

Scaleway Registry

The base image aims at providing the system-wide requirements such as the operating system and other basic components as docker, docker-composes , node_export ... However, we needed to be able to deploy new versions of our applications without rebooting an instance with a new image. As a result, we decoupled the base image from the application that was containerized.

The API and the React website which are bundled in the same container image are hosted on a Scaleway private registry . Once stored on the registry, images can be pulled in the instance by the docker daemon controlled by docker-compose to run the application. That comes in very handy when we need to deploy a new version of our API after a bug fix or a feature enhancement as we only need to push the new container image to the registry and tell docker-compose to use the new version.

Now that our applications are deployed, let's see how we can make our API server reliable using a Load Balancer.

Scaleway Load Balancer

Load Balancers are highly available and fully-managed instances that allow to distribute the workload among your various services. They ensure the scaling of all your applications while securing their continuous availability, even in the event of heavy traffic. Load Balancers are built to use an Internet-facing front-end server to shuttle information to and from backend servers. They provide a dedicated public IP address and forward requests automatically to one of the backend servers based on resource availability.

In the context of Jitsi, we used our Load Balancer to automatically forward requests to our API servers based on resource availability. Our API servers are the ones providing…