Managed Database: how to choose the right type of storage

Managed Database: how to choose the right type of storage Build • Nour ZERHOUNI • 01/03/22 • 5 min read

Data is critical to any organization in many different ways as data is knowledge, and knowledge is power. From customer data to human resources to core business services, inventory tracking, growth monitoring, and more, databases are the cornerstone of any software application.

Cloud databases, also referred to as managed databases or Database-as-a-Service (DBaaS), make the provisioning, configuration, and management of databases dramatically easier and faster, turning databases into a true utility service.

Their agility and intuitive nature enable developers to turn and run services in seconds. Even without any particular expertise in database administration, you can create and automate the provisioning of highly available, backed-up, secure, and ready-to-use databases, allowing you to dedicate your time and energy to building applications and producing value.

This article will shed light on the main differentiators when it comes to choosing the storage type for your managed database.

Understanding managed database

The predicted volume of the global datasphere by 2025 is measured in zettabytes (that’s 21 zeros!) and many aspects of today’s technologies contribute to this growth. The data streams collected by connected devices and sensors (Internet of Things - IoT), the countless applications and services created everyday, and providing new ways to generate data, as well as the changes in software architecture towards microservices requiring multiple databases (while a monolithic application requires a single database) are all contributing to this exponential growth. All those different data types, volumes, and processing requirements, in addition to the growing variety of databases, including both SQL and NoSQL make hosting and supporting them a serious challenge from an IT perspective.

A managed database service enables you to effortlessly set up and maintain the service using a UI or an API, or even an Infrastructure as Code. It includes engine upgrades, automated remote backups, high availability, and security. In addition, Scaleway Managed Databases come with 24/7 assistance so you can be serene regarding your runtime and able to focus on development.

While managed database provides a common set of abstractions through which a database can be consumed, you still need to be able to configure your database instances in order to make the most of this service, in terms of performance, flexibility, security, resiliency, and more.

First, let us start with an overview of the different compartments forming a cloud database service:

Managed databases components

Managed Database manages resources residing on physical hardware provisioned using an Infrastructure-as-a-Service abstraction through the orchestration engine. A set of best practices and corporate standards for database operations and policies are configured in an optimized way, with great attention being paid to the system’s security.

Storage is an integral part of any database, as your data is kept there, and you want to make sure it meets your needs perfectly in terms of data accessibility, scaling, and resilience. You can also choose the storage type that maximizes performance or improves cost management.

Choose the right storage

To understand the storage types compatible with a cloud database and their main characteristics and differentiators, below is a quick overview:

Local volume

When you create a new instance, a boot volume is created directly on the bare-metal hypervisor which can be seen as local storage for your instance. A local volume is tied to that instance, and so is its lifespan. This storage type is particularly adapted to high storage performance requirements

Block volume

You can store data in a block volume beyond the lifespan of your instance. You can also upgrade your block size or change your instance capacity independently from one another

Object Storage

Based on the Simple Storage Service (Amazon S3) protocol where objects are organized in buckets. This is suitable for storing documents and multimedia content that you can easily upload, download, and visualize any time, from anywhere in the world. A wise choice for database backups

Cold Storage

If the data you want to store is rarely used and you want to retain it for a longer period of time then this option is the most cost-efficient.

Scaleway’s Managed Database for PostgreSQL and MySQL supports two types of storage: local storage and Block Storage. The following pages will hopefully answer all your questions on the matter.

Local storage vs Block Storage

Key differences

Below is a summary of the key differences that you should consider when choosing a type of storage for your Managed Database.

Local storage vs Block Storage - Benchmark

Performance benchmark

Bench dataset size = 23GB

Region : Amsterdam

Instance type : db-gp-xs, 4 vCPU, 16GB RAM

TPS : Transaction Per Second

Local storage vs Block Storage - Performance benchmark

In short: Block Storage brings more flexibility and scalablity, while local storage offers optimal performance. Local is by definition faster due to its proximity to the compute node. Block Storage for Managed Database offers greater flexibility allowing cost optimization by reducing the compute power when not needed. In addition, the several upcoming improvements on our Block Storage-based databases make it a better option for cloud native uses. Some of the planned evolutions are:

Snapshot exports in a different AZ

Read replicas for both local storage and Block Storage databases

Block Storage performance improvements

Backups vs. Snapshots

We’ve seen in the table above that local storage offers distant backup possibilities while Block Storage relies on snapshots. Let’s dig a little deeper into the main differences between these two disaster-recovery mechanisms.

Local Storage

On local storage database instances, backups are logical exports of a database.

Logical backups are actually data extractions from the engine which require at least one CPU thread to retrieve and compress data.

In case not all the data is available in the memory, it needs to be retrieved from the disk, generating IOs which will penalize the other ongoing operations.

To ensure a consistent database state, logical backups are made in a single transaction, which impacts the execution time of backup creation,…