The right storage fit for your infrastructure

The right storage fit for your infrastructure Build • Constance Morales • 09/02/23 • 8 min read

I’m Constance, Product Marketing Manager of Block Storage. Since arriving at Scaleway, I realized that users are increasingly investing in storage capability, which has led us to accelerate the development of new Storage solutions to fit as many of our clients’ needs as possible.

I have come to two conclusions:

Storage definitely is a key requirement for numerous cloud use cases

Storage represents a significant expense - “9%” of the overall growth (“+4% in 2022”) of IT spending (even Gartner said so!)

How can startups, companies and individuals make sure their cloud storage usage is optimized and that every cent invested in storage is worth it?

Because, believe it or not, I've seen impressively well-built infrastructures in which storage was a real lever for better performance, cost efficiency, security and compliance.

So how do we exploit storage to fully benefit from it?

Keep reading to find our guide to Scaleway Storage solutions, how they differ, and discover which solution is best-suited to your cloud usage. And a big thank you to Franck Pagny, Scaleway’s PM on Serverless Database, Marie Debard, Scaleway’s PM on Object Storage, and Thomas Deschamps PM on Block Storage who helped to build this guide!

Object Storage

Object Storage allows you to store large amounts of unstructured data (documents, images, videos, etc.) and to distribute them instantly, anywhere in the world.

How does Object Storage work?

Object Storage stores data as distinct units, or "objects". Each object has a unique identifier, and is bundled with highly customizable metadata, making it easier for you to control the way that you upload, download, access and analyze your stored data. Objects are stored in a flat address space, with no file paths, and kept in secure and scalable buckets, making it easier to locate and retrieve your data across regions.

Object Storage specificities

Designed for multi-cloud interoperability

Ease of data management:

Intuitive storage classes (One-zone Infrequent Access, Multi-AZ Standard, Glacier)

Use of HTTP to access data: retrieve data simply via different applications or web browsers. Scaleway Object Storage is compatible with Amazon S3 standards

Lifecycle rules for object deletion or transitioning to a cold storage classes

Tags and metadata

Observability dashboard

Resilient:

Multi-AZ resiliency in the Paris region, with the same coming soon for the Amsterdam and Warsaw regions

Erasure coding

Object Storage powered by Hive, Scaleway's highly scalable and globally distributed database.

Secure:

Object lock to prevent object deletion

Fine-grained permission access management at user and application level, bucket policies

Object Storage use cases

Object Storage is a solution for any organization which needs to store a massive and growing amount of unstructured data in a scalable, efficient, and affordable way. Common use cases include:

Applications & websites

Great when content needs to be highly available and highly durable, such as for streaming videos and serving images,documents and other website files.

Data lakes

Object storage allows you to centralize a vast amount of data in its native format. Advantages like unlimited volume, low price and high scalability make object storage the go-to option to build a data lake.

Machine learning

Often, a large amount of data must be stored to train models, Object Storage is a great solution for this.

Cost optimization on Object Storage

It is essential to choose the right storage for your intended usage.

Scaleway Object Storage offers three different storage classes whose price and performance vary, making it easy to optimize storage costs according to your needs.

Standard class: for data that is accessed frequently. Multi-AZ resiliency is offered in the Paris region, meaning that data is distributed across different Availability Zones (AZ). This keeps data resilient against the total loss of one AZ in the event of an accident or disaster. Standard class in AMS and WAW currently allows you to store data across one AZ, and will soon be multi-AZ.

One Zone - Infrequent Access (IA) class: data is stored across three racks inside the same DC. This storage class is a good choice for storing secondary backup copies or recreatable data.

Glacier class: ideal for data that is accessed infrequently.

You can also leverage storage with microservices. Data stored in Object storage needs to be processed, and this usually means provisioning and configuring VMs, managing load balancers, and tweaking autoscaling rules. This is where Serverless comes in: Serverless functions free you from configuring and managing infrastructure and let you focus on your data. You can schedule automated transformation of all *.jpg or *.png images stored in an Object Storage bucket.

Cold storage

Cold Storage is an Object Storage class that is used to store “cold data”, the opposite of “hot data”. Hot data needs to be easily and quickly accessible as it is accessed and used very frequently. Cold storage, on the contrary, represents data that isn’t used frequently and thus doesn’t need fast access.

How does Cold Storage work?

Scaleway’s cold storage service, Glacier, is engineered on specific hardware:

SMR disks that increase storage density and overall per-drive storage capacity

Motherboards manufactured internally that can power disks on and off on-demand with a SATA bus tree matrix.

This particular hardware configuration allows us to offer competitive pricing, but the compromise is that data stored on cold storage has limitations in relation to data retrieval. What limitations exactly?

An object stored in Glacier class is listed for you to see, but cannot be downloaded instantly. It needs to be restored to the Standard class first. It can take anywhere from a few seconds to 24 hours to retrieve the first byte of an average-sized file. To facilitate restoration and ensure fast restitution of your data, we recommend using average-sized files (larger than 1MB).

Cold Storage specificities

Cost-efficient

Extremely reliable and secure: at Scaleway the SMR disks dedicated to cold storage are stored in our most secure data center: the “data-bunker”, a former nuclear fallout shelter within which your data is totally secure and isolated from all natural and technological risks.

European: Scaleway only stores data in…