Object Storage

​

Use Cases

​

Good Use Cases ✅

• Large volumes of read-only data storage - Store and serve static content efficiently
• High availability requirements - Built-in redundancy across multiple locations
• High scalability needs - Seamlessly scale from gigabytes to petabytes
• Unstructured data - Music, photos, videos, backups, and archives
• Geographically distributed data - Serve content closer to users globally

• Music streaming services (Spotify-like applications)
• Photo-heavy platforms (Instagram, Facebook)
• Long-term backup and archive storage

​

Bad Use Cases ❌

• I/O intensive operations - Database-like workloads requiring frequent reads/writes
• Frequent data modifications - Objects that change constantly
• Temporary file storage - Use ephemeral storage for temporary data
• Transactional data handling - Applications requiring ACID guarantees

​

Pros & Cons

​

Advantages

• Cost-effective - Reduces infrastructure costs for data storage
• Easy scalability - Minimizes management overhead with automatic scaling
• Durability - Built-in redundancy protects against data loss
• Accessibility - Access data from anywhere via HTTP/S APIs

​

Disadvantages

• Not optimal for frequently changing datasets - Every update creates a new version
• Eventual consistency - May be insufficient for applications requiring strong consistency guarantees
• Higher latency - Not suitable for low-latency database operations

​

Using Object Storage

​

AWS S3

1. Access AWS S3 Console
   • Navigate to AWS S3 Console
   • Click Create bucket
2. Create Bucket
   • Choose a unique bucket name
   • Select your region (preferably same as your Qovery cluster)
   • Configure bucket settings (versioning, encryption, etc.)

3. Configure Application
   • Use the AWS SDK in your application code
   • Set up authentication via environment variables

const aws = require('aws-sdk');
const express = require('express');
const multer = require('multer');
const multerS3 = require('multer-s3');

const endpoint = new aws.Endpoint('s3.us-east-2.amazonaws.com');
const s3 = new aws.S3({ endpoint: endpoint });

const upload = multer({
  storage: multerS3({
    s3: s3,
    bucket: 'your-bucket-here',
    acl: 'public-read',
    key: function (request, file, cb) {
      console.log(file);
      cb(null, file.originalname);
    }
  })
}).array('upload', 1);

AWS_ACCESS_KEY_ID=your_access_key_id
AWS_SECRET_ACCESS_KEY=your_secret_access_key

​

Digital Ocean Spaces

1. Navigate to DO Console
   • Go to DigitalOcean Spaces
   • Click Create a Space
2. Create Space
   • Choose a datacenter region
   • Configure CDN settings (optional)
   • Set bucket name and permissions

3. Configure Application
   • Use AWS S3-compatible SDK
   • Set endpoint to DigitalOcean

https://nyc3.digitaloceanspaces.com  (for NYC3 region)
https://sfo3.digitaloceanspaces.com  (for SFO3 region)
https://ams3.digitaloceanspaces.com  (for AMS3 region)

AWS_ACCESS_KEY_ID=your_spaces_access_key
AWS_SECRET_ACCESS_KEY=your_spaces_secret_key

​

Scaleway Object Storage

1. Access Scaleway Console
   • Go to Scaleway Object Storage
   • Click Create a bucket
2. Create Bucket
   • Choose a region (Paris, Amsterdam, Warsaw)
   • Configure bucket settings
   • Set up access permissions

3. Configure Application
   • Use S3-compatible SDK
   • Set endpoint to Scaleway region

https://s3.fr-par.scw.cloud  (Paris)
https://s3.nl-ams.scw.cloud  (Amsterdam)
https://s3.pl-waw.scw.cloud  (Warsaw)

AWS_ACCESS_KEY_ID=your_scaleway_access_key
AWS_SECRET_ACCESS_KEY=your_scaleway_secret_key

​

Next Steps