S3 compatible storage (AWS, MinIO)

Here you can read how to set up AWS

Introduction

Our application supports integration with either AWS S3 or MinIO for file storage. Both services offer S3-compatible APIs, allowing you to choose based on your preference or infrastructure needs.

• AWS S3: A highly scalable object storage service from Amazon Web Services, widely used for storing and retrieving any amount of data at any time, providing strong security, durability, and low-latency performance.

• MinIO: An open-source, high-performance object storage service compatible with Amazon S3 APIs, ideal for on-premise or private cloud deployments, offering a cost-effective alternative to AWS S3.

AWS

https://docs.aws.amazon.com/docs.aws.amazon.com

Creating an S3 Bucket

1. Log in to the AWS Management Console.

2. Go to the S3 Dashboard and click Create bucket.

3. Enter a unique name for your bucket and complete the setup steps.

Creating an IAM User

1. In the IAM Dashboard, click Users > Create user.

2. Select Programmatic access.

3. Create a username and note the access key ID and secret access key.

Permissions: Attach the user with the S3 permissions by assigning a suitable policy, either a predefined S3 access policy or a custom one.

API

install dependencies

yarn add nestjs-s3

Setup variables

.env

AWS_S3_REGION= Your Aws Acoount region
AWS_S3_ACCESS_KEY= Created User Access Key Id
AWS_S3_SECRET_ACCESS_KEY= Created User Access Key Secret

constants.ts

AWS_S3_REGION: process.env.AWS_S3_REGION,
AWS_S3_ACCESS_KEY: process.env.AWS_S3_ACCESS_KEY,
AWS_S3_SECRET_ACCESS_KEY: process.env.AWS_S3_SECRET_ACCESS_KEY,

s3-bucket-name.enum.ts

export enum S3BucketName {
    S3DefaultBucket = 'your bucket name',
}

Initialize S3 Module

S3Module.forRoot({
    config: {
        credentials: {
            accessKeyId: constants.AWS_S3_ACCESS_KEY,
            secretAccessKey: constants.AWS_S3_SECRET_ACCESS_KEY,
        },
        region: constants.AWS_S3_REGION,
    },
}),

MinIO

Adding MinIO as an Alternative Storage Solution

MinIO is a highly scalable, distributed object storage system designed for high-performance applications. By integrating MinIO with your application, you can provide users with a reliable and secure way to store and retrieve data.

• Create a MinIO Account: Go to MinIO Console and register. Alternatively, if you’re self-hosting, you can download the MinIO server for your platform from the MinIO Download Page.

• Set Up MinIO Hosting:

  • Local Installation: Run minio server /path/to/storage to start a local MinIO server.

  • Cloud Installation: Use cloud hosting providers offering MinIO or configure a MinIO instance in your chosen environment.

API

Setup enviroment variables

.env

AWS_S3_HOST=Your MinIO Host
AWS_S3_ACCESS_KEY=Your MinIO Access Key
AWS_S3_SECRET_ACCESS_KEY=Your MinIO Secret Key
AWS_S3_REGION=Your MinIO Region

constants.ts

AWS_S3_REGION: process.env.AWS_S3_REGION,
AWS_S3_ACCESS_KEY: process.env.AWS_S3_ACCESS_KEY,
AWS_S3_SECRET_ACCESS_KEY: process.env.AWS_S3_SECRET_ACCESS_KEY,
AWS_S3_HOST: process.env.AWS_S3_HOST

Add MinIO specific params to S3 configuration

app.module.ts

S3Module.forRoot({
    config: {
        credentials: {
            accessKeyId: constants.AWS_S3_ACCESS_KEY,
            secretAccessKey: constants.AWS_S3_SECRET_ACCESS_KEY,
        },
        region: constants.AWS_S3_REGION,
        
        //Note: use parameters below for setting up minio service instead of AWS S3
        endpoint: constants.AWS_S3_HOST,
        forcePathStyle: true,
    },
}),

---

MinIO AIStor DocumentationAIStor Object Store Documentation