Minima AWS is an open-source, cloud-based Retrieval-Augmented Generation (RAG) framework that integrates with AWS services, including S3, SQS, RDS, and AWS Bedrock for embedding models and LLMs. This setup enables efficient document retrieval and chat-based interaction with your indexed data.

Minima AWS operates as a set of containerized services that work together to:

1. 📤 Upload & process documents from an AWS S3 bucket.
2. 🔍 Index documents for vector search using Qdrant.
3. 🤖 Leverage AWS Bedrock for embeddings and LLM-based querying.
4. 💬 Enable chat-based retrieval with an LLM using indexed documents.

• 🪣 Amazon S3 – Store and retrieve documents. (before running application create dir 'upload' inside bucket)
• 📩 Amazon SQS – Handle document processing requests.
• 🛢️ Amazon RDS (PostgreSQL/MySQL) – Store metadata about indexed documents.
• 🤖 AWS Bedrock – Used for:
  • LLMs (Chat models)
  • Embedding models (Vector representation of documents)

Before running the application, create a .env file in the project root directory and configure the following variables:

# AWS Credentials & Services
AWS_ACCESS_KEY_ID=your_aws_access_key
AWS_SECRET_ACCESS_KEY=your_aws_secret_key
AWS_DEFAULT_REGION=us-east-1
AWS_BUCKET_NAME=your_s3_bucket_name
AWS_FILES_PATH=s3_folder_path

# SQS Configuration
AWS_SQS_QUEUE=your_sqs_queue_name

# Local File Storage (optional)
LOCAL_FILES_PATH=/path/to/local/storage

# RDS Database Configuration
RDS_DB_INSTANCE=your_rds_instance
RDS_DB_NAME=your_rds_database
RDS_DB_USER=your_rds_user
RDS_DB_PASSWORD=your_rds_password
RDS_DB_PORT=5432  # PostgreSQL default

# Vector Search Configuration (Qdrant)
QDRANT_BOOTSTRAP=qdrant
QDRANT_COLLECTION=minima_collection

# AWS Bedrock - LLM & Embedding Model
EMBEDDING_MODEL_ID=amazon.titan-embed-text-v1
EMBEDDING_SIZE=1024
CHAT_MODEL_ID=arn:aws:bedrock:us-west-2:123456789012:model/anthropic.claude-3-sonnet-20240229-v1:0

• EMBEDDING_MODEL_ID → Embedding model for converting documents into vector representations.
• CHAT_MODEL_ID → LLM model for answering user queries.

arn:aws:bedrock:us-west-2:123456789012:model/anthropic.claude-3-sonnet-20240229-v1:0

The Model ID consists of:

• Provider: anthropic
• Model name: claude-3-sonnet-20240229-v1
• Version: 0

To create your own AWS Bedrock ARN, replace the components with your specific values:

1. Use your desired AWS region
2. Insert your actual AWS account ID
3. Select the appropriate model ID from available Bedrock models

Once the .env file is set up, deploy the application using Docker Compose:

docker compose up --build

This will start all required services in separate containers.

The system consists of multiple microservices, each running as a Docker container.

Use the following curl command to upload a file to Minima AWS:

curl -X 'POST' \
  'http://localhost:8001/upload/upload_files/?user_id=4637951a-7b45-4af4-805c-1f1c471668f3' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -F 'files=@example.pdf;type=application/pdf'

• URL: http://localhost:8001/upload/upload_files/
• Method: POST
• Query Parameter: user_id (Required)
• Headers:
  • accept: application/json
  • Content-Type: multipart/form-data
• Body: A file (e.g., example.pdf)

{
  "files": [
    {
      "user_id": "4637951a-7b45-4af4-805c-1f1c471668f3",
      "file_id": "52f206e1-26f7-4b62-84d2-7f9bac19400d",
      "file_path": "uploads/example.pdf",
      "filename": "example.pdf"
    }
  ]
}

1. Open Swagger UI in your browser:

   👉 http://localhost:8001/upload/docs#/default/upload_files_upload_upload_files__post

2. Select POST /upload/upload_files/.

3. Enter your user_id (e.g., 4637951a-7b45-4af4-805c-1f1c471668f3).

4. Upload your file (e.g., Black.pdf).

5. Click "Execute" to send the request.

Minima's UI is under development at Minima UI GitHub Repo.
Until the UI is ready, you can use websocat to interact with the chat service.

To connect to Minima Chat manually, install websocat:

MacOS:

brew install websocat

Linux:

sudo apt install websocat

Windows: Download from: Websocat Releases

websocat ws://localhost:8003/chat/{user-id}/{chat-name}/{file-id},{other-file-id}

• Replace {user-id} with your user ID.
• Replace {file-id} with the file IDs you want to search within.
• You can list multiple files using commas.

websocat ws://localhost:8003/chat/4637951a-7b45-4af4-805c-1f1c471668f3/minima-chat/67890,54321

This command initiates a chat session with files 67890 and 54321 for user 4637951a-7b45-4af4-805c-1f1c471668f3.

Minima AWS is licensed under the Mozilla Public License v2.0 (MPLv2).

📌 GitHub Repository: Minima AWS

• ❓ Issues? Open a GitHub Issue.
• 💬 Questions? Contact us via GitHub Discussions.