s3dol

s3 (through boto3) with a simple (dict-like or list-like) interface

To install: pip install s3dol

Documentation

A major, behaviour-clarifying v1 is in progress (design: misc/docs/architecture.md and the ADRs in misc/docs/decisions/). Today's S3Store will keep working through a compatibility shim, but a few v0 behaviours were bugs (e.g. an explicit endpoint_url= being silently dropped whenever AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY are exported) and v1 fixes them — which can move where your data goes.

Before upgrading to v1, run the diagnosis with the same arguments you pass S3Store today, in the environment you deploy in:

import s3dol

s3dol.diagnose(bucket_name="my-bucket", endpoint_url="...", path="...")

It prints what resolves (endpoint, region, signing, credential source — never a secret), where each value came from, and a v0-vs-v1 divergence table telling you whether the upgrade changes anything for your call. It never raises; a failing resolution is part of the report.

Recommended prerequisite to make getting started easier but not required.

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_DEFAULT_REGION=us-west-2

Add credentails in ~/.aws/credentials

[default]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Install AWS CLI

brew instal awscli

Set credentails with CLI

aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: