tap-s3

tap-s3 is a Singer tap for S3.

Built with the Meltano Tap SDK for Singer Taps.

A full list of supported settings and capabilities for this tap is available by running:

tap-s3 --about

This Singer tap will automatically import any environment variables within the working directory's .env if the --config=ENV is provided, such that config values will be considered if a matching environment variable is set either in the terminal context or in the .env file.

This tap authenticates using your local AWS credential chain (via boto3.Session), so no credentials are stored in the tap config. Make sure you're already logged in before running the tap, e.g.:

aws sso login --profile my-profile

If you use a named profile, set it via the aws_profile config option (or TAP_S3_AWS_PROFILE).

You can easily run tap-s3 by itself or in a pipeline using Meltano.

tap-s3 --version
tap-s3 --help
tap-s3 --config CONFIG --discover > ./catalog.json

Follow these instructions to contribute to this project.

Prerequisites:

• Python 3.10+
• uv

uv sync

Create tests within the tests subfolder and then run:

uv run pytest

You can also test the tap-s3 CLI interface directly using uv run:

uv run tap-s3 --help

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Use Meltano to run an EL pipeline:

# Install meltano
uv tool install meltano

# Test invocation
meltano invoke tap-s3 --version

# Run a test EL pipeline
meltano run tap-s3 target-jsonl

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.