A production-ready serverless image processing pipeline built with AWS, Terraform, and Vue 3. Users upload images directly to S3, which triggers an automated workflow that resizes images, detects labels using AWS Rekognition, and stores metadata in DynamoDB.
- Vue 3 (Composition API) - Modern reactive UI
- Vite - Fast build tool and dev server
- Vanilla JavaScript - No external dependencies
- AWS Lambda (Python 3.11, ARM64) - Serverless compute
- AWS Step Functions - Workflow orchestration
- AWS S3 - Object storage with presigned URLs
- AWS Rekognition - Image label detection
- AWS DynamoDB - Metadata storage
- API Gateway HTTP API - RESTful endpoints
- Terraform - Infrastructure as Code
- IAM - Secure role-based access control
βββββββββββββββ
β Vue Frontendβ
ββββββββ¬βββββββ
β POST /upload-url
βΌ
βββββββββββββββ ββββββββββββββββ
βAPI Gateway ββββββΆβ Lambda β
β β β (Presigned) β
βββββββββββββββ ββββββββββββββββ
β
β PUT (Presigned URL)
βΌ
βββββββββββββββ
β S3 Bucket β
ββββββββ¬βββββββ
β ObjectCreated Event
βΌ
βββββββββββββββ
β Lambda β
β (Trigger) β
ββββββββ¬βββββββ
β Start Execution
βΌ
βββββββββββββββββββββββββββββββββββββββ
β Step Functions State Machine β
β ββββββββββββ ββββββββββββ ββββββββ
β β Resize βββΆβRekognitionβββΆβStoreββ
β β Image β β Labels β βMeta ββ
β ββββββββββββ ββββββββββββ ββββββββ
βββββββββββββββββββββββββββββββββββββββ
β β
βΌ βΌ
βββββββββββββββ βββββββββββββββ
β S3 β β DynamoDB β
β (processed) β β (metadata) β
βββββββββββββββ βββββββββββββββ
β
β GET /results
βΌ
βββββββββββββββ
β Vue Frontendβ
β (Polling) β
βββββββββββββββ
| Component | Purpose |
|---|---|
| S3 Bucket | Stores original uploads and processed images. Triggers pipeline on upload. |
| API Gateway | Exposes REST endpoints for presigned URL generation and results retrieval. |
| Lambda Functions | 5 functions: presigned URL, trigger, resize, Rekognition, store metadata, get results |
| Step Functions | Orchestrates the 3-stage pipeline with retry logic and error handling. |
| Rekognition | Detects up to 10 labels per image with 70% confidence threshold. |
| DynamoDB | Stores image metadata (key, bucket, labels, timestamps) for frontend polling. |
- Upload Request: Frontend requests presigned URL from API Gateway
- Direct Upload: User uploads image directly to S3 using presigned URL
- Event Trigger: S3 ObjectCreated event invokes trigger Lambda
- Pipeline Execution: Step Functions orchestrates:
- ResizeImage: Copies image to
processed/prefix - RekognitionLabels: Detects labels using AWS Rekognition
- StoreMetadata: Saves results to DynamoDB
- ResizeImage: Copies image to
- Polling: Frontend polls
GET /resultsendpoint until processing completes - Display: Results displayed with detected labels
Error:
- DynamoDB table defined with
hash_key = "imageId"but Lambda code wrote"image_key" store_metadataLambda referencedTABLE_NAMEenvironment variable that wasn't configured
Impact:
- Runtime failures: DynamoDB operations rejected due to missing primary key
- Lambda crashes:
TABLE_NAMEwasNone, causingTable(None)initialization errors
Fix:
- Changed DynamoDB table hash key from
"imageId"to"image_key"to match Lambda code - Added
TABLE_NAMEenvironment variable tostore_metadataLambda in Terraform - Moved DynamoDB table initialization inside handler to avoid import-time failures
Files Changed:
infrastructure/dynamodb.tf(line 5)infrastructure/lambda.tf(lines 55-59)lambdas/store_metadata/handler.py(moved table creation inside handler)
Error:
- Presigned URL generated without
ContentTypein Params - Frontend sent
Content-Typeheader in PUT request - S3 rejected uploads with 403 Forbidden due to signature mismatch
Impact:
- All browser uploads failed silently
- Users couldn't upload images
Fix:
- Added
ContentTypeparameter to presigned URL generation in Lambda - Frontend sends matching
Content-Typeheader value - Implemented dynamic content type support (JPEG, PNG, WEBP)
Files Changed:
lambdas/get_presigned_url/handler.py(line 19, added ContentType to Params)frontend/src/App.vue(line 64, sends Content-Type header)
Error:
- Terraform apply failed with:
"States.ALL must appear alone and at end of list" - Retry blocks combined
States.ALLwith other error types:["States.TaskFailed", "States.Timeout", "States.ALL"]
Impact:
- Infrastructure deployment failures
- State machine couldn't be created
Fix:
- Changed all Retry blocks to use only
["States.ALL"](covers all error types) - Removed redundant error type specifications
Files Changed:
infrastructure/stepfunctions.tf(lines 51, 71, 91)
- Error Handling: Added try/except blocks and logging to all Lambda functions
- Step Functions Resilience: Added retry policies (3 attempts, exponential backoff) and catch blocks
- S3 Security: Enabled server-side encryption (AES256) and CORS configuration
- Lambda Configuration: Set timeouts (120s) and memory (512MB) for image processing
- Frontend UX: Implemented two-phase polling (active β background) with timeout protection
- CORS: Fixed API Gateway and S3 CORS for browser-based uploads
aws-img-pl/
βββ infrastructure/ # Terraform IaC
β βββ main.tf # Provider configuration
β βββ variables.tf # Input variables
β βββ outputs.tf # Output values
β βββ s3.tf # S3 bucket, CORS, encryption
β βββ dynamodb.tf # DynamoDB table
β βββ lambda.tf # Lambda function definitions
β βββ stepfunctions.tf # Step Functions state machine
β βββ api-gateway.tf # API Gateway routes
β βββ iam.tf # IAM roles and policies
βββ lambdas/ # Lambda function code
β βββ get_presigned_url/ # Generate S3 presigned URLs
β βββ trigger_step_function/ # S3 event β Step Functions
β βββ resize_image/ # Copy image to processed/
β βββ rekognition_labels/ # AWS Rekognition label detection
β βββ store_metadata/ # Save to DynamoDB
β βββ get_results/ # Query DynamoDB for results
βββ frontend/ # Vue 3 application
βββ src/
βββ App.vue # Main application component
- Fix Key Preservation Issue: Update
rekognition_labelsLambda to pass through originalkeyfield to prevent data loss in Step Functions state - Add Error Handling: Complete error handling for
resize_imageandrekognition_labelsLambdas (currently missing try/except blocks) - CloudWatch Monitoring: Add alarms for Lambda errors, Step Functions failures, and Rekognition throttling
- Implement Actual Image Resizing: Replace file copy with actual image resizing using PIL/Pillow
- Add Dead-Letter Queues: Configure DLQs for failed Lambda invocations and Step Functions executions
- Tighten IAM Permissions: Scope wildcard permissions to specific resources (Step Functions ARN, etc.)
- Add Input Validation: Validate image size, format, and quality before processing
- User Authentication: Add AWS Cognito for user management
- Image Preview: Display uploaded images in frontend
- Batch Processing: Support multiple image uploads
- CloudFront CDN: Add CDN for optimized image delivery
- S3 Lifecycle Policies: Automate cleanup of old processed images
- AWS CLI configured
- Terraform >= 1.5.0
- Node.js >= 20.19.0
- Python 3.11
cd infrastructure
terraform init
terraform plan
terraform applycd lambdas/<function-name>
zip -r build.zip handler.pycd frontend
npm install
npm run devβ Working Features:
- End-to-end image upload and processing
- Presigned URL generation with dynamic content types
- Step Functions orchestration with retry logic
- Rekognition label detection
- DynamoDB metadata storage
- Frontend polling with background processing
- CORS configuration for browser uploads
- Original upload key (
uploads/<uuid>.jpg) is dropped in RekognitionLabels step - Some Lambda functions lack comprehensive error handling
- No monitoring/alarms configured
MIT