S3 API Implementation

Relevant source files

• cmd/api-errors.go
• cmd/api-response.go
• cmd/api-router.go
• cmd/apierrorcode_string.go
• cmd/bucket-handlers.go
• cmd/bucket-replication.go
• cmd/encryption-v1.go
• cmd/encryption-v1_test.go
• cmd/generic-handlers.go
• cmd/generic-handlers_test.go
• cmd/handler-utils.go
• cmd/object-api-datatypes.go
• cmd/object-api-errors.go
• cmd/object-api-options.go
• cmd/object-api-utils.go
• cmd/object-api-utils_test.go
• cmd/object-handlers.go
• cmd/routers.go

Purpose and Scope

This document describes MinIO's implementation of the Amazon S3-compatible HTTP API, covering request routing, handler architecture, authentication flow, object and bucket operations, error handling, and response generation. The S3 API layer serves as the primary interface for client applications, translating HTTP requests into internal storage operations.

Related Documentation:

• For server initialization and startup sequence, see Server Initialization and Request Routing
• For authentication and IAM details, see Identity and Access Management
• For underlying storage implementation, see Storage Abstraction Layer and Erasure Coding Engine
• For distributed coordination between nodes, see Distributed Coordination

Architecture Overview

The S3 API implementation is the HTTP interface layer that translates S3-compatible requests into internal storage operations.

Component Structure

Sources: cmd/api-router.go67-70 cmd/object-handlers.go68-70 cmd/routers.go54-81 cmd/generic-handlers.go54-81

Handler Architecture

objectAPIHandlers Structure

The objectAPIHandlers struct cmd/object-handlers.go68-70 provides access to the storage layer through its ObjectAPI function field, which returns the active ObjectLayer implementation.

Handler Categories:

Category	Key Handlers	Storage Layer Method	Code Location
Object Read	GetObjectHandler, HeadObjectHandler, GetObjectAttributesHandler	GetObjectNInfo, GetObjectInfo	cmd/object-handlers.go717 cmd/object-handlers.go1012 cmd/object-handlers.go988
Object Write	PutObjectHandler, CopyObjectHandler, PostObjectHandler	PutObject, CopyObject	cmd/object-handlers.go1547 cmd/object-handlers.go1037 cmd/object-handlers.go4201
Object Delete	DeleteObjectHandler	DeleteObject	cmd/object-handlers.go2316
Bucket Ops	ListBucketsHandler, PutBucketHandler, DeleteBucketHandler, HeadBucketHandler	ListBuckets, MakeBucket, DeleteBucket, GetBucketInfo	cmd/bucket-handlers.go305 cmd/bucket-handlers.go720 cmd/bucket-handlers.go922
Bucket List	ListObjectsV2Handler, ListObjectsV1Handler, ListObjectVersionsHandler	ListObjects, ListObjectVersions	cmd/bucket-handlers.go1295 cmd/bucket-handlers.go1008
Multipart	NewMultipartUploadHandler, PutObjectPartHandler, CompleteMultipartUploadHandler, AbortMultipartUploadHandler, ListObjectPartsHandler	NewMultipartUpload, PutObjectPart, CompleteMultipartUpload, AbortMultipartUpload, ListObjectParts	cmd/object-handlers.go2456 cmd/object-handlers.go2717 cmd/object-handlers.go3055 cmd/object-handlers.go3666 cmd/object-handlers.go3737
S3 Select	SelectObjectContentHandler	GetObjectNInfo with SQL filtering	cmd/object-handlers.go105
Batch Delete	DeleteMultipleObjectsHandler	DeleteObjects	cmd/bucket-handlers.go413

Standard Handler Pattern:

1. Call api.ObjectAPI() to get storage interface cmd/object-handlers.go722-726
2. Extract bucket/object from mux.Vars(r) cmd/object-handlers.go728-734
3. Authenticate via checkRequestAuthType() or authenticateRequest() cmd/object-handlers.go327-358
4. Parse request options using getOpts() cmd/object-handlers.go319-323
5. Call ObjectLayer method with parsed parameters
6. Generate response via writeSuccessResponseXML() or writeErrorResponse()

Sources: cmd/object-handlers.go68-70 cmd/object-handlers.go105-4687 cmd/bucket-handlers.go200-1480

Request Routing

Routes are registered in registerAPIRouter() cmd/api-router.go207-596 using the mux.Router from the github.com/minio/mux package.

Route Registration Flow

Sources: cmd/routers.go84-94 cmd/api-router.go207-596

Endpoint Mapping

Object Routes (/{bucket}/{object:.+} from cmd/api-router.go413-596):

HTTP Method	Query Params	Handler	Code Location
GET	-	GetObjectHandler	cmd/object-handlers.go717
GET	attributes	GetObjectAttributesHandler	cmd/object-handlers.go988
GET	legal-hold	GetObjectLegalHoldHandler	cmd/object-handlers.go2196
GET	retention	GetObjectRetentionHandler	cmd/object-handlers.go2095
GET	tagging	GetObjectTaggingHandler	cmd/object-handlers.go4005
GET	acl	GetObjectACLHandler	cmd/object-handlers.go3901
GET	select, select-type=2	SelectObjectContentHandler	cmd/object-handlers.go105
HEAD	-	HeadObjectHandler	cmd/object-handlers.go1012
PUT	-	PutObjectHandler	cmd/object-handlers.go1547
PUT	legal-hold	PutObjectLegalHoldHandler	cmd/object-handlers.go2242
PUT	retention	PutObjectRetentionHandler	cmd/object-handlers.go2138
PUT	tagging	PutObjectTaggingHandler	cmd/object-handlers.go4048
PUT	acl	PutObjectACLHandler	cmd/object-handlers.go3985
DELETE	-	DeleteObjectHandler	cmd/object-handlers.go2316
DELETE	tagging	DeleteObjectTaggingHandler	cmd/object-handlers.go4128
POST	restore	PostRestoreObjectHandler	cmd/object-handlers.go4161
POST	-	PostObjectHandler (form upload)	cmd/object-handlers.go4201

Bucket Routes (/{bucket} from cmd/api-router.go299-412):

HTTP Method	Query Params	Handler	Code Location
GET	list-type=2	ListObjectsV2Handler	cmd/bucket-handlers.go1295
GET	-	ListObjectsV1Handler	cmd/bucket-handlers.go1008
GET	versions	ListObjectVersionsHandler	cmd/bucket-handlers.go1482
GET	location	GetBucketLocationHandler	cmd/bucket-handlers.go203
GET	uploads	ListMultipartUploadsHandler	cmd/bucket-handlers.go250
GET	versioning	GetBucketVersioningHandler	cmd/bucket-handlers.go2230
GET	policy	GetBucketPolicyHandler	cmd/bucket-handlers.go1793
GET	tagging	GetBucketTaggingHandler	cmd/bucket-handlers.go2042
GET	object-lock	GetBucketObjectLockConfigHandler	cmd/bucket-handlers.go2284
GET	lifecycle	GetBucketLifecycleHandler	cmd/bucket-handlers.go1867
GET	replication	GetBucketReplicationConfigHandler	cmd/bucket-handlers.go2387
GET	encryption	GetBucketEncryptionHandler	cmd/bucket-handlers.go2156
PUT	-	PutBucketHandler	cmd/bucket-handlers.go720
PUT	versioning	PutBucketVersioningHandler	cmd/bucket-handlers.go2247
PUT	policy	PutBucketPolicyHandler	cmd/bucket-handlers.go1703
PUT	tagging	PutBucketTaggingHandler	cmd/bucket-handlers.go2074
PUT	object-lock	PutBucketObjectLockConfigHandler	cmd/bucket-handlers.go2301
PUT	lifecycle	PutBucketLifecycleHandler	cmd/bucket-handlers.go1898
PUT	replication	PutBucketReplicationConfigHandler	cmd/bucket-handlers.go2494
PUT	encryption	PutBucketEncryptionHandler	cmd/bucket-handlers.go2193
DELETE	-	DeleteBucketHandler	cmd/bucket-handlers.go922
DELETE	policy	DeleteBucketPolicyHandler	cmd/bucket-handlers.go1771
DELETE	tagging	DeleteBucketTaggingHandler	cmd/bucket-handlers.go2116
DELETE	lifecycle	DeleteBucketLifecycleHandler	cmd/bucket-handlers.go1851
DELETE	replication	DeleteBucketReplicationConfigHandler	cmd/bucket-handlers.go2375
DELETE	encryption	DeleteBucketEncryptionHandler	cmd/bucket-handlers.go2140
POST	delete	DeleteMultipleObjectsHandler	cmd/bucket-handlers.go413
HEAD	-	HeadBucketHandler	cmd/bucket-handlers.go1655

Root Routes (/ from cmd/api-router.go226-241):

HTTP Method	Query Params	Handler	Code Location
GET	-	ListBucketsHandler	cmd/bucket-handlers.go305

Route Registration Pattern cmd/api-router.go299-412:

Routes are registered using a fluent API with method chaining:

Key components:

• .Methods(http.MethodGet): Specifies HTTP method (GET, PUT, POST, DELETE, HEAD)
• .HandlerFunc(...): Wraps handler with s3APIMiddleware for metrics, tracing, compression
• .Queries("key", "value"): Matches query parameters to distinguish operations on same path
• Flags: noGZS3HFlag disables compression, traceHdrsS3HFlag traces headers only, noThrottleS3HFlag disables request throttling

The .Queries() method is critical for operation disambiguation when multiple S3 operations share the same HTTP method and path but differ by query parameters.

Sources: cmd/api-router.go207-596 cmd/object-handlers.go105-4687 cmd/bucket-handlers.go200-1480

Request Processing Pipeline

Requests flow through the globalMiddlewares slice cmd/routers.go54-81 and handler-specific middleware from s3APIMiddleware() cmd/api-router.go171-205

Middleware Stack

Sources: cmd/routers.go54-81 cmd/api-router.go171-205 cmd/generic-handlers.go109-472

Key Middleware Functions

addCustomHeadersMiddleware (from globalMiddlewares list):

• Sets x-amz-request-id using mustGetRequestID() for request tracking
• Adds x-amz-id-2 with deployment ID from globalDeploymentID()
• Enables request correlation for audit logging and debugging

httpTracerMiddleware (from globalMiddlewares list):

• Wraps handler with request/response tracing infrastructure
• Creates mcontext.TraceCtxt for structured logging
• Stores trace context via mcontext.ContextTraceKey in request context
• Captures timing, HTTP status, error messages

setAuthMiddleware (from globalMiddlewares list):

• Parses Authorization header to extract credentials
• Detects auth type: AWS Signature V2/V4, Presigned, JWT, Anonymous
• Sets ReqInfo.Cred with parsed credentials for downstream handlers
• Does not enforce authorization (delegated to individual handlers)

setRequestLimitMiddleware (from globalMiddlewares list):

• Rejects reserved metadata headers (prefixed with X-Minio-Internal-)
• Enforces header size limits: 8KB total, 2KB for user-defined metadata
• Wraps request body with http.MaxBytesReader(w, r.Body, requestMaxBodySize)
• Returns ErrMetadataTooLarge or ErrUnsupportedMetadata on violations

setRequestValidityMiddleware (from globalMiddlewares list):

• Validates Host header format using hasBadHost()
• Checks for bad path components (., ..) via hasBadPathComponent()
• Validates bucket names against S3 naming rules using s3utils.CheckValidBucketNameStrict()
• Blocks SSE-C requests over non-TLS connections (security requirement)
• Rejects requests with multiple authentication types via hasMultipleAuth()

setUploadForwardingMiddleware (from globalMiddlewares list):

• Forwards PUT/POST requests to appropriate site in multi-site replication
• Used for site replication feature to route writes to designated cluster

setBucketForwardingMiddleware (from globalMiddlewares list):

• Forwards bucket requests to correct cluster in federated deployments
• Uses DNS-based bucket federation to route requests
• Queries globalDNSConfig for bucket location

s3APIMiddleware (handler-level wrapper):

• collectAPIStats: Increments API counters in globalHTTPStats, measures latency
• maxClients: Limits concurrent requests using semaphore, returns 503 SlowDown on overflow
• gzhttp.GzipHandler: Compresses responses when Accept-Encoding: gzip present
• Configured with flags: noGZS3HFlag, traceHdrsS3HFlag, noThrottleS3HFlag

Sources: cmd/generic-handlers.go46-472 cmd/routers.go54-81 cmd/api-router.go171-205

Authentication and Authorization

Each handler calls checkRequestAuthType() or authenticateRequest() to verify credentials and check IAM permissions.

Authentication Flow

Sources: cmd/auth-handler.go1-300 cmd/iam.go214-287

Authentication Types

Auth Type	Header/Query	Detection	Validation Function
AWS Sig V4	Authorization: AWS4-HMAC-SHA256...	isRequestSignatureV4()	validateV4Signature()
Presigned V4	X-Amz-Signature query param	isRequestPresignedSignatureV4()	validateV4Signature()
AWS Sig V2	Authorization: AWS ...	isRequestSignatureV2()	validateV2Signature()
Presigned V2	Signature query param	isRequestPresignedSignatureV2()	validateV2Signature()
STS Token	X-Amz-Security-Token header	isRequestJWT()	JWT token validation
Anonymous	No credentials	authTypeAnonymous	Bucket policy only

Validation Steps (from cmd/auth-handler.go1-300):

1. Type Detection: Call getRequestAuthType(r) to identify auth method
2. Signature Validation: Verify HMAC signature using secret key
3. Credential Lookup: Fetch from globalIAMSys.GetUser(accessKey)
4. Policy Check: Call globalIAMSys.IsAllowed(policy.Args{...})
5. Return Code: ErrNone for success, ErrAccessDenied/ErrSignatureDoesNotMatch for failure

The checkRequestAuthType() function cmd/auth-handler.go1-300 combines all steps and returns an APIErrorCode.

Sources: cmd/auth-handler.go1-300 cmd/iam.go214-287

Object Operations

GET Object Implementation

Sources: cmd/object-handlers.go313-578

GetObjectHandler Flow cmd/object-handlers.go313-578:

1. Authentication: Verify s3:GetObject permission via authenticateRequest()
2. Range Processing: Parse Range header for partial reads (HTTP 206)
3. Precondition Evaluation: Check If-Match, If-None-Match, If-Modified-Since, If-Unmodified-Since
4. Callback Setup: Configure opts.CheckPrecondFn to decrypt metadata and re-authorize with tags
5. Object Stream: Call objectAPI.GetObjectNInfo() to obtain streaming reader
6. Encryption Handling: Decrypt content if SSE-S3, SSE-KMS, or SSE-C enabled
7. Response Headers: Set Content-Type, Content-Length, ETag, Last-Modified, Cache-Control
8. Streaming Copy: Use xioutil.Copy() to stream object data to client
9. Event Notification: Trigger ObjectAccessedGet event

Conditional Request Support:

• If-Match: Return object only if ETag matches specified value
• If-None-Match: Return 304 Not Modified if ETag matches
• If-Modified-Since: Return object only if modified after date
• If-Unmodified-Since: Return 412 Precondition Failed if modified after date

Sources: cmd/object-handlers.go313-578

PUT Object Implementation

Sources: cmd/object-handlers.go900-1500

PutObjectHandler Processing cmd/object-handlers.go900-1500:

1. Authentication: Verify s3:PutObject permission
2. Header Extraction: Parse Content-Type, Content-Encoding, Cache-Control, Content-Disposition, user metadata (x-amz-meta-*)
3. Content Validation: Verify Content-MD5 checksum if provided in request
4. Storage Class: Determine storage class from x-amz-storage-class header (STANDARD or REDUCED_REDUNDANCY)
5. Encryption: Apply SSE-S3 (x-amz-server-side-encryption: AES256), SSE-KMS (aws:kms), or SSE-C (customer-provided key)
6. Object Lock: Process x-amz-object-lock-mode and x-amz-object-lock-retain-until-date for retention, x-amz-object-lock-legal-hold for legal hold
7. Hash Calculation: Compute MD5 and SHA256 for integrity
8. Write Operation: Delegate to objectAPI.PutObject() which triggers erasure coding
9. Event Notification: Send ObjectCreated:Put event to configured targets
10. Response Generation: Return ETag in quotes and version ID if versioning enabled

Metadata Handling cmd/object-handlers.go310-325:

• Standard Headers: Extracted from specific HTTP headers (Content-Type, Cache-Control, etc.)
• User Metadata: Headers prefixed with x-amz-meta- (maximum 2KB total size)
• System Metadata: Internal metadata like encryption keys, erasure coding parameters
• Object Tags: Via x-amz-tagging header (up to 10 tags, 128 char keys, 256 char values)

Sources: cmd/object-handlers.go900-1500 cmd/object-handlers.go310-325

DELETE Object Implementation

Sources: cmd/object-handlers.go1800-2000

DeleteObjectHandler Logic cmd/object-handlers.go1800-2000:

1. Authentication: Verify s3:DeleteObject or s3:DeleteObjectVersion permission
2. Version ID Parsing: Extract versionId query parameter if present
3. Object Lock Validation: Check if object has active retention or legal hold; if so, return error
4. Versioning Behavior:
   • Versioning disabled: Permanently delete object from storage
   • Versioning enabled, no version ID: Create new delete marker (soft delete)
   • Versioning enabled, with version ID: Permanently delete specified version
5. Response Headers: Set x-amz-version-id (version deleted or delete marker created), x-amz-delete-marker (true/false)
6. Event Notification: Send ObjectRemoved:Delete or ObjectRemoved:DeleteMarkerCreated event
7. HTTP Status: Return 204 No Content on success

Sources: cmd/object-handlers.go1800-2000

HEAD Object Implementation

HeadObjectHandler cmd/object-handlers.go2200-2400:

Returns all object metadata without transferring the object body:

1. Authentication: Verify s3:GetObject permission
2. Metadata Retrieval: Call objectAPI.GetObjectInfo() to fetch object metadata
3. Precondition Evaluation: Check If-Match, If-None-Match, If-Modified-Since, If-Unmodified-Since
4. Metadata Decryption: Decrypt encryption metadata if object is encrypted
5. Header Setting: Populate all response headers (Content-Length, ETag, Last-Modified, encryption headers, object lock headers, user metadata)
6. Response: Return 200 OK with headers only, empty body

Response Headers (from setObjectHeaders() cmd/object-api-utils.go1015-1262):

• Content-Length: Object size in bytes
• Content-Type: MIME type from metadata or binary/octet-stream default
• ETag: Object checksum (MD5 for non-encrypted, format varies for encrypted/multipart)
• Last-Modified: Last modification time in RFC 1123 format
• x-amz-version-id: Version ID if versioning enabled
• x-amz-server-side-encryption-*: Encryption headers (algorithm, key ID, context)
• x-amz-object-lock-mode: Retention mode (GOVERNANCE or COMPLIANCE)
• x-amz-object-lock-retain-until-date: Retention expiry date
• x-amz-object-lock-legal-hold: Legal hold status (ON or OFF)
• x-amz-tagging-count: Number of tags attached to object
• x-amz-meta-*: All user-defined metadata headers
• Cache-Control, Content-Encoding, Content-Language, Content-Disposition: Standard HTTP headers
• x-amz-replication-status: Replication status if applicable
• x-amz-storage-class: Storage class (STANDARD, REDUCED_REDUNDANCY, GLACIER, etc.)

Sources: cmd/object-handlers.go746-983 cmd/object-api-utils.go1015-1262

Multipart Upload

Multipart upload enables uploading large objects in independent parts, supporting parallel uploads and failure recovery.

Multipart Upload Workflow

Sources: cmd/object-handlers.go2600-4000

Multipart Operations

NewMultipartUploadHandler cmd/object-handlers.go2600-2800:

• Initiates multipart upload session
• Generates unique upload ID (UUID-based)
• Stores metadata (encryption keys, content-type, user metadata)
• Returns upload ID in XML: <UploadId>xyz</UploadId>

PutObjectPartHandler cmd/object-handlers.go2900-3200:

• Uploads individual part (5 MB minimum, 5 GB maximum per part)
• Validates part number (1-10,000 range)
• Computes MD5 ETag for part
• Stores part data in temporary location (.minio.sys/tmp/uploadId/partNumber)
• Returns ETag in response header for client tracking

CompleteMultipartUploadHandler cmd/object-handlers.go3300-3500:

• Receives XML with list of part numbers and ETags
• Validates all required parts present
• Verifies ETags match uploaded parts
• Concatenates parts in order
• Creates final xl.meta with combined metadata
• Deletes temporary part files
• Returns final object ETag (format: {partETag1}-{partCount})

AbortMultipartUploadHandler cmd/object-handlers.go3600-3700:

• Cancels in-progress multipart upload
• Deletes all uploaded parts from storage
• Frees disk space
• Returns 204 No Content

ListMultipartUploadsHandler cmd/bucket-handlers.go243-293:

• Lists all in-progress multipart uploads in bucket
• Supports pagination via key-marker and upload-id-marker
• Returns upload IDs, keys, initiator, owner, initiated timestamp
• XML format: <ListMultipartUploadsResult>

ListObjectPartsHandler cmd/object-handlers.go3800-4000:

• Lists parts for specific multipart upload
• Supports pagination via part-number-marker
• Returns part numbers, ETags, sizes, last modified dates
• XML format: <ListPartsResult>

Sources: cmd/object-handlers.go2600-4000 cmd/bucket-handlers.go243-293

Bucket Operations

Core Bucket Handlers

Handler	HTTP Method	Purpose	Key Functionality
ListBucketsHandler	GET /	List all buckets	Returns bucket names and creation dates
HeadBucketHandler	HEAD /{bucket}	Check bucket exists	Returns 200 if exists, 404 if not
GetBucketLocationHandler	GET /{bucket}?location	Get bucket region	Returns region (empty for MinIO)
PutBucketHandler	PUT /{bucket}	Create bucket	Creates bucket, validates DNS, initializes versioning
DeleteBucketHandler	DELETE /{bucket}	Delete bucket	Removes empty bucket, fails if non-empty
ListObjectsV1Handler	GET /{bucket}?delimiter=	List objects v1	Legacy listing with marker-based pagination
ListObjectsV2Handler	GET /{bucket}?list-type=2	List objects v2	Modern listing with continuation token
GetBucketVersioningHandler	GET /{bucket}?versioning	Get versioning status	Returns Enabled/Suspended/not set
PutBucketVersioningHandler	PUT /{bucket}?versioning	Enable versioning	Enables or suspends versioning
GetBucketPolicyHandler	GET /{bucket}?policy	Get bucket policy	Returns IAM policy JSON
PutBucketPolicyHandler	PUT /{bucket}?policy	Set bucket policy	Validates and applies IAM policy

Sources: cmd/bucket-handlers.go1-1500

Bucket Listing Flow

Sources: cmd/bucket-handlers.go800-1100

ListObjectsV2Handler cmd/bucket-handlers.go800-1100:

Query Parameters:

• prefix: Filter objects starting with this prefix
• delimiter: Group keys by common prefix (typically / for directory simulation)
• max-keys: Maximum objects to return (default 1000, max 1000)
• continuation-token: Opaque token to resume listing from previous page
• start-after: Begin listing after this key lexicographically
• fetch-owner: Include owner information (MinIO extension)

Response Structure:

• Contents[]: Array of objects with Key, Size, ETag, LastModified, StorageClass
• CommonPrefixes[]: Grouped prefixes when delimiter used (simulates directories)
• KeyCount: Total number of keys returned
• MaxKeys: Maximum keys requested
• IsTruncated: Boolean indicating more results available
• NextContinuationToken: Token for retrieving next page
• Prefix: The prefix filter applied
• Delimiter: The delimiter used for grouping

Performance Optimization: Uses metacache system to cache directory listings, significantly accelerating repeated list operations. Cache invalidated on write operations.

Sources: cmd/bucket-handlers.go800-1100

Error Handling

MinIO converts internal errors to S3-compatible error responses using the errorCodes map cmd/api-errors.go482-1400

Error Response Structure

Errors are returned as XML via writeErrorResponse() cmd/api-response.go644-719:

Common Error Codes

Error Code	HTTP Status	Condition
NoSuchKey	404	Object does not exist
NoSuchBucket	404	Bucket does not exist
NoSuchUpload	404	Multipart upload ID not found
AccessDenied	403	Authentication failed or insufficient permissions
InvalidAccessKeyId	403	Access key not found in IAM system
SignatureDoesNotMatch	403	Request signature validation failed
InvalidDigest	400	Content-MD5 header does not match computed MD5
EntityTooLarge	400	Object size exceeds maximum (5TB)
EntityTooSmall	400	Object smaller than minimum for operation
InvalidArgument	400	Invalid parameter value
InvalidBucketName	400	Bucket name violates naming rules
MethodNotAllowed	405	HTTP method not supported for resource
PreconditionFailed	412	Conditional request failed
InternalError	500	Server-side error
ServiceUnavailable	503	Server temporarily overloaded or maintenance

Sources: cmd/api-errors.go84-250

Error Handling Pipeline

Sources: cmd/object-api-errors.go30-300 cmd/api-errors.go482-1400 cmd/api-response.go644-719

Error Translation Steps:

1. Storage Error: ObjectLayer returns typed error (e.g., ObjectNotFound{} from cmd/object-api-errors.go301-421)
2. Convert to API Error: Call toAPIError(ctx, err) cmd/object-api-errors.go30-300 which maps errors to APIErrorCode
3. Get Error Details: Call errorCodes.ToAPIErr(code) cmd/api-errors.go476-478 to get APIError struct with HTTP status and message
4. Encode Response: Call writeErrorResponse() cmd/api-response.go644-719 to marshal XML and write to client

Error Code Examples cmd/api-errors.go84-455:

• ErrNoSuchKey (404): Object does not exist
• ErrNoSuchBucket (404): Bucket does not exist
• ErrAccessDenied (403): Insufficient permissions
• ErrSignatureDoesNotMatch (403): Invalid request signature
• ErrInvalidDigest (400): Content-MD5 mismatch
• ErrInternalError (500): Unexpected server error

APIErrorResponse Fields cmd/api-errors.go64-76:

• Code: S3 error code string
• Message: Human-readable description
• RequestID: From x-amz-request-id header
• HostID: From x-amz-id-2 header
• Resource, BucketName, Key: Request context

Sources: cmd/object-api-errors.go30-421 cmd/api-errors.go54-1400 cmd/api-response.go644-719

Response Generation

Standard Response Headers

Common Response Headers (set by setObjectHeaders() cmd/object-api-utils.go1015-1262):

Header	Purpose	Example
Content-Type	MIME type	image/jpeg
Content-Length	Object size	1048576
ETag	Object checksum	"5d41402abc4b2a76b9719d911017c592"
Last-Modified	Modification time	Wed, 15 Nov 2023 12:00:00 GMT
x-amz-version-id	Version ID	NjhkNDQ5M2EtY2U3NC00
x-amz-request-id	Request ID	F2A8BABEEC3F
x-amz-id-2	Deployment ID	From globalDeploymentID()

Encryption Headers (from cmd/object-handlers.go506-521):

• x-amz-server-side-encryption: AES256 (SSE-S3) or aws:kms (SSE-KMS)
• x-amz-server-side-encryption-aws-kms-key-id: KMS key ID
• x-amz-server-side-encryption-customer-algorithm: SSE-C algorithm
• x-amz-server-side-encryption-customer-key-md5: SSE-C key MD5

Object Lock Headers (from cmd/object-handlers.go210-216):

• x-amz-object-lock-mode: GOVERNANCE or COMPLIANCE
• x-amz-object-lock-retain-until-date: Retention expiry (ISO 8601)
• x-amz-object-lock-legal-hold: ON or OFF

Replication Headers:

• x-amz-replication-status: PENDING, COMPLETED, FAILED, REPLICA
• x-amz-delete-marker: true or false

Sources: cmd/object-api-utils.go1015-1262 cmd/object-handlers.go210-216 cmd/object-handlers.go506-521

Response Generation Functions

writeSuccessResponseXML() cmd/api-response.go721-737:

• Encodes struct to XML via encodeResponse()
• Sets Content-Type: application/xml
• Writes to http.ResponseWriter
• Used for ListObjects, ListBuckets, etc.

encodeResponse() cmd/api-response.go739-772:

• Marshals struct to XML using xxml.Marshal()
• Returns []byte for writing
• Can encode custom XML types

setObjectHeaders() cmd/object-api-utils.go1015-1262:

• Sets standard object headers (Content-Type, Content-Length, ETag)
• Handles Range requests (206 Partial Content)
• Adds storage class, versioning, encryption headers
• Called from GetObjectHandler and HeadObjectHandler

writeErrorResponse() cmd/api-response.go644-719:

• Encodes APIError to XML
• Sets HTTP status code from APIError.HTTPStatusCode
• Adds request/host ID headers

Sources: cmd/api-response.go644-772 cmd/object-api-utils.go1015-1262

Advanced S3 Features

Server-Side Copy

CopyObjectHandler cmd/object-handlers.go1600-1800:

Copies objects entirely server-side without client download/upload:

1. Source Parsing: Extract source bucket and object from x-amz-copy-source header
2. Authorization: Verify s3:GetObject on source, s3:PutObject on destination
3. Metadata Directive:
   • COPY: Preserve original object metadata
   • REPLACE: Use metadata from copy request headers
4. Efficient Copy: Storage layer performs block-level copy without re-encoding
5. Range Copy: Supports x-amz-copy-source-range for partial object copy

Use Cases:

• Rename object (copy to new key, delete old key)
• Update object metadata (copy to self with REPLACE directive)
• Copy between buckets
• Copy between storage classes

Sources: cmd/object-handlers.go1600-1800

SelectObjectContent (S3 Select)

SelectObjectContentHandler cmd/object-handlers.go100-311:

Execute SQL queries directly on objects without downloading:

1. Request Parsing: Extract SQL expression, input/output serialization from XML request body
2. Format Detection: Support CSV, JSON, Parquet formats
3. SQL Engine: Use s3select package to parse and execute SQL
4. Streaming Execution: Process object in chunks, stream filtered results
5. Result Encoding: Return results in requested format (CSV or JSON)

Supported SQL:

• SELECT with column selection and wildcard (*)
• WHERE clauses with comparison operators
• Aggregate functions: COUNT, SUM, AVG, MIN, MAX
• LIMIT clause for row limiting

Sources: cmd/object-handlers.go100-311

Presigned URLs

Presigned URLs provide temporary authenticated access without exposing credentials.

Client-Side Generation:

1. Construct canonical request string
2. Compute HMAC-SHA256 signature using secret key
3. Append signature to URL as query parameters
4. Set expiry timestamp (maximum 7 days)

Server-Side Validation cmd/handler-utils.go200-400:

1. Extract signature from X-Amz-Signature query parameter
2. Recompute expected signature from request details
3. Perform constant-time comparison to prevent timing attacks
4. Check expiry via X-Amz-Date and X-Amz-Expires parameters
5. Validate that requested action matches signed action
6. Return ErrExpiredPresignRequest if expired

Sources: cmd/handler-utils.go200-400

PostObject (HTML Form Upload)

PostObjectHandler cmd/object-handlers.go4200-4600:

Supports browser-based uploads via HTML forms:

1. Parse Form: Extract multipart/form-data fields
2. Policy Document: Parse and decode Base64-encoded policy JSON
3. Policy Validation: Check conditions (bucket, key pattern, size limits, expiration)
4. Signature Verification: Validate form signature matches policy
5. File Upload: Extract file field and upload to storage
6. Success Action: Redirect to success_action_redirect URL or return success_action_status code

Policy Conditions:

• bucket: Exact bucket name
• key: Object key (supports ${filename} variable)
• content-length-range: Min and max size
• x-amz-*: Metadata and header constraints

Sources: cmd/object-handlers.go4200-4600

Performance Optimizations

HTTP Connection Pooling

globalInternodeTransport cmd/globals.go100-200:

• Maintains persistent HTTP connections between cluster nodes
• Configures MaxIdleConnsPerHost for connection reuse
• Enables HTTP/2 when TLS configured
• Reduces connection setup overhead for node-to-node RPCs

Response Compression

gzhttp.GzipHandler cmd/api-router.go50-100:

• Automatically compresses responses when client sends Accept-Encoding: gzip
• Skips compression for responses smaller than 1KB (overhead not worthwhile)
• Skips compression for range requests (partial content)
• Reduces bandwidth usage and improves transfer speed for large responses

Request Concurrency Limiting

maxClients() cmd/generic-handlers.go400-450:

• Limits concurrent requests per API endpoint type
• Uses counting semaphore pattern with buffered channels
• Returns 503 Service Unavailable when limit exceeded
• Prevents resource exhaustion from excessive concurrent requests
• Configurable per-API limits

Metadata Caching

Metacache System:

• Caches compressed directory listings in memory and on disk
• Significantly accelerates repeated ListObjects operations
• Automatically invalidates cache entries on object writes
• Bloom filters for fast negative lookups
• Reduces disk I/O for frequently accessed buckets

Sources: cmd/globals.go100-200 cmd/api-router.go50-100 cmd/generic-handlers.go400-450

Summary

MinIO's S3 API implementation provides comprehensive Amazon S3 compatibility through a well-structured handler architecture:

Core Architecture:

• Handler Layer: objectAPIHandlers with methods for all S3 operations
• Routing: Gorilla Mux-based routing with query parameter-aware endpoint matching
• Middleware: Layered processing for panic recovery, CORS, tracing, metrics, authentication, and compression
• Interface: ObjectLayer abstraction decouples API from storage implementation

Key Operations:

• Object Operations: GET, PUT, DELETE, HEAD with full support for ranges, conditionals, encryption
• Bucket Operations: List, create, delete, with versioning and policy management
• Multipart Upload: Complete workflow from initiation through part upload to completion
• Advanced Features: Server-side copy, S3 Select, presigned URLs, browser POST uploads

Cross-Cutting Concerns:

• Authentication: AWS Signature V2/V4, presigned URLs, STS tokens, anonymous access
• Error Handling: S3-compatible error codes with detailed XML responses
• Performance: Connection pooling, response compression, concurrency limiting, metadata caching

The implementation maintains strict S3 API compatibility while leveraging MinIO's distributed erasure-coded storage for high availability and data durability.

Sources: cmd/object-handlers.go1-4600 cmd/bucket-handlers.go1-1500 cmd/api-router.go1-500 cmd/api-errors.go1-1500 cmd/api-response.go1-500 cmd/generic-handlers.go1-500 cmd/handler-utils.go1-500