Skip to content

Explore replacing AWS SDK S3 client with lightweight JDK HttpClient approach #144

Description

@gunnarmorling

Context

The hardwood-s3 module currently depends on software.amazon.awssdk:s3, which pulls in 31 transitive JARs totaling ~8 MB. The S3 service client JAR alone is 4 MB. Yet we use exactly 4 types from the SDK:

  • S3Client.getObjectAsBytes() — HTTP GET with Range header
  • GetObjectResponse — to read Content-Range
  • ResponseBytes — to get the response body
  • S3Exception — error handling

The actual I/O is two HTTP operations: a suffix-range GET (for open()) and byte-range GETs (for readRange()). Both are simple HTTP GETs with SigV4 signing.

Options

Approach Deps Credential chain Effort
Status quo (aws-sdk:s3) 8 MB / 31 JARs Full Done
aws-sdk:auth + JDK HttpClient ~2 MB / ~10 JARs Full Medium
Pure JDK, env/profile credentials only 0 extra Partial (no IMDS/ECS) Medium
Pure JDK, full credential chain 0 extra Full Large

Recommended exploration: aws-sdk:auth + JDK HttpClient

Keep the AWS SDK's auth and regions modules for credential resolution and SigV4 signing, but replace the S3 service client with direct HTTP calls via java.net.http.HttpClient. This would:

  • Drop ~6 MB of dependencies (S3 service client, XML/query protocols, checksums, retries, eventstream)
  • Keep the full credential provider chain (env vars, profiles, IMDS, ECS, web identity)
  • Align with Hardwood's minimal-dependency philosophy

The implementation would be a small HTTP wrapper that:

  1. Resolves credentials via AwsCredentialsProvider
  2. Signs requests with SigV4 (Aws4Signer or the newer AwsV4HttpSigner)
  3. Sends GET requests with Range headers via java.net.http.HttpClient
  4. Parses Content-Range from responses

Alternative: pure JDK

If we're willing to limit credential support to environment variables, system properties, and AWS profile files (covering ~90% of real-world usage outside containers), we could implement SigV4 signing directly (~200-300 lines) and have zero external dependencies. EC2/ECS instance metadata (IMDS) is the part that makes a full from-scratch implementation painful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions