This repository is a reference architecture for governing and placing enterprise AI workloads on AWS. Its thesis is simple: model access is only one part of an AI platform; the durable architecture is the control plane around identity, data access, workload placement, agent tools, inference, accelerator capacity, observability, security, reliability, and cost.
The design uses a portfolio approach rather than forcing every workload onto one platform. Amazon Bedrock and Amazon Bedrock AgentCore cover managed model and agent capabilities, Amazon EKS covers custom inference, Amazon SageMaker AI and Amazon SageMaker HyperPod cover managed ML and distributed training, and AWS ParallelCluster + Slurm covers HPC-oriented workloads. AWS Trainium, AWS Inferentia, NVIDIA GPUs, and external capacity are evaluated as accelerator or sourcing choices within that model.
Design boundary: This repository provides reference patterns and decision guidance, not a turnkey deployment or an AWS-endorsed architecture. Validate service capabilities, security controls, quotas, regional availability, performance, and cost against workload-specific requirements before implementation.
| In scope | Out of scope |
|---|---|
| Enterprise identity, authorization, network, data, audit, and cost controls | A turnkey deployable platform |
| Placement across Amazon Bedrock, Amazon Bedrock AgentCore, Amazon EKS, Amazon SageMaker AI, Amazon SageMaker HyperPod, and AWS ParallelCluster + Slurm | A universal recommendation for every AI workload |
| Custom inference, managed agents, RAG, training, batch, HPC, and accelerator choices | Model development tutorials or application-specific prompt design |
| Operational concerns such as SLOs, observability, incident response, capacity, and unit economics | Workload-specific benchmarks, validated service limits, or service guarantees |
| Architecture decisions, tradeoffs, diagrams, and example runbooks | Replacement for AWS documentation, security review, or workload testing |
| Read this | Use it for |
|---|---|
| Start here | Choose a reading path and understand the document set |
| Reference architecture | See the target control plane, workload platforms, and shared governance layers |
| Workload placement matrix | Compare managed services, Kubernetes, ML platforms, HPC, and accelerators |
| Well-Architected mapping | Review the design against AWS Well-Architected concerns |
| Security model | Review shared and platform-specific controls for each workload placement path |
| Observability model | Review metrics, logs, traces, SLOs, showback, and incident signals |
| Cost and capacity model | Review unit economics, routing, accelerator capacity, commitments, and chargeback |
| Architecture decision records | Inspect the major platform and accelerator tradeoffs |
| Runbooks | See example operational response procedures |
| Diagrams | Find architecture diagram notes and assets |
| Local simulation | Explore the local lab area |
| AWS blueprint | Explore the AWS implementation blueprint area |
The repository is an evolving architecture foundation. Some sections are design guidance or placeholders rather than complete implementations; each decision should be tested before adoption.