New serverless pattern - lambda-microvms-multi-tenant-ai-agents

lambda-microvms-multi-tenant-ai-agents/.gitignore

@@ -0,0 +1 @@
+.gateway-token

lambda-microvms-multi-tenant-ai-agents/README.md

@@ -0,0 +1,105 @@
+# Multi-tenant AI agents on AWS Lambda MicroVMs
+
+This pattern runs a self-hosted AI agent ([OpenClaw](https://github.com/openclaw/openclaw)) **one isolated Lambda MicroVM per tenant**, with per-tenant state persisted on Amazon EFS, model calls served by Amazon Bedrock through a VPC endpoint, and an orchestrator Lambda behind API Gateway that cold-starts, resumes, and reaps tenant VMs on demand.
+
+Most AI agents sit idle most of the time, yet an "always-on" deployment bills 24/7. Lambda MicroVMs flip that model: an idle tenant's VM auto-suspends (barely billed), a fully idle tenant is terminated with its state parked on EFS for ≈$0, and a returning tenant resumes from a Firecracker snapshot in seconds — memory intact. Each tenant gets a dedicated micro-VM, so isolation is a hard security boundary by design, and the workload obtains AWS credentials from the MicroVM's IMDSv2 execution role with no static keys.
+
+Learn more about this pattern at Serverless Land Patterns: << Add the live URL here >>
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI v2 >= 2.35](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured — must include the `lambda-microvms` and `lambda-core` subcommands (check with `aws lambda-microvms help`)
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* Python 3 and [uv](https://docs.astral.sh/uv/) (or a recent pip), plus the `zip` and `openssl` utilities
+* A [Lambda MicroVMs launch region](https://docs.aws.amazon.com/lambda/latest/dg/lambda-microvms.html) (us-east-1, us-east-2, us-west-2, eu-west-1, ap-northeast-1)
+* Amazon Bedrock **model access for Anthropic Claude** enabled in the target region
+
+Docker is **not** required locally — the container image build runs on AWS as part of the `AWS::Lambda::MicrovmImage` resource.
+
+## Deployment Instructions
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ```
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+1. Change directory to the pattern directory:
+    ```
+    cd serverless-patterns/lambda-microvms-multi-tenant-ai-agents
+    ```
+1. Deploy the whole system with one command (takes ~10 minutes: CloudFormation stack + server-side MicroVM image build + VPC egress connector):
+    ```
+    ./deploy.sh <stack-name> <region>
+    # e.g. ./deploy.sh openclaw-mt us-east-1
+    ```
+    The script pre-flights the CLI and target region, uploads two zip artifacts to S3 (the MicroVM image source and the bundled orchestrator Lambda — both under content-hashed keys, so a code change always reaches AWS and an unchanged redeploy is a true no-op) and then runs a single `aws cloudformation deploy`. Everything else — VPC with NAT internet egress, EFS, Bedrock VPC endpoints (runtime + control plane), IAM roles, DynamoDB tenant registry, the MicroVM image (built server-side by CloudFormation via `AWS::Lambda::MicrovmImage`), the VPC egress connector, the orchestrator Lambda, API Gateway, and the EventBridge sweeper — is declared in `template.yaml`. A random per-checkout gateway token is minted into `.gateway-token` (override with `$GATEWAY_TOKEN`).
+1. Register a tenant in the DynamoDB registry:
+    ```
+    ./add-tenant.sh <stack-name> <region> tenant1
+    ```
+1. Note the `ApiEndpoint` output printed at the end of the deploy. Tenant webhook URLs take the form `<ApiEndpoint>/tg/<tenantId>`.
+
+## How it works
+
+![Architecture](images/architecture.png)
+
+1. **Image build**: CloudFormation creates the `AWS::Lambda::MicrovmImage` resource. Lambda downloads the zip artifact from S3, executes the Dockerfile server-side (installs the OpenClaw agent, a sidecar, an EFS mount daemon, and a persistent gateway bridge), and takes a Firecracker snapshot.
+2. **Message routing**: a message for a tenant arrives at API Gateway and invokes the orchestrator Lambda (router role). The router ACKs immediately and hands the message to an async worker invocation.
+3. **Tenant lookup**: the worker checks the DynamoDB tenant registry. If the tenant's MicroVM is alive (RUNNING or SUSPENDED — suspended VMs auto-resume in seconds), the turn is forwarded. If the tenant is cold, the worker calls `run-microvm` (guarded by a conditional-write lock so concurrent messages launch only one VM), waits for the VM to mount the tenant's EFS subdirectory, and then forwards the turn.
+4. **Agent turn**: inside the MicroVM, a sidecar receives the turn and passes it to the OpenClaw gateway over a persistent WebSocket bridge — the gateway holds the agent state warm in memory, so a turn takes ~2 seconds instead of re-reading state over NFS on every message. If a long-running turn outlives the worker invocation, the worker relays polling to a fresh async self-invocation, so a turn is bounded by the VM's 8-hour lifetime rather than Lambda's 15 minutes.
+5. **Model call**: the agent calls Amazon Bedrock through VPC interface endpoints (runtime for inference, control plane for live model discovery at cold start). General internet egress for the agent's web search/fetch tools goes through a NAT gateway, while Bedrock and EFS traffic stays on the private VPCE/mount-target paths. Credentials come from the MicroVM's IMDSv2 execution role — no static keys anywhere.
+6. **State persistence**: the tenant's full agent state (config + conversation memory) lives under `/tenants/<id>` on EFS. It survives suspend, resume, termination, and the MicroVM's 8-hour maximum lifetime — a relaunched VM adopts the existing state and the agent remembers everything.
+7. **Lifecycle sweep**: an EventBridge rule invokes the orchestrator (sweeper role) every 10 minutes to terminate VMs idle beyond the threshold and reconcile the registry against ground truth. Tenants flow hot (RUNNING) → warm (SUSPENDED) → cold (TERMINATED, state on EFS) automatically.
+
+## Testing
+
+Chat with a tenant synchronously (bypasses API Gateway's 30s timeout so you can watch a cold start, which takes ~90 seconds; subsequent warm turns take ~2 seconds):
+
+```bash
+./chat.sh <stack-name> <region> tenant1 "Remember my lucky number is 7777."
+# → cold: True  | reply: (agent confirms)
+
+./chat.sh <stack-name> <region> tenant1 "What's my lucky number?"
+# → cold: False | reply: 7777
+```
+
+To verify cross-generation persistence, terminate the tenant's MicroVM (`aws lambda-microvms terminate-microvm ...` or simply wait for the sweeper), then ask again — the relaunched VM adopts the EFS state and still answers `7777` with `cold: True`.
+
+Tenant isolation: register a second tenant and confirm it cannot see the first tenant's memory:
+
+```bash
+./add-tenant.sh <stack-name> <region> tenant2
+./chat.sh <stack-name> <region> tenant2 "What's my lucky number?"
+# → the agent does not know — tenant2 has its own EFS subdirectory and its own VM
+```
+
+### Optional: Telegram push front-end
+
+Each tenant can be bound to its own Telegram bot. Create a bot with [@BotFather](https://t.me/BotFather) (`/newbot` → token), then register the tenant with the token and a webhook secret of your choosing:
+
+```bash
+./add-tenant.sh <stack-name> <region> tenant3 <BOT_TOKEN> <WEBHOOK_SECRET>
+```
+
+The script sets the bot's webhook to `<ApiEndpoint>/tg/tenant3`. Messaging the bot then drives the same router → worker → MicroVM flow, and replies are delivered back through the Telegram Bot API. Use a dedicated bot per tenant — registering overwrites the bot's existing webhook.
+
+Over Telegram the worker additionally provides:
+
+* **Streaming replies** — a placeholder message that grows via `editMessageText` while the model generates, with a `▌` cursor until the final edit.
+* **Images** — send a photo with or without a caption; the worker pulls it from Telegram, ships it into the VM as a base64 attachment, and the agent answers about what it sees.
+* **`/model` switching** — e.g. `/model amazon-bedrock/us.anthropic.claude-sonnet-5`, `/model default` to reset. The model catalog is discovered live from Bedrock at each cold start, so newly launched models are switchable without a redeploy.
+
+## Cleanup
+
+```bash
+./teardown.sh <stack-name> <region>
+```
+
+The script terminates this stack's running MicroVMs (matched by image ARN, so VMs from other stacks are untouched), deletes the CloudFormation stack (which removes the MicroVM image, network connector, EFS, VPC, IAM roles, DynamoDB table, Lambda, and API Gateway), and finally empties and deletes the artifact bucket.
+
+----
+Copyright 2026 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0

lambda-microvms-multi-tenant-ai-agents/add-tenant.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+# Register a tenant. For Telegram push, also pass a bot token + secret and this
+# script sets the webhook to <api>/tg/<tenantId>.
+# Usage: ./add-tenant.sh STACK REGION TENANT_ID [BOT_TOKEN] [WEBHOOK_SECRET]
+set -euo pipefail
+if [ $# -lt 3 ]; then
+  echo "usage: $0 STACK REGION TENANT_ID [BOT_TOKEN] [WEBHOOK_SECRET]" >&2; exit 1
+fi
+STACK="$1"; REGION="$2"; TID="$3"
+BOT="${4:-}"; SECRET="${5:-}"
+API="$(aws cloudformation describe-stacks --region "$REGION" --stack-name "$STACK" \
+  --query "Stacks[0].Outputs[?OutputKey=='ApiEndpoint'].OutputValue" --output text)"
+
+ITEM="{\"tenantId\":{\"S\":\"$TID\"},\"state\":{\"S\":\"COLD\"},\"generation\":{\"N\":\"0\"}"
+[ -n "$BOT" ]    && ITEM="$ITEM,\"botToken\":{\"S\":\"$BOT\"}"
+[ -n "$SECRET" ] && ITEM="$ITEM,\"webhookSecret\":{\"S\":\"$SECRET\"}"
+ITEM="$ITEM}"
+aws dynamodb put-item --region "$REGION" --table-name "${STACK}-tenants" --item "$ITEM"
+echo "registered tenant '$TID' (state=COLD)"
+
+if [ -n "$BOT" ] && [ -n "$SECRET" ]; then
+  # Parse Telegram's response instead of dumping raw JSON: a bad bot token must
+  # fail loudly (non-zero), not scroll past as {"ok":false,...}.
+  curl -s "https://api.telegram.org/bot${BOT}/setWebhook" \
+    --data-urlencode "url=${API}/tg/${TID}" \
+    --data-urlencode "secret_token=${SECRET}" \
+    --data-urlencode "drop_pending_updates=true" | python3 -c '
+import json, sys
+d = json.load(sys.stdin)
+if d.get("ok"):
+    print("setWebhook: ok")
+else:
+    print("ERROR: setWebhook failed:", d.get("description", d), file=sys.stderr)
+    print("       Check the bot token (from @BotFather) and try again.", file=sys.stderr)
+    sys.exit(1)
+'
+  echo "webhook -> ${API}/tg/${TID}"
+fi

lambda-microvms-multi-tenant-ai-agents/chat.sh

@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Synchronous test chat for a tenant (cold-starts the VM if needed). Invokes the
+# orchestrator worker directly so it isn't bound by API Gateway's 30s timeout —
+# handy for validating cold starts (~90s) from a terminal.
+# Usage: ./chat.sh STACK REGION TENANT_ID "message" [sessionKey]
+set -euo pipefail
+if [ $# -lt 4 ]; then
+  echo "usage: $0 STACK REGION TENANT_ID \"message\" [sessionKey]" >&2; exit 1
+fi
+STACK="$1"; REGION="$2"; TID="$3"; MSG="$4"; SESS="${5:-cli}"
+FN="${STACK}-orchestrator"
+
+# Heads-up before the blocking invoke: a cold tenant means a ~90s MicroVM cold start,
+# which would otherwise look like a silent hang.
+STATE="$(aws dynamodb get-item --region "$REGION" --table-name "${STACK}-tenants" \
+  --key "{\"tenantId\":{\"S\":\"$TID\"}}" --query 'Item.state.S' --output text 2>/dev/null || true)"
+if [ "$STATE" != "RUNNING" ]; then
+  echo "Tenant is cold — starting MicroVM, first turn takes ~90s ..."
+fi
+PAYLOAD="$(python3 -c 'import json,sys; print(json.dumps({"_worker":{"tenantId":sys.argv[1],"update":{"message":{"chat":{"id":sys.argv[2]},"text":sys.argv[3]}}}}))' "$TID" "$SESS" "$MSG")"
+OUT="$(mktemp)"
+aws lambda invoke --region "$REGION" --function-name "$FN" \
+  --cli-binary-format raw-in-base64-out --payload "$PAYLOAD" \
+  --cli-read-timeout 300 "$OUT" >/dev/null
+python3 -c 'import json,sys; d=json.load(open(sys.argv[1])); print("cold:",d.get("cold"),"| reply:",d.get("reply"))' "$OUT"

lambda-microvms-multi-tenant-ai-agents/deploy.sh

@@ -0,0 +1,143 @@
+#!/usr/bin/env bash
+# End-to-end deploy for multi-tenant OpenClaw on Lambda MicroVMs.
+#
+# CloudFormation does almost everything — including the MicroVM image and the VPC egress
+# connector, which ARE native CFN resources (AWS::Lambda::MicrovmImage /
+# AWS::Lambda::NetworkConnector). The only thing CFN can't do is put *bytes* in S3, so this
+# script's imperative part is just: ensure a bucket + upload the two zips. Then one
+# `cloudformation deploy` builds the image, provisions the connector, and wires the
+# orchestrator Lambda via GetAtt.
+#
+# Prereqs: AWS CLI v2 >= 2.35 (has the lambda-microvms models), python3, uv (or a modern
+# pip), zip. Credentials for a MicroVMs launch region with Bedrock Claude access.
+#
+# Usage:  ./deploy.sh [STACK_NAME] [REGION]
+set -euo pipefail
+
+STACK="${1:-openclaw-mt}"
+REGION="${2:-us-east-1}"
+HERE="$(cd "$(dirname "$0")" && pwd)"
+
+say(){ printf '\n\033[1;36m== %s ==\033[0m\n' "$*"; }
+
+# ---------- 0. Pre-flight (fail fast, before anything is created) ----------
+say "0/4 Pre-flight checks"
+if ! aws lambda-microvms help >/dev/null 2>&1; then
+  echo "ERROR: this AWS CLI has no 'lambda-microvms' subcommands (needs AWS CLI v2 >= 2.35)."
+  echo "       installed: $(aws --version 2>&1)"
+  exit 1
+fi
+# Cheap read against the target region: catches not-yet-launched regions (the launch
+# list keeps growing — probe instead of hardcoding it) and missing credentials.
+if ! ERR="$(aws lambda-microvms list-microvms --region "$REGION" 2>&1 >/dev/null)"; then
+  echo "ERROR: Lambda MicroVMs isn't reachable in region '$REGION':"
+  printf '       %s\n' "$ERR"
+  echo "       If the service hasn't launched in '$REGION' yet, use a launch region and re-run."
+  exit 1
+fi
+echo "  aws cli ok; region '$REGION' ok"
+
+ACCOUNT="$(aws sts get-caller-identity --query Account --output text)"
+# Gateway token: explicit $GATEWAY_TOKEN wins; otherwise reuse .gateway-token, minting
+# it once per checkout (random) so redeploys are idempotent and no shared default ships.
+TOKEN_FILE="$HERE/.gateway-token"
+if [ -z "${GATEWAY_TOKEN:-}" ]; then
+  [ -f "$TOKEN_FILE" ] || openssl rand -hex 16 > "$TOKEN_FILE"
+  GATEWAY_TOKEN="$(cat "$TOKEN_FILE")"
+  TOKEN_NOTE="(kept in ${TOKEN_FILE#$HERE/}; redeploys reuse it)"
+else
+  TOKEN_NOTE="(from \$GATEWAY_TOKEN)"
+fi
+BUCKET="${STACK}-artifact-${ACCOUNT}-${REGION}"
+# S3 keys carry a content hash: CloudFormation only rebuilds the MicroVM image /
+# updates the Lambda when a resource PROPERTY changes, so a fixed key means code
+# changes silently never reach AWS on redeploy. Hash of the inputs fixes that
+# (and makes an unchanged redeploy a true no-op).
+IMG_SRC="Dockerfile openclaw.json hooks.py start.sh efs-monitor.sh gw-bridge.cjs materialize-models.mjs"
+IMG_HASH="$(cd "$HERE/src/microvm" && cat $IMG_SRC | shasum -a 256 | cut -c1-12)"
+IMG_KEY="microvm-images/openclaw-${IMG_HASH}.zip"
+CODE_HASH="$(shasum -a 256 "$HERE/src/orchestrator/handler.py" | cut -c1-12)"
+CODE_KEY="lambda/orchestrator-${CODE_HASH}.zip"
+
+# ---------- 1. Artifact bucket (the one imperative prerequisite) ----------
+say "1/4 Ensure artifact bucket: $BUCKET"
+if ! aws s3api head-bucket --bucket "$BUCKET" 2>/dev/null; then
+  if [ "$REGION" = "us-east-1" ]; then
+    aws s3api create-bucket --bucket "$BUCKET" --region "$REGION" >/dev/null
+  else
+    aws s3api create-bucket --bucket "$BUCKET" --region "$REGION" \
+      --create-bucket-configuration LocationConstraint="$REGION" >/dev/null
+  fi
+  aws s3api put-public-access-block --bucket "$BUCKET" \
+    --public-access-block-configuration BlockPublicAcls=true,BlockPublicPolicy=true,IgnorePublicAcls=true,RestrictPublicBuckets=true
+fi
+
+# ---------- 2. MicroVM image artifact (Dockerfile + app) ----------
+say "2/4 Package & upload MicroVM image artifact ($IMG_KEY)"
+IMGZIP="$(mktemp -d)/microvm.zip"
+( cd "$HERE/src/microvm" && zip -j -q "$IMGZIP" $IMG_SRC )
+aws s3 cp "$IMGZIP" "s3://${BUCKET}/${IMG_KEY}" --region "$REGION"
+
+# ---------- 3. Orchestrator Lambda zip (boto3 + lambda-microvms model overlay + handler) ----------
+say "3/4 Bundle & upload orchestrator Lambda"
+PKG="$(mktemp -d)"
+if command -v uv >/dev/null 2>&1; then
+  uv pip install --quiet --python "$(command -v python3)" --target "$PKG" boto3
+else
+  python3 -m pip install --quiet --upgrade --target "$PKG" boto3 \
+    || { echo "ERROR: install uv (https://docs.astral.sh/uv/) or a recent pip"; exit 1; }
+fi
+# PyPI boto3 may lag the lambda-microvms service model; overlay it from the local AWS CLI's
+# botocore data (the CLI ships it — that's how this very script calls the service).
+CLI_DATA="$(python3 - "$(readlink "$(command -v aws)" || command -v aws)" <<'PY'
+import os,sys,glob
+root=os.path.realpath(sys.argv[1])
+for _ in range(6):
+    root=os.path.dirname(root)
+    hit=glob.glob(os.path.join(root,'**','botocore','data','lambda-microvms'),recursive=True)
+    if hit: print(os.path.dirname(hit[0])); break
+PY
+)"
+if [ -n "$CLI_DATA" ] && [ -d "$PKG/botocore/data" ]; then
+  cp -R "$CLI_DATA/lambda-microvms" "$PKG/botocore/data/" 2>/dev/null || true
+  cp -R "$CLI_DATA/lambda-core"     "$PKG/botocore/data/" 2>/dev/null || true
+fi
+# Hard gate: the bundle MUST contain the model, or the orchestrator only fails at invoke
+# time with an unrelated-looking UnknownServiceError. Fail the deploy here instead.
+if ! python3 -c "import sys;sys.path.insert(0,'$PKG');import boto3;assert 'lambda-microvms' in boto3.session.Session().get_available_services();print('  boto3',boto3.__version__,'+ lambda-microvms model')" 2>/dev/null; then
+  echo "ERROR: couldn't get the 'lambda-microvms' service model into the Lambda bundle."
+  echo "       Bundled boto3 lacks it, and extracting it from the local AWS CLI failed"
+  echo "       (CLI data dir found: '${CLI_DATA:-none}')."
+  echo "       Fix: install AWS CLI v2 >= 2.35 via the official installer, or upgrade"
+  echo "       boto3 on PyPI to a version that ships the lambda-microvms model."
+  exit 1
+fi
+cp "$HERE/src/orchestrator/handler.py" "$PKG/"
+CODEZIP="$(mktemp -d)/orchestrator.zip"; ( cd "$PKG" && zip -q -r "$CODEZIP" . )
+echo "  code key: $CODE_KEY"
+aws s3 cp "$CODEZIP" "s3://${BUCKET}/${CODE_KEY}" --region "$REGION"
+
+# ---------- 4. One CloudFormation deploy: builds image, connector, everything, wired ----------
+say "4/4 CloudFormation deploy (builds MicroVM image ~5min + connector + all infra)"
+aws cloudformation deploy --region "$REGION" --stack-name "$STACK" \
+  --template-file "$HERE/template.yaml" \
+  --capabilities CAPABILITY_NAMED_IAM \
+  --parameter-overrides \
+      ProjectName="$STACK" \
+      GatewayToken="$GATEWAY_TOKEN" \
+      ArtifactBucketName="$BUCKET" \
+      MicrovmImageKey="$IMG_KEY" \
+      OrchestratorCodeKey="$CODE_KEY"
+
+API="$(aws cloudformation describe-stacks --region "$REGION" --stack-name "$STACK" \
+  --query "Stacks[0].Outputs[?OutputKey=='ApiEndpoint'].OutputValue" --output text)"
+
+say "DONE"
+cat <<EOF
+
+  API endpoint : ${API}
+  Gateway token: ${GATEWAY_TOKEN}  ${TOKEN_NOTE}
+  Add a tenant : ./add-tenant.sh ${STACK} ${REGION} <tenantId> [telegramBotToken] [webhookSecret]
+  Test (HTTP)  : ./chat.sh ${STACK} ${REGION} <tenantId> "your message"
+  Teardown     : ./teardown.sh ${STACK} ${REGION}
+EOF