Skip to content

docs(#5138): context composition breakdown — PRD §5.6 + roadmap Phase-3 #246

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Skobeltsyn wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs(#5138): context composition breakdown — PRD §5.6 + roadmap Phase-3 #246

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions docs/prd.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -622,6 +622,61 @@ val coder = agent<Specification, CodeBundle>("coder") {

All callbacks are synchronous — they execute inline before the agentic loop continues. For async telemetry, emit to a channel inside the callback.

### Context Composition Breakdown *(planned — [#5138](https://github.com/Deep-CodeAI/Agents.KT/issues/5138))*

Budgets cap the window; the **context breakdown** shows *what fills it*. Before each model call the
runtime can report the token composition of the outgoing request, attributed to the framework construct
that produced each slice. It is the runtime companion to the permission manifest: the manifest says what
*can* enter context (the capability graph); the breakdown says what *did*, per turn, and how close it is
to the model's window.

```kotlin
val breakdown: ContextBreakdown = coder.contextBreakdown(spec) // dry-run — composes the request, no model call
println(breakdown.render()) // the table below
// …or observe it live, once per turn, right before each request goes out:
coder.observe { e -> if (e is PipelineEvent.ContextComposed) report(e.breakdown) }
```

```
Context Usage 41.6K / 200K (21%)
System prompt agent.prompt + selected skill base 0.5K
Tool definitions KSP/@Generable + MCP tool schemas 8.3K
Skills descriptions + knowledge index 0.4K
Knowledge eager toLlmContext() content 3.5K
Memory MemoryBank entries in-window 2.5K
MCP remote tool/prompt/resource defs 0.8K
Sub-agent defs delegate / handoff tool schemas 0.8K
Conversation agentic-loop message history 24.8K
```

**Category taxonomy** — each slice maps to the construct that emitted it, so a fat number points at a
fixable cause (a chatty tool schema, an over-eager knowledge pack, an un-compacted history):

| Category | Source in Agents.KT |
|---|---|
| System prompt | `agent.prompt` + the selected skill's eager `toLlmContext()` base |
| Tool definitions | KSP / `@Generable` JSON schemas for the skill's tool allowlist |
| Skills | skill `description` / `llmDescription` + knowledge-index headers |
| Knowledge | `Skill.knowledge(...)` content injected eagerly (not the lazy knowledge-tool path) |
| Memory | `MemoryBank` entries currently in-window, per typed namespace (§8.5) |
| MCP | tool / prompt / resource definitions from connected MCP servers (§5.8) |
| Sub-agent defs | delegate / `handoff` / Forum-member tool schemas (composition, §7) |
| Conversation | the multi-turn message history the agentic loop accumulates |

**Typed surface.** `ContextBreakdown` is a value — `slices: Map<ContextCategory, TokenSlice>`, `total`,
`contextWindow` (the model's limit), `fraction` — where `ContextCategory` is a sealed enum of the rows
above. Counts are **estimated** pre-flight (reusing the memory layer's `estimateTokens`, §8.5) and
**reconciled** against provider `TokenUsage` after the call when the provider reports prompt-token
detail. Cache-eligible slices (the stable prefix) are flagged, so the breakdown doubles as the input to
prompt-caching decisions (#2655).

**Why it earns its place.** (1) **Cost/latency** — the biggest per-run lever is a bloated stable prefix,
and tool schemas + eager knowledge are the usual silent hogs; you can't shrink what you can't see.
(2) **Caching (#2655)** — it names exactly which slices form the cacheable prefix. (3) **Compaction
(§5.7)** — it tells the compactor *what* to compact, not just *that* the window is over threshold.
(4) **Audit** — composition-per-turn is reviewable evidence, joinable to the manifest hash: "what did
this agent send, and what was it made of," squarely on the boundaries-you-can-measure line.

---

## 5.7 Session Model
Expand Down
1 change: 1 addition & 0 deletions docs/roadmap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ The 0.6.0 epic ([#1911](../../issues/1911)) tracks the full acceptance criteria.
- [x] `:agents-kt-otel` — OpenTelemetry adapter using the GenAI semantic conventions: skill = root span (`agent.invoke`), model turn = child span (`gen_ai.operation.name=chat`, `gen_ai.system`, token-usage attrs), tool call = child span (`gen_ai.operation.name=tool`, `tool.name`, `tool.call.id`), errors as span status, and interceptor decisions as span events. Parent-context propagation via `Context.current()`. ([#1908](../../issues/1908))
- [x] `:agents-kt-langsmith` — LangSmith run-tree adapter (chain → llm → tool runs), async batch dispatch with oldest-drop backpressure logging, and fixture-style tests with no live LangSmith calls. ([#1909](../../issues/1909))
- [x] `:agents-kt-langfuse` — Langfuse traces / spans / generations adapter with native ingestion, async batch dispatch, oldest-drop backpressure logging, and fixture-style tests with no live Langfuse calls. ([#1910](../../issues/1910))
- [ ] **Context composition breakdown** — per-turn, per-category token accounting of *what fills the model window* (system prompt / tool defs / skills / knowledge / memory / MCP / sub-agent defs / conversation), as a typed `ContextBreakdown` via `agent.contextBreakdown(input)` (dry-run) and `PipelineEvent.ContextComposed` (live, on the existing `observe { }` / `ObservabilityBridge`). The runtime companion to the permission manifest — the manifest says what *can* enter context, the breakdown says what *did*, joinable to the manifest hash for audit. Feeds prompt-caching (names the cacheable stable prefix, [#2655](../../issues/2655)) and compaction (*what* to compact, not just *that* the window is full); estimated pre-flight via the memory layer's `estimateTokens`, reconciled against provider `TokenUsage`. See PRD §5.6. ([#5138](../../issues/5138))
- [x] **Threat-model + deployment-pattern guide** — **shipped**: `docs/threat-model.md` (now the canonical what's-enforced-where table) with worked scenarios (safe local assistant; internal business tool; MCP server behind gateway; anti-patterns), each calling out which Agents.KT guardrails apply and which gaps the deployer must close themselves. Linked from README security section and `SECURITY.md`. ([#1904](../../issues/1904))
- [ ] **Release-signing hardening** — replace the no-passphrase GPG example in the publishing guide with a passphrase-protected default; add a CI-signing section (secrets-manager-injected passphrase, short-lived subkey, or OIDC-to-signing-service); demote the no-protection variant to a clearly-labelled "local-only sandbox keys" subsection. ([#1905](../../issues/1905))
- [ ] **Post-release 0.6.x demos** — *(1)* safe MCP filesystem agent (read-only allowlist, rejection visible in audit log), *(2)* typed approval workflow for high-risk paths (the shipped primitives: `humanApproval { }` → `resumeWith(HumanDecision)` #2489, or `HumanGateRegistry` #3868 — the originally-sketched `Escalate` decision type was never shipped), *(3)* multi-agent audit pipeline binding every model + tool call to the manifest hash. Each lives in `examples/<name>/`, runs against Ollama by default, emits manifest + JSONL audit on one invocation. Validates the 0.6.0 story end-to-end. ([#1918](../../issues/1918))
Expand Down
Loading