diff --git a/docs/prd.md b/docs/prd.md index 971abe2..726206f 100644 --- a/docs/prd.md +++ b/docs/prd.md @@ -622,6 +622,61 @@ val coder = agent("coder") { All callbacks are synchronous — they execute inline before the agentic loop continues. For async telemetry, emit to a channel inside the callback. +### Context Composition Breakdown *(planned — [#5138](https://github.com/Deep-CodeAI/Agents.KT/issues/5138))* + +Budgets cap the window; the **context breakdown** shows *what fills it*. Before each model call the +runtime can report the token composition of the outgoing request, attributed to the framework construct +that produced each slice. It is the runtime companion to the permission manifest: the manifest says what +*can* enter context (the capability graph); the breakdown says what *did*, per turn, and how close it is +to the model's window. + +```kotlin +val breakdown: ContextBreakdown = coder.contextBreakdown(spec) // dry-run — composes the request, no model call +println(breakdown.render()) // the table below +// …or observe it live, once per turn, right before each request goes out: +coder.observe { e -> if (e is PipelineEvent.ContextComposed) report(e.breakdown) } +``` + +``` +Context Usage 41.6K / 200K (21%) + System prompt agent.prompt + selected skill base 0.5K + Tool definitions KSP/@Generable + MCP tool schemas 8.3K + Skills descriptions + knowledge index 0.4K + Knowledge eager toLlmContext() content 3.5K + Memory MemoryBank entries in-window 2.5K + MCP remote tool/prompt/resource defs 0.8K + Sub-agent defs delegate / handoff tool schemas 0.8K + Conversation agentic-loop message history 24.8K +``` + +**Category taxonomy** — each slice maps to the construct that emitted it, so a fat number points at a +fixable cause (a chatty tool schema, an over-eager knowledge pack, an un-compacted history): + +| Category | Source in Agents.KT | +|---|---| +| System prompt | `agent.prompt` + the selected skill's eager `toLlmContext()` base | +| Tool definitions | KSP / `@Generable` JSON schemas for the skill's tool allowlist | +| Skills | skill `description` / `llmDescription` + knowledge-index headers | +| Knowledge | `Skill.knowledge(...)` content injected eagerly (not the lazy knowledge-tool path) | +| Memory | `MemoryBank` entries currently in-window, per typed namespace (§8.5) | +| MCP | tool / prompt / resource definitions from connected MCP servers (§5.8) | +| Sub-agent defs | delegate / `handoff` / Forum-member tool schemas (composition, §7) | +| Conversation | the multi-turn message history the agentic loop accumulates | + +**Typed surface.** `ContextBreakdown` is a value — `slices: Map`, `total`, +`contextWindow` (the model's limit), `fraction` — where `ContextCategory` is a sealed enum of the rows +above. Counts are **estimated** pre-flight (reusing the memory layer's `estimateTokens`, §8.5) and +**reconciled** against provider `TokenUsage` after the call when the provider reports prompt-token +detail. Cache-eligible slices (the stable prefix) are flagged, so the breakdown doubles as the input to +prompt-caching decisions (#2655). + +**Why it earns its place.** (1) **Cost/latency** — the biggest per-run lever is a bloated stable prefix, +and tool schemas + eager knowledge are the usual silent hogs; you can't shrink what you can't see. +(2) **Caching (#2655)** — it names exactly which slices form the cacheable prefix. (3) **Compaction +(§5.7)** — it tells the compactor *what* to compact, not just *that* the window is over threshold. +(4) **Audit** — composition-per-turn is reviewable evidence, joinable to the manifest hash: "what did +this agent send, and what was it made of," squarely on the boundaries-you-can-measure line. + --- ## 5.7 Session Model diff --git a/docs/roadmap.md b/docs/roadmap.md index 0efdc8e..66725ad 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -114,6 +114,7 @@ The 0.6.0 epic ([#1911](../../issues/1911)) tracks the full acceptance criteria. - [x] `:agents-kt-otel` — OpenTelemetry adapter using the GenAI semantic conventions: skill = root span (`agent.invoke`), model turn = child span (`gen_ai.operation.name=chat`, `gen_ai.system`, token-usage attrs), tool call = child span (`gen_ai.operation.name=tool`, `tool.name`, `tool.call.id`), errors as span status, and interceptor decisions as span events. Parent-context propagation via `Context.current()`. ([#1908](../../issues/1908)) - [x] `:agents-kt-langsmith` — LangSmith run-tree adapter (chain → llm → tool runs), async batch dispatch with oldest-drop backpressure logging, and fixture-style tests with no live LangSmith calls. ([#1909](../../issues/1909)) - [x] `:agents-kt-langfuse` — Langfuse traces / spans / generations adapter with native ingestion, async batch dispatch, oldest-drop backpressure logging, and fixture-style tests with no live Langfuse calls. ([#1910](../../issues/1910)) +- [ ] **Context composition breakdown** — per-turn, per-category token accounting of *what fills the model window* (system prompt / tool defs / skills / knowledge / memory / MCP / sub-agent defs / conversation), as a typed `ContextBreakdown` via `agent.contextBreakdown(input)` (dry-run) and `PipelineEvent.ContextComposed` (live, on the existing `observe { }` / `ObservabilityBridge`). The runtime companion to the permission manifest — the manifest says what *can* enter context, the breakdown says what *did*, joinable to the manifest hash for audit. Feeds prompt-caching (names the cacheable stable prefix, [#2655](../../issues/2655)) and compaction (*what* to compact, not just *that* the window is full); estimated pre-flight via the memory layer's `estimateTokens`, reconciled against provider `TokenUsage`. See PRD §5.6. ([#5138](../../issues/5138)) - [x] **Threat-model + deployment-pattern guide** — **shipped**: `docs/threat-model.md` (now the canonical what's-enforced-where table) with worked scenarios (safe local assistant; internal business tool; MCP server behind gateway; anti-patterns), each calling out which Agents.KT guardrails apply and which gaps the deployer must close themselves. Linked from README security section and `SECURITY.md`. ([#1904](../../issues/1904)) - [ ] **Release-signing hardening** — replace the no-passphrase GPG example in the publishing guide with a passphrase-protected default; add a CI-signing section (secrets-manager-injected passphrase, short-lived subkey, or OIDC-to-signing-service); demote the no-protection variant to a clearly-labelled "local-only sandbox keys" subsection. ([#1905](../../issues/1905)) - [ ] **Post-release 0.6.x demos** — *(1)* safe MCP filesystem agent (read-only allowlist, rejection visible in audit log), *(2)* typed approval workflow for high-risk paths (the shipped primitives: `humanApproval { }` → `resumeWith(HumanDecision)` #2489, or `HumanGateRegistry` #3868 — the originally-sketched `Escalate` decision type was never shipped), *(3)* multi-agent audit pipeline binding every model + tool call to the manifest hash. Each lives in `examples//`, runs against Ollama by default, emits manifest + JSONL audit on one invocation. Validates the 0.6.0 story end-to-end. ([#1918](../../issues/1918))