Skip to content

Global chat: Add answer streaming to planner#573

Draft
hanna-paasivirta wants to merge 3 commits into
mainfrom
planner-streaming-bug
Draft

Global chat: Add answer streaming to planner#573
hanna-paasivirta wants to merge 3 commits into
mainfrom
planner-streaming-bug

Conversation

@hanna-paasivirta

Copy link
Copy Markdown
Contributor

Short Description

The planner buffered all model text and flushed it only after the response finished. Users saw nothing while the planner worked. Text the model wrote before tool calls ("I'll build this for you...") was silently discarded.

Fixes #572

Implementation Details

Live streaming (planner.py)

  • Text deltas are now forwarded to the client as they arrive.
  • This includes narration before tool calls. Each round's text lands in its own content block, separated by the existing status events.
  • Model thinking is still never streamed. Only our curated status messages are.
  • Workflow YAML is still sent before the text that references it. It now goes out lazily, right before the first text delta that follows a change.

New response_segments field (planner.py, router.py, global_chat.py)

  • The full transcript of the turn, in stream order: {"type": "text" | "status", "content": "..."}.
  • Lets the frontend re-render the woven stream view after a page reload.
  • On direct routes it's a single text segment wrapping response.

Unchanged contracts

  • response is still a string: the final answer only.
  • history is still flat strings. Narration is not persisted there, same as before.

Supporting change (streaming_util.py)

  • send_thinking now returns the status text it picked, so the planner can record what the user actually saw. Backward compatible.

AI Usage

Please disclose whether you've used AI in this work (it's cool, we just want to
know!):

  • Yes, I have not used AI
  • No, I have not used AI

You can read more details in our
Responsible AI Policy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Global chat: Fix planner streaming

1 participant