LLM inference callback support#1689

Draft

SteveSandersonMS wants to merge 27 commits into

mainfrom

stevesandersonms/llm-inference-callbacks

Commits on Jun 22, 2026

Add LLM inference callback support to Node SDK

authored and
SteveSandersonMS
committed
feat: register llm inference handler globally on the SDK client

SteveSandersonMS
and
Copilot
committed
test: assert /models catalog request now intercepts in callback

SteveSandersonMS
and
Copilot
committed
feat: streaming LLM inference callback (httpStreamStart + e2e)

SteveSandersonMS
and
Copilot
committed
test: e2e for LLM inference callback error mapping

SteveSandersonMS
and
Copilot
committed
refactor(llm-callback): drop inferred request metadata field

SteveSandersonMS
and
Copilot
committed
feat(llm-callback): collapse to a single onLlmRequest with chunked body

SteveSandersonMS
and
Copilot
committed
feat(llm-callback): surface req.signal and propagate cancellation

SteveSandersonMS
and
Copilot
committed
test(llm-callback): cover consumer-initiated cancellation

SteveSandersonMS
and
Copilot
committed
Add WebSocket transport to the LLM inference provider

SteveSandersonMS
and
Copilot
committed
Add LlmRequestHandler base class for SDK consumers

SteveSandersonMS
and
Copilot
committed
Harden LLM inference SDK adapter + WS handler; add unit tests

SteveSandersonMS
and
Copilot
committed
Add SDK e2e asserting sessionId reaches the LLM callback (CAPI + BYOK)

SteveSandersonMS
and
Copilot
committed
Port LLM inference callbacks to the .NET SDK

SteveSandersonMS
and
Copilot
committed
Collapse LLM inference callback public API to LlmRequestHandler

SteveSandersonMS
and
Copilot
committed
Refine LLM inference callback handlers

SteveSandersonMS
and
Copilot
committed
Add Python SDK support for LLM inference callbacks

SteveSandersonMS
and
Copilot
committed
Add Go SDK support for LLM inference callbacks

SteveSandersonMS
and
Copilot
committed
Add Rust SDK support for LLM inference callbacks

SteveSandersonMS
and
Copilot
committed
Java SDK: LLM inference callbacks

SteveSandersonMS
and
Copilot
committed
Simplify .NET LLM inference callbacks

SteveSandersonMS
and
Copilot
committed
Further simplify .NET LLM inference callbacks

SteveSandersonMS
and
Copilot
committed
Rename public .NET callback types to Copilot* prefix

SteveSandersonMS
and
Copilot
committed
Simplify and rename Node SDK LLM callbacks to CopilotRequestHandler

SteveSandersonMS
and
Copilot
committed
Fix .NET WebSocket upgrade deadlock and add WS e2e test

SteveSandersonMS
and
Copilot
committed
Regenerate C#/Node codegen against published runtime 1.0.64-1

SteveSandersonMS
and
Copilot
committed
Add cancel + error e2e coverage for CopilotRequestHandler (Node + .NET)

SteveSandersonMS
and
Copilot
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LLM inference callback support#1689

LLM inference callback support#1689
SteveSandersonMS wants to merge 27 commits into
mainfrom
stevesandersonms/llm-inference-callbacks

Commits on Jun 22, 2026

Add LLM inference callback support to Node SDK

feat: register llm inference handler globally on the SDK client

test: assert /models catalog request now intercepts in callback

feat: streaming LLM inference callback (httpStreamStart + e2e)

test: e2e for LLM inference callback error mapping

refactor(llm-callback): drop inferred request metadata field

feat(llm-callback): collapse to a single onLlmRequest with chunked body

feat(llm-callback): surface req.signal and propagate cancellation

test(llm-callback): cover consumer-initiated cancellation

Add WebSocket transport to the LLM inference provider

Add LlmRequestHandler base class for SDK consumers

Harden LLM inference SDK adapter + WS handler; add unit tests

Add SDK e2e asserting sessionId reaches the LLM callback (CAPI + BYOK)

Port LLM inference callbacks to the .NET SDK

Collapse LLM inference callback public API to LlmRequestHandler

Refine LLM inference callback handlers

Add Python SDK support for LLM inference callbacks

Add Go SDK support for LLM inference callbacks

Add Rust SDK support for LLM inference callbacks

Java SDK: LLM inference callbacks

Simplify .NET LLM inference callbacks

Further simplify .NET LLM inference callbacks

Rename public .NET callback types to Copilot* prefix

Simplify and rename Node SDK LLM callbacks to CopilotRequestHandler

Fix .NET WebSocket upgrade deadlock and add WS e2e test

Regenerate C#/Node codegen against published runtime 1.0.64-1

Add cancel + error e2e coverage for CopilotRequestHandler (Node + .NET)