Skip to content

fix(agents): keep streamed thinking ahead of text and tool calls - #11837

Open
filipkujawa wants to merge 1 commit into
fix/anthropic-preserved-thinkingfrom
fix/thinking-block-order
Open

fix(agents): keep streamed thinking ahead of text and tool calls#11837
filipkujawa wants to merge 1 commit into
fix/anthropic-preserved-thinkingfrom
fix/thinking-block-order

Conversation

@filipkujawa

@filipkujawa filipkujawa commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Fixed part of #11800

Summary

Both agent loops moved thinking that streamed before the text onto the tool-call message, so a turn the model produced as [thinking, text, tool_use] was persisted and replayed as [text, thinking, tool_use]. Anthropic binds each thinking signature to the blocks before it in the same message, so on Claude Fable 5.1 this alone drops the block on the next request of every tool round. Users lose the model's reasoning mid-task without any error.

Thinking now stays where the model produced it.

  • Unrolled agent loop: the relocation is removed. Same-id chunks already coalesce in stream order. The only normalization left drops a copy of reasoning a provider repeats on the tool-call chunk.
  • Legacy loop: the first split tool-call message carries only thinking that arrived on the tool-call chunk. Later split messages repeat the turn's thinking, which DeepSeek and Kimi require on every tool-call message; fix_conversation already removes the signed copies before an Anthropic request.

Net deletion. The wire payload for OpenAI-compatible, Google, and Responses providers is unchanged.

Testing

Existing DeepSeek, Kimi, and multi-tool reasoning tests pass unchanged. The Anthropic test now asserts signed thinking appears once and leads the tool-call message; the unrolled-loop pipeline test pins the block order. Wire-verified on Fable 5.1 with block binding: both loops replay [thinking, text, tool_use] with input_transformations: [].

Stack

  1. fix(anthropic): preserved-thinking compliance for the provider layer #11836 provider layer: block binding, signature-aware history
  2. fix(agents): keep streamed thinking ahead of text and tool calls #11837 agent loops: keep streamed thinking ahead of text and tool calls (this PR)
  3. feat(agents): freeze the system prompt per conversation and deliver changes as system messages #11864 agent loops: freeze the system prompt, deliver changes as system messages

Both loops moved thinking that streamed before the text onto the tool-call
message, which turned [thinking, text, tool_use] into [text, thinking,
tool_use] on replay. Anthropic binds each thinking signature to the blocks
before it in the same message, so Claude Fable 5.1 dropped the block on the
next request of every tool round.

Thinking now stays where the model produced it. The unrolled agent loop only
drops a copy of reasoning the provider repeats on the tool-call chunk. The
legacy loop copies the turn's thinking onto split tool-call messages after the
first one, which DeepSeek and Kimi still require, and fix_conversation removes
the signed copies before an Anthropic request.

Wire-verified on Claude Fable 5.1 with block binding: both loops replay
[thinking, text, tool_use] with input_transformations [].
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant