Skip to content

fix(agents): stop react(submit=False) telling the model to call submit() - #124

Open
naimenz wants to merge 1 commit into
mainfrom
fix/react-no-submit-proceed-prompt
Open

naimenz wants to merge 1 commit into
mainfrom
fix/react-no-submit-proceed-prompt

Conversation

@naimenz

@naimenz naimenz commented Sep 26, 2026

Copy link
Copy Markdown

Problem

react(submit=False) removes the submit tool but keeps the default continue messaging. After any turn with no tool calls the model is told "If you believe you have completed the task, please call the submit() tool", and limit warnings say "Prepare to submit your answer". The model then either calls a tool it doesn't have or decides it is done and stalls.

This happens regardless of limits: the proceed prompt is sent after every tool-less turn even when no limits are set.

There is also no way to fix it from an eval-set YAML: inspect's react_no_submit rejects a plain str on_continue, limit_message_config: null falls back to the same prompt as a str (also rejected), and a callable can't be written in YAML.

Fix

When submit resolves to False, react() now:

  • uses a proceed prompt and limit warnings that don't mention submit (NO_SUBMIT_PROCEED_PROMPT / NO_SUBMIT_LIMIT_MESSAGE_CONFIG; the checkpoint agent's constants are now aliases of these);
  • wraps a plain str on_continue in _constant_on_continue, so a custom message works;
  • swaps the no-submit warning defaults into a custom limit_message_config unless it sets defaults itself.

react() also gains a proceed_prompt argument (mirroring react_with_gated_submit) so a YAML config can change the text while keeping the limit usage messages.

react_with_checkpoint_submit now delegates to this logic instead of duplicating it.

Behaviour change to note

_constant_on_continue previously played its message after every turn, including turns with tool calls. It now only fires after a tool-less turn, matching what a str on_continue does when a submit tool is present, and returns the unchanged state for an empty message rather than emitting an empty user turn. This affects react_with_checkpoint_submit and react_with_handoff_submit when limit_message_config=None.

Testing

13 new tests in test_agent.py cover the default no-submit prompt and warnings, string on_continue, limit_message_config=None and custom configs with and without defaults, proceed_prompt, and _constant_on_continue. Full packages/agents suite, ruff, and basedpyright pass.

🤖 Generated with Claude Code

…xed lengths).

`submit=False` removed the submit tool but left the default proceed prompt ("...please call the `submit()` tool") and limit warnings ("Prepare to submit your answer") in place, so the model either called a tool that did not exist or decided it was done and stalled.

When `submit` resolves to False, `react()` now:

- uses a proceed prompt and limit warnings that do not mention submit (shared with `react_with_checkpoint_submit`, whose constants are now aliases of the new `NO_SUBMIT_*` ones);
- wraps a plain `str` `on_continue` in `_constant_on_continue`, which inspect's `react_no_submit` otherwise rejects;
- swaps the no-submit warning defaults into a custom `limit_message_config` unless it sets `defaults` itself.

`react()` also gains an explicit `proceed_prompt` argument (mirroring `react_with_gated_submit`) so YAML configs can change the text while keeping the limit usage messages. `_constant_on_continue` now only plays its message after a tool-less turn, matching what a `str` `on_continue` does when a submit tool is present, and returns the unchanged state for an empty message instead of emitting an empty user turn.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@JonathanGabor JonathanGabor left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Astra found one minor issue:

[P2] Preserve positional argument compatibility — agents.py:642. Inserting proceed_prompt before limit_message_config shifts existing positional arguments. A previously valid call passing {"cost": None} as its eighth argument now treats that dictionary as prompt text and raises a ChatMessageUser validation error. Append the new parameter after the existing parameters instead. Reproduced against both the base and PR versions.

Also, I'm not super familiar with this repo, maybe @pipmc should give their take as well!

@JonathanGabor
JonathanGabor requested a review from pipmc October 2, 2026 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants