feat(subagents): attribute subagent spans to parent thread's Langfuse… by heart-scalpel · Pull Request #3611 · bytedance/deer-flow

heart-scalpel · 2026-06-16T17:43:06Z

Why

Subagent LLM/tool spans were landing in Langfuse — they were not invisible. What was missing was grouping and attribution: the subagent execution path did not call inject_langfuse_metadata(...) and built its model with attach_tracing=True, so every subagent run produced an isolated top-level trace with a fresh session_id and the default user. They were findable in the unfiltered trace list (the subagent:<name> tag narrowed things down) but could not be pivoted to from the parent thread's Sessions view, and Langfuse cost attribution for subagent traffic did not line up with the parent conversation — even though DeerFlow's internal SubagentTokenCollector accounting was already correct.

This extends the lead-agent tracing wiring to the subagent path.

What changed

From the operator's perspective: opening a thread in the Langfuse Sessions page now shows all traces for that conversation — the lead agent trace and every subagent trace — under the same session card, carrying the same user_id, with each subagent trace clearly named (subagent:general-purpose, subagent:bash, …) so it stays distinguishable from the lead trace.

Trace topology is unchanged: subagent traces remain separate top-level traces in the same session. Nesting them as child spans under the lead trace is a harder follow-up (Plan B) and intentionally out of scope.

Three concrete code changes implement this:

subagents/executor.py — In _aexecute, build_tracing_callbacks() is appended to run_config["callbacks"] (alongside the existing SubagentTokenCollector, not replacing it) and inject_langfuse_metadata(...) is called with the parent thread_id, the constructor-supplied user_id, and a normalized subagent:<name> trace name. _create_agent now builds the model with attach_tracing=False — the same pairing the lead agent uses, so model-level tracing does not double-count with the graph-root callbacks.
tools/builtins/task_tool.py — user_id is resolved via resolve_runtime_user_id(runtime) at the parent tool layer (before the background thread starts) and threaded through SubagentExecutor.__init__. This matters because _current_user is a ContextVar and the subagent runs in _execution_pool, where the contextvar is not guaranteed to be propagated.
backend/CLAUDE.md — Tracing System section now lists subagents/executor.py::_aexecute as a third trace-attribute injection point alongside runtime/runs/worker.py::run_agent and client.py::DeerFlowClient.stream, and the field-mapping table documents the subagent-specific sources (user_id via resolve_runtime_user_id, trace name subagent:<name> with _ → - normalization).

Name normalization is inline (lowercase, _ → -) because runtime/runs/naming.py::resolve_root_run_name only handles the lead-agent path; there is no shared helper to call.

Surface area

Frontend UI — page / component / setting / interaction under frontend/
Backend API — endpoint / SSE event / request-response shape under backend/app
Agents / LangGraph — agent node, graph wiring, langgraph.json, or prompt change
Sandbox — docker/ or sandboxed execution
Skills — change under skills/
Dependencies — new/upgraded entry in backend/pyproject.toml or frontend/package.json (say what it buys us)
Default behavior change — changes existing behavior without the user opting in (default model, default setting, data shape)
Docs / tests / CI only — no runtime behavior change

Runtime behavior change is limited to tracing attribution. No user-facing API or default setting changes shape — the Langfuse session card just gains the subagent traces that already existed as isolated top-level traces. Operators who have not enabled Langfuse are unaffected (inject_langfuse_metadata no-ops when Langfuse is not in the enabled providers, and build_tracing_callbacks returns an empty list).

Test verification

New behavior is pinned by a dedicated test class added before the implementation was considered done (TDD):

Test path that locks in the new wiring: backend/tests/test_subagent_executor.py::TestSubagentTracingWiring
- test_aexecute_appends_tracing_callbacks_to_run_config — build_tracing_callbacks() output is appended (not replacing) to run_config["callbacks"], and SubagentTokenCollector survives the merge.
- test_aexecute_injects_langfuse_session_user_and_trace_name — when Langfuse is enabled, run_config["metadata"] carries langfuse_session_id == parent_thread_id, langfuse_user_id from the constructor, langfuse_trace_name == "subagent:general-purpose" (underscore→hyphen normalization), and a model:<name> tag.
- test_aexecute_skips_langfuse_metadata_when_disabled — LangSmith-only deployments stay inert.
- test_user_id_defaults_when_not_supplied, test_trace_name_falls_back_when_config_name_empty, test_environment_tag_emitted_from_deer_flow_env — pin the metadata field sources so the wiring cannot regress silently.
The pre-existing test_create_agent_threads_explicit_app_config_to_model_and_middlewares was extended to assert attach_tracing=False, locking in the pairing with graph-root callbacks.

Validation

cd backend
# Lint
uv run ruff check \
  packages/harness/deerflow/subagents/executor.py \
  packages/harness/deerflow/tools/builtins/task_tool.py \
  tests/test_subagent_executor.py
# All checks passed!

# Targeted: new + updated tests
PYTHONPATH=. uv run pytest tests/test_subagent_executor.py::TestSubagentTracingWiring -v
# 6 passed

# Full executor suite (no regression)
PYTHONPATH=. uv run pytest tests/test_subagent_executor.py
# 55 passed

# All tracing-related suites stay aligned
PYTHONPATH=. uv run pytest \
  tests/test_subagent_executor.py \
  tests/test_tracing_factory.py \
  tests/test_tracing_metadata.py \
  tests/test_worker_langfuse_metadata.py \
  tests/test_client_langfuse_metadata.py \
  tests/test_lead_agent_model_resolution.py
# 94 passed

# Harness/app boundary still holds (executor.py now imports deerflow.tracing)
PYTHONPATH=. uv run pytest tests/test_harness_boundary.py tests/test_reload_boundary.py
# 11 passed

End-to-end Langfuse UI verification is pending — the unit tests pin the wiring (run_config["callbacks"] contents and run_config["metadata"] keys) but do not exercise the Langfuse handler against a live Langfuse instance. Reviewers with a Langfuse project configured should confirm in Sessions view that a thread with subagent delegations shows the subagent traces under the parent session card.

Manual E2E Validation

Verified on a live Langfuse-enabled instance:

Subagent traces grouped under parent session card
Parent session_id / user_id correctly inherited
No duplicate / double-counted traces

Screenshots attached below.

AI assistance

Tool(s) used: Claude Code

How you used it:

Initial Plan A implementation (staged diff: build_tracing_callbacks append, inject_langfuse_metadata call, attach_tracing=False, user_id plumbing through task_tool.py) was reviewed by Claude Code against the lead-agent pattern, and the review flagged the stale build_run_config comment reference, the inconsistent run_config["callbacks"] merge shape, the misleading user_id docstring, the missing TDD coverage, and the stale CLAUDE.md Tracing section.
Polish (comments, docstring, callback merge shape) and the entire TestSubagentTracingWiring test class were written by Claude Code. The test class was iterated against the real executor until all six cases passed; one case (model: tag emission) caught a real setup issue (self.model_name was None because _create_agent was mocked) and was fixed by passing parent_model through _make_executor so __init__ resolves the model name eagerly.
I (the author) reviewed every diff line, ran the test/lint matrix above, and take responsibility for the change.
I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

fancyboi999

Read the full diff and traced the wiring against the existing tracing layer — this looks correct and lands cleanly. What I verified:

Signature match (the main risk for this kind of change): inject_langfuse_metadata(config, *, thread_id, user_id, assistant_id, model_name, environment) (tracing/metadata.py:73-82) matches the call in executor.py exactly — no kwarg drift.
No-op when Langfuse is off is real: inject_langfuse_metadata returns early when build_langfuse_trace_metadata() yields {} (metadata.py:99-100), so LangSmith-only / no-provider deployments are unaffected — consistent with test_aexecute_skips_langfuse_metadata_when_disabled. And the setdefault merge keeps any caller-supplied langfuse_* override, same as the worker/client paths.
Pattern parity: graph-root build_tracing_callbacks() + attach_tracing=False on the model mirrors the lead-agent wiring documented in backend/CLAUDE.md, so there's no double-counted trace. The append ([*existing_callbacks, *tracing_callbacks]) preserves the SubagentTokenCollector, which test_aexecute_appends_tracing_callbacks_to_run_config pins (asserts >=2 callbacks).
user_id capture: resolve_runtime_user_id (runtime/user_context.py:112) and DEFAULT_USER_ID = "default" (:97) exist as the tests assume; the default-fallback assertion is accurate, and resolve_runtime_user_id(None) is safe (signature takes object | None), so calling it unconditionally in task_tool won't blow up when there's no runtime.
Mapping: thread_id -> langfuse_session_id matches the field table you updated in CLAUDE.md.
Tests are genuine regression anchors (session_id == parent thread_id, user_id == captured, _->- trace-name normalization, env tag, empty-name fallback, disabled no-op) rather than vacuous mocks.

Scope is tight — all four files serve the attribution goal and the doc is updated to match. Looks correct to me; good to merge once CI is green. One optional non-blocking nit inline.

fancyboi999 · 2026-06-17T00:56:11Z

Please fix the lint check error @heart-scalpel

… session The subagent execution path did not call inject_langfuse_metadata(...) and built its model with attach_tracing=True, so subagent LLM/tool spans landed in Langfuse as isolated top-level traces carrying fresh session ids and the default user. They were findable in the unfiltered trace list but did not group under the parent thread's session card, and Langfuse cost attribution for subagent traffic did not line up with the parent conversation — even though DeerFlow's internal token accounting (SubagentTokenCollector) was already correct. Extend the lead-agent tracing wiring to the subagent path so a single subagent run produces one trace that shares the parent thread's session_id and user_id, with a subagent:<name> trace name: - subagents/executor.py: append build_tracing_callbacks() output to run_config["callbacks"] (preserving SubagentTokenCollector) and call inject_langfuse_metadata(...) with thread_id, user_id, and the normalized subagent:<name> trace name. Build the model with attach_tracing=False so model-level tracing does not double-count with the graph-root callbacks — the same pairing the lead agent uses. - tools/builtins/task_tool.py: resolve user_id via resolve_runtime_user_id(runtime) at the parent tool layer (before the background thread starts) and thread it through SubagentExecutor.__init__, because the _current_user contextvar is not guaranteed to survive the _execution_pool boundary. Trace topology is unchanged: subagent traces remain separate top-level traces in the same session, not nested as child spans under the lead trace (Plan B follow-up). Tests: tests/test_subagent_executor.py::TestSubagentTracingWiring covers the callback append, the session/user/trace-name injection, the disabled-langfuse no-op, the DEFAULT_USER_ID fallback, the empty-name trace-name fallback, and the env-tag emission. Existing test_create_agent_threads_explicit_app_config_to_model_and_middlewares now also asserts attach_tracing=False. Docs: CLAUDE.md Tracing System section documents subagents/executor.py as a third injection point alongside worker.py and client.py.

heart-scalpel · 2026-06-17T01:51:31Z

@fancyboi999 Thank you very much for your thorough review and valuable suggestions.

fancyboi999 · 2026-06-17T02:08:51Z

Thanks! And your call on the subagent-name normalization is the right one — since the names come from config.yaml and are shown back to the operator verbatim, slugifying them further would just obscure them. That note was non-blocking anyway, so no change needed there.

For the failing Lint Check: it isn't a ruff check rule — that step passes clean. It's the ruff format --check . half of make lint, and it flags a single file: backend/tests/test_subagent_executor.py. A handful of long one-liners exceed the format width and ruff wants to wrap them (the inline lambdas in the storage stub ~L78, the long AssertionError(...) message ~L266, and the monkeypatch.setattr(...) calls ~L355). Just run make format in backend/ (or uvx ruff format tests/test_subagent_executor.py) and commit the result — CI should go green after that.

heart-scalpel · 2026-06-17T02:19:56Z

Bash(uvx ruff format --check tests/test_subagent_executor.py && uvx ruff check
tests/test_subagent_executor.py)
⎿ 1 file already formatted
All checks passed!

@fancyboi999
I’ve finished the work and committed the changes with git amend.

fancyboi999 · 2026-06-17T02:36:35Z

Confirmed on my end. I pulled the current PR head (cb02d2a2) and ran the repo's lint config (backend/ruff.toml, line-length 240) on tests/test_subagent_executor.py with ruff 0.15.17 (what uvx ruff resolves to in CI): ruff format --check → 1 file already formatted, ruff check → All checks passed!. So the formatting is good.

Two heads-ups so the red ❌ makes sense:

The Lint Check workflow is currently sitting in action_required — GitHub holds CI runs from first-time contributors until a maintainer approves them, so it hasn't re-run since the original failure. It needs a maintainer to approve/re-run the workflow before it goes green; nothing left for you to format.
The PR still shows just the original commit cb02d2a2, so if your git commit --amend was meant to push a change it may not have force-pushed (git push --force-with-lease). As it stands the file already passes lint, so you're fine either way — just flagging in case you expected to see a new commit.

heart-scalpel · 2026-06-17T02:41:15Z

I've pushed the changes here. The checks need to be manually re-triggered.

heart-scalpel · 2026-06-17T03:08:29Z

Confirmed on my end. I pulled the current PR head (cb02d2a2) and ran the repo's lint config (backend/ruff.toml, line-length 240) on tests/test_subagent_executor.py with ruff 0.15.17 (what uvx ruff resolves to in CI): ruff format --check → 1 file already formatted, ruff check → All checks passed!. So the formatting is good.

Two heads-ups so the red ❌ makes sense:

The Lint Check workflow is currently sitting in action_required — GitHub holds CI runs from first-time contributors until a maintainer approves them, so it hasn't re-run since the original failure. It needs a maintainer to approve/re-run the workflow before it goes green; nothing left for you to format.

The PR still shows just the original commit cb02d2a2, so if your git commit --amend was meant to push a change it may not have force-pushed (git push --force-with-lease). As it stands the file already passes lint, so you're fine either way — just flagging in case you expected to see a new commit.

It was my fault for not communicating clearly. I’ll lay everything out in detail next time. Thank you for your thoughtful advice and help — it’s truly appreciated.

… session (bytedance#3611) The subagent execution path did not call inject_langfuse_metadata(...) and built its model with attach_tracing=True, so subagent LLM/tool spans landed in Langfuse as isolated top-level traces carrying fresh session ids and the default user. They were findable in the unfiltered trace list but did not group under the parent thread's session card, and Langfuse cost attribution for subagent traffic did not line up with the parent conversation — even though DeerFlow's internal token accounting (SubagentTokenCollector) was already correct. Extend the lead-agent tracing wiring to the subagent path so a single subagent run produces one trace that shares the parent thread's session_id and user_id, with a subagent:<name> trace name: - subagents/executor.py: append build_tracing_callbacks() output to run_config["callbacks"] (preserving SubagentTokenCollector) and call inject_langfuse_metadata(...) with thread_id, user_id, and the normalized subagent:<name> trace name. Build the model with attach_tracing=False so model-level tracing does not double-count with the graph-root callbacks — the same pairing the lead agent uses. - tools/builtins/task_tool.py: resolve user_id via resolve_runtime_user_id(runtime) at the parent tool layer (before the background thread starts) and thread it through SubagentExecutor.__init__, because the _current_user contextvar is not guaranteed to survive the _execution_pool boundary. Trace topology is unchanged: subagent traces remain separate top-level traces in the same session, not nested as child spans under the lead trace (Plan B follow-up). Tests: tests/test_subagent_executor.py::TestSubagentTracingWiring covers the callback append, the session/user/trace-name injection, the disabled-langfuse no-op, the DEFAULT_USER_ID fallback, the empty-name trace-name fallback, and the env-tag emission. Existing test_create_agent_threads_explicit_app_config_to_model_and_middlewares now also asserts attach_tracing=False. Docs: CLAUDE.md Tracing System section documents subagents/executor.py as a third injection point alongside worker.py and client.py.

- telegram-streaming: 29/29 checkbox ticked (839fa99 bytedance#3534) - minimax-providers: 34/34 checkbox ticked (cd5beda bytedance#3437) - event-store-history: 9/15 ticked + 6 TODO 2026-Q3 (229c809) - langfuse-tracing: status: completed (PR bytedance#1717 + bytedance#2944 bytedance#3611) - No code changes, no checkbox added/removed

… session (bytedance#3611) The subagent execution path did not call inject_langfuse_metadata(...) and built its model with attach_tracing=True, so subagent LLM/tool spans landed in Langfuse as isolated top-level traces carrying fresh session ids and the default user. They were findable in the unfiltered trace list but did not group under the parent thread's session card, and Langfuse cost attribution for subagent traffic did not line up with the parent conversation — even though DeerFlow's internal token accounting (SubagentTokenCollector) was already correct. Extend the lead-agent tracing wiring to the subagent path so a single subagent run produces one trace that shares the parent thread's session_id and user_id, with a subagent:<name> trace name: - subagents/executor.py: append build_tracing_callbacks() output to run_config["callbacks"] (preserving SubagentTokenCollector) and call inject_langfuse_metadata(...) with thread_id, user_id, and the normalized subagent:<name> trace name. Build the model with attach_tracing=False so model-level tracing does not double-count with the graph-root callbacks — the same pairing the lead agent uses. - tools/builtins/task_tool.py: resolve user_id via resolve_runtime_user_id(runtime) at the parent tool layer (before the background thread starts) and thread it through SubagentExecutor.__init__, because the _current_user contextvar is not guaranteed to survive the _execution_pool boundary. Trace topology is unchanged: subagent traces remain separate top-level traces in the same session, not nested as child spans under the lead trace (Plan B follow-up). Tests: tests/test_subagent_executor.py::TestSubagentTracingWiring covers the callback append, the session/user/trace-name injection, the disabled-langfuse no-op, the DEFAULT_USER_ID fallback, the empty-name trace-name fallback, and the env-tag emission. Existing test_create_agent_threads_explicit_app_config_to_model_and_middlewares now also asserts attach_tracing=False. Docs: CLAUDE.md Tracing System section documents subagents/executor.py as a third injection point alongside worker.py and client.py.

fancyboi999 reviewed Jun 17, 2026

View reviewed changes

Comment thread backend/packages/harness/deerflow/subagents/executor.py

heart-scalpel force-pushed the feat/subagent-langfuse-session-attribution branch from 2694417 to cb02d2a Compare June 17, 2026 01:50

WillemJiang added this to the 2.0.0 milestone Jun 17, 2026

WillemJiang approved these changes Jun 17, 2026

View reviewed changes

WillemJiang merged commit a72af8e into bytedance:main Jun 17, 2026
11 checks passed

yu-xin-c mentioned this pull request Aug 25, 2026

fix(subagents): preserve user id in child runtime context #3108

Closed

Navigation Menu

Uh oh!

feat(subagents): attribute subagent spans to parent thread's Langfuse… - #3611

feat(subagents): attribute subagent spans to parent thread's Langfuse…#3611
WillemJiang merged 1 commit into
bytedance:mainfrom
heart-scalpel:feat/subagent-langfuse-session-attribution

heart-scalpel commented Jun 16, 2026

Uh oh!

fancyboi999 left a comment

Uh oh!

Uh oh!

fancyboi999 commented Jun 17, 2026

Uh oh!

heart-scalpel commented Jun 17, 2026

Uh oh!

fancyboi999 commented Jun 17, 2026

Uh oh!

heart-scalpel commented Jun 17, 2026

Uh oh!

fancyboi999 commented Jun 17, 2026

Uh oh!

heart-scalpel commented Jun 17, 2026

Uh oh!

heart-scalpel commented Jun 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

heart-scalpel commented Jun 16, 2026

Why

What changed

Surface area

Test verification

Validation

Manual E2E Validation

AI assistance

Uh oh!

fancyboi999 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fancyboi999 commented Jun 17, 2026

Uh oh!

heart-scalpel commented Jun 17, 2026

Uh oh!

fancyboi999 commented Jun 17, 2026

Uh oh!

heart-scalpel commented Jun 17, 2026

Uh oh!

fancyboi999 commented Jun 17, 2026

Uh oh!

heart-scalpel commented Jun 17, 2026

Uh oh!

heart-scalpel commented Jun 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants