Workbench integration

Group workbench agent traces by plan, wave, task, role, and pipeline step with automatic tags.

workbench dispatches coding agents through implementation, testing, review, fix, and wrap-up stages. Each invocation becomes a separate Claude Code, Codex, Cursor, or Copilot session. thirdeye can tag those sessions automatically so the complete run remains easy to query as one plan.

Set up the integration

Install both CLIs. The Homebrew package for thirdeye includes its browser UI and Pydantic Logfire support:

brew install duncanmckinnon/tap/workbench
brew install duncanmckinnon/tap/thirdeye

Register thirdeye hooks for every supported harness your workbench profile uses:

thirdeye add --claude
thirdeye add --codex
thirdeye add --cursor
thirdeye add --copilot

Then allow thirdeye to capture workbench's metadata variables and start workbench from the same shell:

export THIRDEYE_CAPTURE_ENV='WB_*'
wb run my-plan

Add the export to your shell profile to enable it permanently. The pattern captures only variables beginning with WB_; thirdeye rejects a bare * so the entire environment cannot be captured accidentally.

Workbench enables --trace and --trace-env by default, so no workbench flag is required. Its --trace-prompt option is separate from automatic tagging and can remain off.

thirdeye currently captures workbench roles assigned to Claude Code, Codex, Cursor, or Copilot. Roles assigned to another harness will continue to run, but their sessions will not appear in thirdeye.

Workbench metadata tags

Workbench supplies five environment variables to task-pipeline agents. thirdeye lowercases their values, removes the leading WB_ from the field name, and writes the resulting tags on the session-start event. When Logfire export is enabled, thirdeye also carries these variables into the exported trace — see In Logfire below.

Most task sessions receive all five. Standalone stages such as planning, final review, and PR writing have no wave or task, so only the applicable metadata becomes tags.

Spaces and unsupported punctuation in values become dashes. _, ., -, and # are preserved. A derived tag is skipped if it is invalid or longer than 64 characters.

In Logfire

The tags above are local thirdeye annotations for thirdeye list --tag and the browser UI. When Logfire export is enabled, thirdeye also carries the same workbench context into the exported trace:

• Every span — from the session root down to individual tool calls — gets a wb.* attribute for each variable: WB_PLAN becomes wb.plan, and likewise wb.wave, wb.task, wb.agent, and wb.step. The value is the raw environment value with its original case and characters intact — no plan- prefix and no dash substitution.
• The session span additionally carries each non-empty value as a value-only Logfire tag (auth-refactor, tester, test#1), deduplicated across the trace.

Filter a plan's root traces by tag, then group or break down by the wb.* attributes at any depth — in Logfire queries, the Logfire MCP, and dashboards.

Workbench previously supplied this context through OTEL_RESOURCE_ATTRIBUTES, but Claude Code replaces that variable on the spans it emits, so the WB_* variables are the supported path. An attribute already present on a span is left untouched; the captured wb.* value does not overwrite it.

Tags for each pipeline step

Use the agent tag to select a role across attempts, or the step tag for an exact stage:

Task review and final branch review both use step-review#N. Combine the step with agent-reviewer or agent-branch_reviewer to distinguish them.

Quote tags containing # in shell commands; many shells otherwise treat the remainder as a comment.

Filter workbench traces

Repeated --tag filters use AND semantics. Start with the plan tag, then narrow by task, wave, role, or step:

# Every captured session belonging to a plan.
thirdeye list --tag plan-auth-refactor --tree
 
# Every session for task 3.
thirdeye list \
  --tag plan-auth-refactor \
  --tag task-task-3 \
  --tree
 
# First test attempts in wave 2.
thirdeye list \
  --tag plan-auth-refactor \
  --tag wave-2 \
  --tag 'step-test#1' \
  --tree
 
# Task reviewers only.
thirdeye list \
  --tag plan-auth-refactor \
  --tag agent-reviewer \
  --tree
 
# The second final branch-review attempt.
thirdeye list \
  --tag plan-auth-refactor \
  --tag agent-branch_reviewer \
  --tag 'step-review#2' \
  --tree

Use thirdeye tags to inspect the global tag inventory. thirdeye tag <session-id> --list shows the tags and event sequence for one session. The same automatic tags appear in the browser UI session filters.

Disable or customize metadata

Disable environment metadata for one workbench run with:

wb run my-plan --no-trace-env
 
# Disable all workbench trace metadata, including optional prompt metadata.
wb run my-plan --no-trace

The equivalent plan frontmatter is:

---
trace: true
trace_env: true
---

Built-in workbench adapters inject metadata by default. For a custom adapter in .workbench/agents.yaml, set inject_env: true; custom adapters default to not receiving environment metadata.

Troubleshooting

• Sessions appear without workbench tags. Verify that THIRDEYE_CAPTURE_ENV='WB_*' is exported in the shell that starts wb, and that the run does not use --no-trace or --no-trace-env.
• Workbench tags appear in thirdeye list but not in Logfire. Confirm Logfire export is on with thirdeye logfire status. The wb.* span attributes and the session-span tags are only written when export is enabled.
• No sessions appear. Register the harness with thirdeye add --claude, thirdeye add --codex, thirdeye add --cursor, or thirdeye add --copilot, then start a new workbench run.
• Only some roles appear. Inspect the effective workbench profile. thirdeye captures Claude Code, Codex, Cursor, and Copilot, so roles assigned to other harnesses are not recorded.
• A custom adapter has no tags. Set inject_env: true in its workbench adapter configuration.
• A review query mixes task and final review. Add agent-reviewer for task review or agent-branch_reviewer for final review.

See the workbench thirdeye integration guide for the workbench-side configuration reference.