Oh My Pi

Trace Oh My Pi (omp) terminal coding sessions, model calls, tool usage, and token costs with Arize AX for full observability.

Oh My Pi (omp) is a terminal coding agent that loads its extensions in-process inside its Bun runtime. The Arize Coding Harness Tracing instruments omp lifecycle events and exports OpenInference spans to Arize AX. Each agent run is captured as a trace with model calls, tool invocations, and inline token usage.

Launch Arize AX

To get started, sign up for a free Arize AX account and get your Space ID and API Key:

Log in at app.arize.com
Click Settings and copy the Space ID
Open the API Keys tab and create or copy an API key

Install

Curl installer

macOS / Linux:

curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- omp

Windows (PowerShell):

iwr -useb https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.bat -OutFile $env:TEMP\install.bat
& $env:TEMP\install.bat omp

Local clone

git clone https://github.com/Arize-ai/coding-harness-tracing.git
cd coding-harness-tracing
./install.sh omp           # macOS / Linux
install.bat omp            # Windows

The installer writes credentials to ~/.arize/harness/config.json, copies the hook shim into ~/.omp/extensions/arize-tracing.ts, and registers the shim’s absolute path in the extensions array of ~/.omp/agent/settings.json. omp does not auto-discover an extensions directory, so this explicit registration is required — the installer handles it for you. Open a new omp session after install so the extension loads. The installer runs a short interactive setup. Every harness in the Arize Coding Harness Tracing repo asks the same questions, in the same order.

Setup walkthrough

1. Backend selection

Choose where spans are sent:

Phoenix — your own Phoenix instance.
Arize AX — the hosted Arize platform.

2. Credentials

The prompts depend on the backend you picked.

Arize AX
Phoenix

API key — create one on the API keys tab.
Space ID — shown on the same settings tab as your API keys.
OTLP endpoint — defaults to otlp.arize.com:443. Override it only for a hosted or dedicated instance.

Endpoint — defaults to http://localhost:6006.
API key — optional. Leave it blank when Phoenix runs without auth.

If you have already configured another harness against the same backend, the installer offers a copy-from menu so you can reuse those credentials instead of retyping them.

3. Project name

The project that this harness’s spans are grouped under. Defaults to the harness name.

4. User ID (optional)

A free-form identifier attached to every span as user.id. Useful when teammates share one backend. Leave it blank to skip.

5. Content logging

Three [Y/n] opt-outs that apply to every harness, not only the one you are installing:

Log user prompts?
Log what tools were asked to do (commands, file paths, URLs)?
Log what tools returned (file contents, command output)?

You are asked these only the first time you install any harness. Later installs reuse the existing logging block in ~/.arize/harness/config.json, which you can edit at any time.

Install flags

Flag	Effect
`--with-skills`	Symlink this harness’s management skill into `.agents/skills/`, so a coding agent in the workspace can manage the tracing config for you
`--non-interactive`, `-y`	Ask nothing, and read every value from the environment instead
`--branch NAME`	Install from a specific branch instead of `main`
`--wheel-dir DIR`	Install from local wheels in `DIR`: no network access and no remote code execution

Non-interactive install

Pass --non-interactive (or -y) to skip every prompt above and take each value from the environment instead. Nothing is asked, and a missing required value is an error rather than a prompt, which makes this the mode to use from a script, from CI, or when a coding agent is driving the install itself. Values come from the environment, or from a dotenv file named explicitly with ARIZE_ENV_FILE. Naming a file keeps the API key out of the command line and your shell history.

ARIZE_ENV_FILE=~/.arize/onboarding.env ./install.sh <harness> --non-interactive

Variable	Default	Description
`ARIZE_API_KEY` + `ARIZE_SPACE_ID`	—	Arize AX credentials. Both are required for the Arize backend.
`PHOENIX_ENDPOINT`, `PHOENIX_API_KEY`	`http://localhost:6006`	Phoenix endpoint and optional API key.
`ARIZE_BACKEND`	inferred	`arize` or `phoenix`. A space ID implies Arize AX and a Phoenix endpoint implies Phoenix. When both are present the install stops and asks you to set this rather than guess, since guessing would discard one backend’s credentials.
`ARIZE_PROJECT_NAME`	harness name	Project that spans are grouped under. Read from the dotenv file only. An installed harness exports its own project name into every session, so an inherited environment value is ignored here.
`ARIZE_USER_ID`	—	Optional `user.id` on every span.
`ARIZE_OTLP_ENDPOINT`	`otlp.arize.com:443`	Override for a hosted or dedicated Arize instance.
`ARIZE_LOG_PROMPTS`	`false`	Set `true` to capture prompt text.
`ARIZE_LOG_TOOL_DETAILS`	`false`	Set `true` to capture tool commands, file paths, and URLs.
`ARIZE_LOG_TOOL_CONTENT`	`false`	Set `true` to capture tool output.
`ARIZE_ENV_FILE`	—	Dotenv file to read. No file is read unless this is set, and a path that is not a readable file is an error rather than a fallback to the environment.
`ARIZE_WHEEL_DIR`	—	Same as `--wheel-dir`. Install from local wheels instead of downloading the repo.

A named ARIZE_ENV_FILE outranks the environment, and there is deliberately no automatic ./.env search. Reading the working directory would let a cloned repository’s dotenv choose ARIZE_OTLP_ENDPOINT or PHOENIX_ENDPOINT while your real credentials came from the environment, installing a config that ships spans and a bearer API key to an endpoint the repo picked, for every later session on that machine. Name the file you mean.

Content logging is off by default in this mode, unlike the interactive wizard where each question defaults to yes. A [Y/n] default is a person declining to change an answer they were shown; the same default unattended would capture prompts, commands, and file contents that nobody agreed to. Set the ARIZE_LOG_* variables you want to true.

The API key is never echoed. The installer reports only that it found one and where it came from, and every resolved value is reported with its source, so a wrong-credentials install stays diagnosable:

[arize] Backend: Arize AX at otlp.arize.com:443 (from default)
[arize]   space ID: my-space (from /path/to/.env)
[arize]   API key: found (from /path/to/.env)
[arize] Project name: codex (from default)

Check what’s installed

status reports which harnesses are configured and whether their hooks are actually wired into each harness’s own settings file. Both have to be true for traces to appear.

./install.sh status
./install.sh status --json    # machine-readable

hooks: NOT registered means credentials are saved but the harness was never wired up, or something removed the hooks. Re-run the install for that harness. Use --json from a script or a coding agent to gate on the exit code without parsing output: 0 means every configured harness is wired up, 1 means nothing is configured, and 2 means at least one harness’s hooks are missing. The payload contains no secrets — an API key appears only as "api_key_present": true — so it is safe to paste into a bug report.

Keep it up to date

update pulls the latest code and re-registers every harness already in config.json.

./install.sh update

Re-registering runs each harness’s installer, so in a terminal it still asks for each project name. With no terminal to answer on, in CI or a cron job, it takes the stored values instead of failing: credentials are not re-read on that path, and the project name keeps whatever is in config.json.

Configuration

Credentials live in ~/.arize/harness/config.json. Environment variables override values in config.json and can be set in your shell profile before launching omp.

export ARIZE_API_KEY="<your-api-key>"
export ARIZE_SPACE_ID="<your-space-id>"
export ARIZE_PROJECT_NAME="omp"
export ARIZE_TRACE_ENABLED="true"

Variable	Purpose
`ARIZE_TRACE_ENABLED`	Toggle tracing on or off
`ARIZE_PROJECT_NAME`	Destination project name (defaults to `omp`)
`ARIZE_DRY_RUN`	Run the hook without sending spans, for validation
`ARIZE_USER_ID`	Attribute traces to a specific user
`ARIZE_VERBOSE`	Log routine handler activity (event dispatch, span emits, state transitions)
`ARIZE_TRACE_DEBUG`	Dump raw event payloads under `~/.arize/harness/state/debug/` for inspection

See the main README’s Environment variables section for the full list of runtime overrides.

Redaction controls

Each ARIZE_LOG_* flag accepts "true" or "false" and defaults to "true". Set to "false" to opt out per category:

export ARIZE_LOG_PROMPTS="false"
export ARIZE_LOG_TOOL_DETAILS="false"
export ARIZE_LOG_TOOL_CONTENT="false"

Flag	Redacts
`ARIZE_LOG_PROMPTS`	User prompt and assistant response text
`ARIZE_LOG_TOOL_DETAILS`	Tool names and arguments
`ARIZE_LOG_TOOL_CONTENT`	Tool call output content

Observe

Once tracing is enabled, omp activity is streamed to Arize AX. There is one trace per agent run — a user prompt through the agent’s internal turn/tool-use loop to its final answer. Each trace is a tree:

Turn traces — the root span for each agent run, with the user prompt as input and the final assistant message as output
LLM spans — one per model call in the loop, with model name, provider, prompt/completion/reasoning token counts, cache read/write tokens, and cost
Tool spans — one per tool call, pairing the tool invocation with its result and recording name, input args, and output
Session grouping — all runs from the same session grouped by session.id

Token usage is captured directly on each LLM span — omp surfaces cumulative usage inline on assistant messages, so prompt, completion, reasoning, cache, and cost values are available on every model call.

Spans Captured

omp exposes rich, once-fired lifecycle events that already carry final, structured data. The Arize AX hook forwards a small whitelist of them and emits the following spans:

Span	Kind	Description
`Turn`	CHAIN	Root span for the agent run. Input is the user prompt (`before_agent_start`); output is the final assistant message.
`LLM: <model>`	LLM	Child of `Turn`. One per `turn_end`, carrying model info, token counts, cache tokens, and cost.
`<tool>`	TOOL	Child of `Turn`. One per tool result in a `turn_end`, paired with its originating tool call by id and recording input args and output.

Lifecycle events forwarded: before_agent_start, turn_end, agent_end, and session_shutdown.

Verifying tracing

Run any omp session as you normally would. omp loads the registered extension on startup and forwards lifecycle events to the Arize AX hook.

Errors and handler stderr land in ~/.arize/harness/logs/omp.log. Set export ARIZE_VERBOSE=true before launching omp to also see routine handler activity.
Set export ARIZE_TRACE_DEBUG=true to dump the raw event payloads under ~/.arize/harness/state/debug/ for inspection.
Confirm spans appear in your configured project in Arize AX.

Reference

For the full list of environment variables, default file paths, and troubleshooting steps, see the omp tracing README.

Uninstall

curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- uninstall omp

Uninstall removes the shim’s path from the extensions array in ~/.omp/agent/settings.json, deletes the hook file at ~/.omp/extensions/arize-tracing.ts (only if it carries the Arize header marker, so your own extensions are left alone), and removes the harnesses.omp block from ~/.arize/harness/config.json.

OpenTelemetry

LLM Providers

Agent Frameworks

Coding Agents

Platforms

Orchestration

Evaluation

Launch Arize AX

Install

Curl installer

Local clone

Setup walkthrough

1. Backend selection

2. Credentials

3. Project name

4. User ID (optional)

5. Content logging

Install flags

Non-interactive install

Check what’s installed

Keep it up to date

Configuration

Redaction controls

Observe

Spans Captured

Verifying tracing

Reference

Uninstall

Resources

Arize Coding Harness Tracing

OpenInference

Oh My Pi

​Launch Arize AX

​Install

​Curl installer

​Local clone

​Setup walkthrough

​1. Backend selection

​2. Credentials

​3. Project name

​4. User ID (optional)

​5. Content logging

​Install flags

​Non-interactive install

​Check what’s installed

​Keep it up to date

​Configuration

​Redaction controls

​Observe

​Spans Captured

​Verifying tracing

​Reference

​Uninstall

​Resources

Arize Coding Harness Tracing

OpenInference

Oh My Pi

Launch Arize AX

Install

Curl installer

Local clone

Setup walkthrough

1. Backend selection

2. Credentials

3. Project name

4. User ID (optional)

5. Content logging

Install flags

Non-interactive install

Check what’s installed

Keep it up to date

Configuration

Redaction controls

Observe

Spans Captured

Verifying tracing

Reference

Uninstall

Resources