> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Oh My Pi
> Trace Oh My Pi (omp) terminal coding sessions, model calls, tool usage, and token costs in Arize AX using the Arize Coding Harness Tracing.
> Trace Oh My Pi (omp) terminal coding sessions, model calls, tool usage, and token costs with Arize AX for full observability.
[Oh My Pi (omp)](https://github.com/can1357/oh-my-pi) is a terminal coding agent that loads its extensions in-process inside its Bun runtime. The [Arize Coding Harness Tracing](https://github.com/Arize-ai/coding-harness-tracing) instruments omp lifecycle events and exports [OpenInference](https://github.com/Arize-ai/openinference) spans to Arize AX. Each agent run is captured as a trace with model calls, tool invocations, and inline token usage.
## Launch Arize AX
To get started, sign up for a free [Arize AX account](https://app.arize.com/auth/join) and get your Space ID and API Key:
1. Log in at [app.arize.com](https://app.arize.com)
2. Click **Settings** and copy the **Space ID**
3. Open the **API Keys** tab and create or copy an API key
## Install
### Curl installer
**macOS / Linux:**
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- omp
```
**Windows (PowerShell):**
```powershell theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
iwr -useb https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.bat -OutFile $env:TEMP\install.bat
& $env:TEMP\install.bat omp
```
### Local clone
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
git clone https://github.com/Arize-ai/coding-harness-tracing.git
cd coding-harness-tracing
./install.sh omp # macOS / Linux
install.bat omp # Windows
```
The installer writes credentials to `~/.arize/harness/config.json`, copies the hook shim into `~/.omp/extensions/arize-tracing.ts`, and registers the shim's absolute path in the `extensions` array of `~/.omp/agent/settings.json`. omp does not auto-discover an extensions directory, so this explicit registration is required — the installer handles it for you.
Open a new omp session after install so the extension loads.
The installer runs a short interactive setup. Every harness in the Arize Coding Harness Tracing repo asks the same questions, in the same order.
### Setup walkthrough
#### 1. Backend selection
Choose where spans are sent:
* **Phoenix** — your own [Phoenix](https://github.com/Arize-ai/phoenix) instance.
* **Arize AX** — the hosted Arize platform.
#### 2. Credentials
The prompts depend on the backend you picked.
* **API key** — create one on the [API keys](/docs/ax/security-and-settings/api-keys) tab.
* **Space ID** — shown on the same settings tab as your API keys.
* **OTLP endpoint** — defaults to `otlp.arize.com:443`. Override it only for a hosted or dedicated instance.
* **Endpoint** — defaults to `http://localhost:6006`.
* **API key** — optional. Leave it blank when Phoenix runs without auth.
If you have already configured another harness against the same backend, the installer offers a copy-from menu so you can reuse those credentials instead of retyping them.
#### 3. Project name
The project that this harness's spans are grouped under. Defaults to the harness name.
#### 4. User ID (optional)
A free-form identifier attached to every span as `user.id`. Useful when teammates share one backend. Leave it blank to skip.
#### 5. Content logging
Three `[Y/n]` opt-outs that apply to **every** harness, not only the one you are installing:
* Log user prompts?
* Log what tools were asked to do (commands, file paths, URLs)?
* Log what tools returned (file contents, command output)?
You are asked these only the first time you install any harness. Later installs reuse the existing `logging` block in `~/.arize/harness/config.json`, which you can edit at any time.
### Install flags
| Flag | Effect |
| :------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------- |
| `--with-skills` | Symlink this harness's management skill into `.agents/skills/`, so a coding agent in the workspace can manage the tracing config for you |
| `--non-interactive`, `-y` | Ask nothing, and read every value from the environment instead |
| `--branch NAME` | Install from a specific branch instead of `main` |
| `--wheel-dir DIR` | Install from local wheels in `DIR`: no network access and no remote code execution |
### Non-interactive install
Pass `--non-interactive` (or `-y`) to skip every prompt above and take each value from the environment instead. Nothing is asked, and a missing required value is an error rather than a prompt, which makes this the mode to use from a script, from CI, or when a coding agent is driving the install itself.
Values come from the environment, or from a dotenv file named explicitly with `ARIZE_ENV_FILE`. Naming a file keeps the API key out of the command line and your shell history.
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
ARIZE_ENV_FILE=~/.arize/onboarding.env ./install.sh --non-interactive
```
| Variable | Default | Description |
| :------------------------------------ | :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `ARIZE_API_KEY` + `ARIZE_SPACE_ID` | — | Arize AX credentials. Both are required for the Arize backend. |
| `PHOENIX_ENDPOINT`, `PHOENIX_API_KEY` | `http://localhost:6006` | Phoenix endpoint and optional API key. |
| `ARIZE_BACKEND` | inferred | `arize` or `phoenix`. A space ID implies Arize AX and a Phoenix endpoint implies Phoenix. When both are present the install stops and asks you to set this rather than guess, since guessing would discard one backend's credentials. |
| `ARIZE_PROJECT_NAME` | harness name | Project that spans are grouped under. **Read from the dotenv file only.** An installed harness exports its own project name into every session, so an inherited environment value is ignored here. |
| `ARIZE_USER_ID` | — | Optional `user.id` on every span. |
| `ARIZE_OTLP_ENDPOINT` | `otlp.arize.com:443` | Override for a hosted or dedicated Arize instance. |
| `ARIZE_LOG_PROMPTS` | `false` | Set `true` to capture prompt text. |
| `ARIZE_LOG_TOOL_DETAILS` | `false` | Set `true` to capture tool commands, file paths, and URLs. |
| `ARIZE_LOG_TOOL_CONTENT` | `false` | Set `true` to capture tool output. |
| `ARIZE_ENV_FILE` | — | Dotenv file to read. No file is read unless this is set, and a path that is not a readable file is an error rather than a fallback to the environment. |
| `ARIZE_WHEEL_DIR` | — | Same as `--wheel-dir`. Install from local wheels instead of downloading the repo. |
A named `ARIZE_ENV_FILE` outranks the environment, and there is deliberately **no automatic `./.env` search**. Reading the working directory would let a cloned repository's dotenv choose `ARIZE_OTLP_ENDPOINT` or `PHOENIX_ENDPOINT` while your real credentials came from the environment, installing a config that ships spans and a bearer API key to an endpoint the repo picked, for every later session on that machine. Name the file you mean.
Content logging is **off by default** in this mode, unlike the interactive wizard where each question defaults to yes. A `[Y/n]` default is a person declining to change an answer they were shown; the same default unattended would capture prompts, commands, and file contents that nobody agreed to. Set the `ARIZE_LOG_*` variables you want to `true`.
The API key is never echoed. The installer reports only that it found one and where it came from, and every resolved value is reported with its source, so a wrong-credentials install stays diagnosable:
```
[arize] Backend: Arize AX at otlp.arize.com:443 (from default)
[arize] space ID: my-space (from /path/to/.env)
[arize] API key: found (from /path/to/.env)
[arize] Project name: codex (from default)
```
### Check what's installed
`status` reports which harnesses are configured and whether their hooks are actually wired into each harness's own settings file. Both have to be true for traces to appear.
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
./install.sh status
./install.sh status --json # machine-readable
```
`hooks: NOT registered` means credentials are saved but the harness was never wired up, or something removed the hooks. Re-run the install for that harness.
Use `--json` from a script or a coding agent to gate on the exit code without parsing output: `0` means every configured harness is wired up, `1` means nothing is configured, and `2` means at least one harness's hooks are missing. The payload contains no secrets — an API key appears only as `"api_key_present": true` — so it is safe to paste into a bug report.
### Keep it up to date
`update` pulls the latest code and re-registers every harness already in `config.json`.
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
./install.sh update
```
Re-registering runs each harness's installer, so in a terminal it still asks for each project name. With no terminal to answer on, in CI or a cron job, it takes the stored values instead of failing: credentials are not re-read on that path, and the project name keeps whatever is in `config.json`.
## Configuration
Credentials live in `~/.arize/harness/config.json`. Environment variables override values in `config.json` and can be set in your shell profile before launching omp.
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
export ARIZE_API_KEY=""
export ARIZE_SPACE_ID=""
export ARIZE_PROJECT_NAME="omp"
export ARIZE_TRACE_ENABLED="true"
```
| Variable | Purpose |
| :-------------------- | :--------------------------------------------------------------------------- |
| `ARIZE_TRACE_ENABLED` | Toggle tracing on or off |
| `ARIZE_PROJECT_NAME` | Destination project name (defaults to `omp`) |
| `ARIZE_DRY_RUN` | Run the hook without sending spans, for validation |
| `ARIZE_USER_ID` | Attribute traces to a specific user |
| `ARIZE_VERBOSE` | Log routine handler activity (event dispatch, span emits, state transitions) |
| `ARIZE_TRACE_DEBUG` | Dump raw event payloads under `~/.arize/harness/state/debug/` for inspection |
See the [main README's Environment variables section](https://github.com/Arize-ai/coding-harness-tracing#environment-variables) for the full list of runtime overrides.
### Redaction controls
Each `ARIZE_LOG_*` flag accepts `"true"` or `"false"` and defaults to `"true"`. Set to `"false"` to opt out per category:
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
export ARIZE_LOG_PROMPTS="false"
export ARIZE_LOG_TOOL_DETAILS="false"
export ARIZE_LOG_TOOL_CONTENT="false"
```
| Flag | Redacts |
| :----------------------- | :-------------------------------------- |
| `ARIZE_LOG_PROMPTS` | User prompt and assistant response text |
| `ARIZE_LOG_TOOL_DETAILS` | Tool names and arguments |
| `ARIZE_LOG_TOOL_CONTENT` | Tool call output content |
## Observe
Once tracing is enabled, omp activity is streamed to Arize AX. There is one trace per **agent run** — a user prompt through the agent's internal turn/tool-use loop to its final answer. Each trace is a tree:
* **Turn traces** — the root span for each agent run, with the user prompt as input and the final assistant message as output
* **LLM spans** — one per model call in the loop, with model name, provider, prompt/completion/reasoning token counts, cache read/write tokens, and cost
* **Tool spans** — one per tool call, pairing the tool invocation with its result and recording name, input args, and output
* **Session grouping** — all runs from the same session grouped by `session.id`
Token usage is captured directly on each LLM span — omp surfaces cumulative usage inline on assistant messages, so prompt, completion, reasoning, cache, and cost values are available on every model call.
### Spans Captured
omp exposes rich, once-fired lifecycle events that already carry final, structured data. The Arize AX hook forwards a small whitelist of them and emits the following spans:
| Span | Kind | Description |
| :------------- | :---- | :------------------------------------------------------------------------------------------------------------------------------------- |
| `Turn` | CHAIN | Root span for the agent run. Input is the user prompt (`before_agent_start`); output is the final assistant message. |
| `LLM: ` | LLM | Child of `Turn`. One per `turn_end`, carrying model info, token counts, cache tokens, and cost. |
| `` | TOOL | Child of `Turn`. One per tool result in a `turn_end`, paired with its originating tool call by id and recording input args and output. |
Lifecycle events forwarded: `before_agent_start`, `turn_end`, `agent_end`, and `session_shutdown`.
## Verifying tracing
Run any omp session as you normally would. omp loads the registered extension on startup and forwards lifecycle events to the Arize AX hook.
* Errors and handler stderr land in `~/.arize/harness/logs/omp.log`. Set `export ARIZE_VERBOSE=true` before launching omp to also see routine handler activity.
* Set `export ARIZE_TRACE_DEBUG=true` to dump the raw event payloads under `~/.arize/harness/state/debug/` for inspection.
* Confirm spans appear in your configured project in Arize AX.
## Reference
For the full list of environment variables, default file paths, and troubleshooting steps, see the [omp tracing README](https://github.com/Arize-ai/coding-harness-tracing/blob/main/tracing/omp/README.md).
## Uninstall
```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- uninstall omp
```
Uninstall removes the shim's path from the `extensions` array in `~/.omp/agent/settings.json`, deletes the hook file at `~/.omp/extensions/arize-tracing.ts` (only if it carries the Arize header marker, so your own extensions are left alone), and removes the `harnesses.omp` block from `~/.arize/harness/config.json`.
## Resources