> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# GitHub Copilot

> Trace GitHub Copilot sessions in VS Code and the Copilot CLI in Arize AX using the Arize Coding Harness Tracing or Copilot's native OpenTelemetry export.

> Trace GitHub Copilot sessions in VS Code and the Copilot CLI in Arize AX for full observability.

[GitHub Copilot](https://github.com/features/copilot) is GitHub's AI pair programmer for VS Code and the command line. There are two ways to get Copilot activity into Arize AX: the [Arize Coding Harness Tracing](https://github.com/Arize-ai/coding-harness-tracing), which installs hooks and exports [OpenInference](https://github.com/Arize-ai/openinference) spans directly, or Copilot's own OpenTelemetry export, which Arize AX normalizes on ingestion.

## Choose an approach

|                     | Arize Coding Harness Tracing                                      | Native OpenTelemetry export                                                                |
| :------------------ | :---------------------------------------------------------------- | :----------------------------------------------------------------------------------------- |
| **Setup scope**     | Once per repository, since the hooks file lives in each workspace | Once per user, device, or enterprise                                                       |
| **Central rollout** | Each developer runs the installer                                 | Through Copilot managed settings (MDM, server-managed, or file-based)                      |
| **Credentials**     | `~/.arize/harness/config.json`, or environment variables          | Managed headers stay inside the extension's exporter and never enter environment variables |
| **Content capture** | On by default, with three separate `ARIZE_LOG_*` redaction flags  | One `captureContent` toggle, off by default, lockable by an admin                          |
| **Span fidelity**   | Native OpenInference, no convention mapping in between            | Normalized from `gen_ai.*` on ingestion, so hook spans arrive unclassified                 |

For a team or an enterprise, prefer the [native OpenTelemetry export](#native-opentelemetry-export). It scales without touching every repository, and an administrator can mandate it centrally so developers configure nothing. Use the harness when you need per-category redaction control, or when you want OpenInference spans with no convention mapping in between.

Both paths need an Arize AX Space ID and API key.

## Launch Arize AX

To get started, sign up for a free [Arize AX account](https://app.arize.com/auth/join) and get your Space ID and API Key:

1. Log in at [app.arize.com](https://app.arize.com)
2. Click **Settings** and copy the **Space ID**
3. Open the **API Keys** tab and create or copy an API key

## Install

This installs the Arize Coding Harness. Copilot hooks are installed at the **project level** under `.github/hooks/`, so run the installer from the root of each repository where you want tracing.

To avoid a per-repository install entirely, use the [native OpenTelemetry export](#native-opentelemetry-export) instead, which needs no installer.

### Curl installer

**macOS / Linux:**

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
cd /path/to/your/project
curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- copilot
```

**Windows (PowerShell):**

```powershell theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
cd C:\path\to\your\project
iwr -useb https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.bat -OutFile $env:TEMP\install.bat
& $env:TEMP\install.bat copilot
```

### Local clone

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
git clone https://github.com/Arize-ai/coding-harness-tracing.git
cd /path/to/your/project
/path/to/coding-harness-tracing/install.sh copilot         # macOS / Linux
\path\to\coding-harness-tracing\install.bat copilot        # Windows
```

The installer writes credentials to `~/.arize/harness/config.json` and registers VS Code (`.github/hooks/*.json`) and Copilot CLI (`.github/hooks/hooks.json`) hooks in the current project.

The installer runs a short interactive setup. Every harness in the Arize Coding Harness Tracing repo asks the same questions, in the same order.

### Setup walkthrough

#### 1. Backend selection

Choose where spans are sent:

* **Phoenix** — your own [Phoenix](https://github.com/Arize-ai/phoenix) instance.
* **Arize AX** — the hosted Arize platform.

#### 2. Credentials

The prompts depend on the backend you picked.

<Tabs>
  <Tab title="Arize AX">
    * **API key** — create one on the [API keys](/docs/ax/security-and-settings/api-keys) tab.
    * **Space ID** — shown on the same settings tab as your API keys.
    * **OTLP endpoint** — defaults to `otlp.arize.com:443`. Override it only for a hosted or dedicated instance.
  </Tab>

  <Tab title="Phoenix">
    * **Endpoint** — defaults to `http://localhost:6006`.
    * **API key** — optional. Leave it blank when Phoenix runs without auth.
  </Tab>
</Tabs>

If you have already configured another harness against the same backend, the installer offers a copy-from menu so you can reuse those credentials instead of retyping them.

#### 3. Project name

The project that this harness's spans are grouped under. Defaults to the harness name.

#### 4. User ID (optional)

A free-form identifier attached to every span as `user.id`. Useful when teammates share one backend. Leave it blank to skip.

#### 5. Content logging

Three `[Y/n]` opt-outs that apply to **every** harness, not only the one you are installing:

* Log user prompts?
* Log what tools were asked to do (commands, file paths, URLs)?
* Log what tools returned (file contents, command output)?

You are asked these only the first time you install any harness. Later installs reuse the existing `logging` block in `~/.arize/harness/config.json`, which you can edit at any time.

### Install flags

| Flag                      | Effect                                                                                                                                   |
| :------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------- |
| `--with-skills`           | Symlink this harness's management skill into `.agents/skills/`, so a coding agent in the workspace can manage the tracing config for you |
| `--non-interactive`, `-y` | Ask nothing, and read every value from the environment instead                                                                           |
| `--branch NAME`           | Install from a specific branch instead of `main`                                                                                         |
| `--wheel-dir DIR`         | Install from local wheels in `DIR`: no network access and no remote code execution                                                       |

### Non-interactive install

Pass `--non-interactive` (or `-y`) to skip every prompt above and take each value from the environment instead. Nothing is asked, and a missing required value is an error rather than a prompt, which makes this the mode to use from a script, from CI, or when a coding agent is driving the install itself.

Values come from the environment, or from a dotenv file named explicitly with `ARIZE_ENV_FILE`. Naming a file keeps the API key out of the command line and your shell history.

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
ARIZE_ENV_FILE=~/.arize/onboarding.env ./install.sh <harness> --non-interactive
```

| Variable                              | Default                 | Description                                                                                                                                                                                                                           |
| :------------------------------------ | :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `ARIZE_API_KEY` + `ARIZE_SPACE_ID`    | —                       | Arize AX credentials. Both are required for the Arize backend.                                                                                                                                                                        |
| `PHOENIX_ENDPOINT`, `PHOENIX_API_KEY` | `http://localhost:6006` | Phoenix endpoint and optional API key.                                                                                                                                                                                                |
| `ARIZE_BACKEND`                       | inferred                | `arize` or `phoenix`. A space ID implies Arize AX and a Phoenix endpoint implies Phoenix. When both are present the install stops and asks you to set this rather than guess, since guessing would discard one backend's credentials. |
| `ARIZE_PROJECT_NAME`                  | harness name            | Project that spans are grouped under. **Read from the dotenv file only.** An installed harness exports its own project name into every session, so an inherited environment value is ignored here.                                    |
| `ARIZE_USER_ID`                       | —                       | Optional `user.id` on every span.                                                                                                                                                                                                     |
| `ARIZE_OTLP_ENDPOINT`                 | `otlp.arize.com:443`    | Override for a hosted or dedicated Arize instance.                                                                                                                                                                                    |
| `ARIZE_LOG_PROMPTS`                   | `false`                 | Set `true` to capture prompt text.                                                                                                                                                                                                    |
| `ARIZE_LOG_TOOL_DETAILS`              | `false`                 | Set `true` to capture tool commands, file paths, and URLs.                                                                                                                                                                            |
| `ARIZE_LOG_TOOL_CONTENT`              | `false`                 | Set `true` to capture tool output.                                                                                                                                                                                                    |
| `ARIZE_ENV_FILE`                      | —                       | Dotenv file to read. No file is read unless this is set, and a path that is not a readable file is an error rather than a fallback to the environment.                                                                                |
| `ARIZE_WHEEL_DIR`                     | —                       | Same as `--wheel-dir`. Install from local wheels instead of downloading the repo.                                                                                                                                                     |

<Warning>
  A named `ARIZE_ENV_FILE` outranks the environment, and there is deliberately **no automatic `./.env` search**. Reading the working directory would let a cloned repository's dotenv choose `ARIZE_OTLP_ENDPOINT` or `PHOENIX_ENDPOINT` while your real credentials came from the environment, installing a config that ships spans and a bearer API key to an endpoint the repo picked, for every later session on that machine. Name the file you mean.
</Warning>

<Note>
  Content logging is **off by default** in this mode, unlike the interactive wizard where each question defaults to yes. A `[Y/n]` default is a person declining to change an answer they were shown; the same default unattended would capture prompts, commands, and file contents that nobody agreed to. Set the `ARIZE_LOG_*` variables you want to `true`.
</Note>

The API key is never echoed. The installer reports only that it found one and where it came from, and every resolved value is reported with its source, so a wrong-credentials install stays diagnosable:

```
[arize] Backend: Arize AX at otlp.arize.com:443 (from default)
[arize]   space ID: my-space (from /path/to/.env)
[arize]   API key: found (from /path/to/.env)
[arize] Project name: codex (from default)
```

### Check what's installed

`status` reports which harnesses are configured and whether their hooks are actually wired into each harness's own settings file. Both have to be true for traces to appear.

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
./install.sh status
./install.sh status --json    # machine-readable
```

`hooks: NOT registered` means credentials are saved but the harness was never wired up, or something removed the hooks. Re-run the install for that harness.

Use `--json` from a script or a coding agent to gate on the exit code without parsing output: `0` means every configured harness is wired up, `1` means nothing is configured, and `2` means at least one harness's hooks are missing. The payload contains no secrets — an API key appears only as `"api_key_present": true` — so it is safe to paste into a bug report.

### Keep it up to date

`update` pulls the latest code and re-registers every harness already in `config.json`.

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
./install.sh update
```

Re-registering runs each harness's installer, so in a terminal it still asks for each project name. With no terminal to answer on, in CI or a cron job, it takes the stored values instead of failing: credentials are not re-read on that path, and the project name keeps whatever is in `config.json`.

## Configuration

Credentials live in `~/.arize/harness/config.json` and apply across all projects. Per-project environment variables override `config.json` and can be set in your shell profile.

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
export ARIZE_API_KEY="<your-api-key>"
export ARIZE_SPACE_ID="<your-space-id>"
export ARIZE_PROJECT_NAME="copilot"
export ARIZE_TRACE_ENABLED="true"
```

### Redaction controls

Each `ARIZE_LOG_*` flag accepts `"true"` or `"false"` and defaults to `"true"`. Set to `"false"` to opt out per category:

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
export ARIZE_LOG_PROMPTS="false"
export ARIZE_LOG_TOOL_DETAILS="false"
export ARIZE_LOG_TOOL_CONTENT="false"
```

| Flag                     | Redacts                                 |
| :----------------------- | :-------------------------------------- |
| `ARIZE_LOG_PROMPTS`      | User prompt and assistant response text |
| `ARIZE_LOG_TOOL_DETAILS` | Tool names and arguments                |
| `ARIZE_LOG_TOOL_CONTENT` | Tool call output content                |

## Observe

Once tracing is enabled, Copilot activity from both VS Code and the CLI streams to Arize AX. You'll see:

* **Turn traces** — each prompt → response cycle as a parent span
* **Tool spans** — nested spans for each tool call with inputs, outputs, and duration
* **Subagent spans** — activity from any subagents Copilot spawns (VS Code)
* **Error spans** — `errorOccurred` events from the CLI
* **Session grouping** — all turns from the same session grouped by `session_id`

### Hooks Captured

Copilot in VS Code and on the CLI use different event names. The Agent Kit handles both with one install command.

| Surface | Hook File                          | Events                                                                                            |
| :------ | :--------------------------------- | :------------------------------------------------------------------------------------------------ |
| VS Code | `.github/hooks/session-start.json` | `SessionStart`                                                                                    |
| VS Code | `.github/hooks/user-prompt.json`   | `UserPromptSubmit`                                                                                |
| VS Code | `.github/hooks/pre-tool.json`      | `PreToolUse`                                                                                      |
| VS Code | `.github/hooks/post-tool.json`     | `PostToolUse`                                                                                     |
| VS Code | `.github/hooks/stop.json`          | `Stop`                                                                                            |
| VS Code | `.github/hooks/subagent-stop.json` | `SubagentStop`                                                                                    |
| CLI     | `.github/hooks/hooks.json`         | `sessionStart`, `userPromptSubmitted`, `preToolUse`, `postToolUse`, `errorOccurred`, `sessionEnd` |

## Native OpenTelemetry export

Copilot Chat emits its own OpenTelemetry traces following the [OTel GenAI semantic conventions](https://code.visualstudio.com/docs/agents/guides/monitoring-agents), with no hooks and no installer. Arize AX [normalizes `gen_ai.*` spans into OpenInference on ingestion](/docs/ax/concepts/otel-openinference/semantic-conventions#how-arize-ax-ingests-genai-spans), so you can point Copilot straight at the Arize AX OTLP endpoint.

Enabling OTel instruments every agent type at once: the foreground agent, Copilot CLI sessions, Claude agent sessions, and any subagents, with trace context propagated across them.

### Set up a single developer

In **settings.json**:

```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.otlpEndpoint": "https://otlp.arize.com",
  "github.copilot.chat.otel.captureContent": true
}
```

There is no user-level setting for OTLP headers, so credentials and project routing go through environment variables:

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
export OTEL_EXPORTER_OTLP_HEADERS="space_id=<your-space-id>,api_key=<your-api-key>"
export OTEL_RESOURCE_ATTRIBUTES="openinference.project.name=copilot"
```

Without `openinference.project.name`, spans have no project to land in. `OTEL_SERVICE_NAME` does not route them.

<Warning>
  The exported URL must resolve to `https://otlp.arize.com/v1/traces`. VS Code appends `/v1/traces` to the endpoint you configure, the same way it does for the `http://localhost:4318` collector default. If no spans arrive, set `otlpEndpoint` to the full path instead. Posting to the bare host returns an HTTP success and silently discards the spans.
</Warning>

Leave `captureContent` off and you get metadata only: model names, token counts, durations, and tool names, but no prompts, responses, or tool arguments.

### Roll out across an enterprise

Administrators can mandate the whole configuration through the `telemetry` block in [Copilot managed settings](https://code.visualstudio.com/docs/enterprise/ai-settings#_configure-telemetry-export-with-opentelemetry), delivered by MDM (Intune, Jamf, Group Policy), server-managed settings from your `.github-private` repository, or a file-based `managed-settings.json`. Developers configure nothing.

```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
{
  "telemetry": {
    "enabled": true,
    "endpoint": "https://otlp.arize.com",
    "protocol": "http/protobuf",
    "captureContent": true,
    "lockCaptureContent": true,
    "resourceAttributes": {
      "openinference.project.name": "copilot"
    },
    "headers": {
      "space_id": "<your-space-id>",
      "api_key": "<your-api-key>"
    }
  }
}
```

A managed value always wins, taking precedence over environment variables and user settings. `lockCaptureContent` prevents developers from changing whether prompt and tool content is captured.

<Note>
  Managed `telemetry.headers` are applied only to the Copilot Chat extension's exporter and are deliberately never passed through environment variables, so an API key cannot leak into the tool subprocesses the agent host spawns. As a consequence, managed headers do not currently reach the agent host, so Copilot CLI terminal sessions still need `OTEL_EXPORTER_OTLP_HEADERS` set in the environment.
</Note>

### Spans Captured

Copilot's operation names normalize to OpenInference span kinds as follows:

| Copilot span   | `gen_ai.operation.name` | Arize AX span kind |
| :------------- | :---------------------- | :----------------- |
| `invoke_agent` | `invoke_agent`          | `AGENT`            |
| `chat`         | `chat`                  | `LLM`              |
| `execute_tool` | `execute_tool`          | `TOOL`             |
| `execute_hook` | `execute_hook`          | Unclassified       |

On the agent and chat spans you get prompts and responses as `llm.input_messages` and `llm.output_messages`, token counts, model name, and computed cost. Tool spans carry `tool.name`, `tool.parameters`, and the tool result as `output.value`. `gen_ai.conversation.id` becomes `session.id`, so turns group into sessions.

The raw `gen_ai.*`, `github.copilot.*`, and `copilot_chat.*` attributes are all preserved on the span, so you can still filter on things like `github.copilot.git.branch`.

### What does not carry over

* **Hook spans are unclassified.** `execute_hook` is not a recognized GenAI operation, so those spans arrive with no span kind. Their `github.copilot.hook.*` attributes are preserved and still queryable.
* **Metrics and events are dropped.** Arize AX ingests traces. The `copilot_chat.*` metrics (edit acceptance, edit survival, time to first token) and the OTel events are not spans and are not stored.
* **The provider stays unmapped.** `gen_ai.provider.name` is `github`, which is not in the [provider table](/docs/ax/concepts/otel-openinference/span-kinds#provider-and-system), so it passes through to `llm.system` unchanged and `llm.provider` is left unset.

## Reference

For the full list of environment variables, default file paths, and troubleshooting steps, see the [Copilot tracing README](https://github.com/Arize-ai/coding-harness-tracing/blob/main/tracing/copilot/README.md).

For the native export, see [Monitor agent usage with OpenTelemetry](https://code.visualstudio.com/docs/agents/guides/monitoring-agents) in the VS Code documentation.

## Uninstall

Run from the project root where you installed the hooks:

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
curl -sSL https://raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.sh | bash -s -- uninstall copilot
```

To turn off the native export, set `github.copilot.chat.otel.enabled` to `false`, or set `telemetry.enabled` to `false` in managed settings.

## Resources

<CardGroup>
  <Card icon="github" href="https://github.com/Arize-ai/coding-harness-tracing" title="Arize Coding Harness Tracing" horizontal />

  <Card icon="github" href="https://github.com/Arize-ai/openinference" title="OpenInference" horizontal />

  <Card icon="book-open" href="https://github.com/features/copilot" title="GitHub Copilot" horizontal />
</CardGroup>