Create evaluator version

Endpoint: POST /v2/evaluators/{evaluator_id}/versions

Create a new version of an existing evaluator. The new version becomes the latest version immediately (versioning is append-only).

Payload Requirements

• commit_message describes the changes in this version.
• Provide exactly one of template_config, code_config, remote_config, or decision_judge_config to match the evaluator’s type. code_config.type is a separate inner discriminator (MANAGED or CUSTOM) and is unrelated to the top-level type. Schema and constraints match Create Evaluator.
• For a template version, template_config.llm_config.ai_integration_id must reference an AI integration that exists and is accessible to the evaluator’s space; otherwise the request fails with 404.
• For REMOTE evaluators: remote_config.integration_id must reference an EVALUATOR integration. Each version may reference a different integration; editing an integration affects every version that references it. type: REMOTE requires the remote evaluators feature to be enabled.
• For DECISION_JUDGE evaluators: decision_judge_config replaces the whole configuration, including every question. Decision Judge must be enabled for the account. A config of a different type than the evaluator returns 400. Responses
• 201 — version created; returns the new EvaluatorVersion.
• 400 — malformed request: evaluator_id fails ID-format validation, the body is missing, or the JSON is malformed.
• 401 — missing or invalid credentials.
• 403 — the evaluator is readable but the caller lacks permission to create a version on it, or Decision Judge is not enabled for the account.
• 404 — evaluator_id does not exist or is not readable by the caller (Evaluator not found), or template_config.llm_config.ai_integration_id does not exist or is not accessible to this space (LLM integration not found or not accessible to this space), or remote_config.integration_id does not exist or is not applicable to this space (Integration not found), or decision_judge_config.ai_integration_id does not exist or is not available in the space (LLM integration not found or not accessible to this space).
• 422 — the body is well-formed JSON but fails validation, for example a missing commit_message, a config mismatch, an invalid template column name, a remote integration with the wrong type, a model_name that is not a Decision Judge model, or questions that break the question rules.
• 429 — rate limit exceeded.

Valid example (template version)

{
  "commit_message": "Improve prompt template for better accuracy",
  "template_config": {
    "name": "hallucination",
    "template": "Given the input: {input}\nand output: {output}\nIs the output a hallucination? Explain your reasoning.",
    "include_explanations": true,
    "use_function_calling": true,
    "classification_choices": {"hallucinated": 0, "factual": 1},
    "llm_config": {
      "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
      "model_name": "gpt-4o",
      "invocation_parameters": {"temperature": 0},
      "provider_parameters": {}
    }
  }
}

Invalid example (missing required commit_message)

{
  "template_config": {
    "name": "hallucination",
    "template": "Is this a hallucination?",
    "include_explanations": false,
    "use_function_calling": false,
    "llm_config": {
      "ai_integration_id": "TGxtSW50ZWdyYXRpb246MTI6YUJjRA==",
      "model_name": "gpt-4o",
      "invocation_parameters": {},
      "provider_parameters": {}
    }
  }
}

Response 422:

{
  "status": 422,
  "title": "Unprocessable Entity",
  "type": "https://arize.com/docs/ax/rest-reference/errors#validation-error",
  "detail": "Invalid input"
}

Invalid example (ai_integration_id does not exist or is not accessible to this space)

{
  "commit_message": "Try a nonexistent integration",
  "template_config": {
    "name": "hallucination",
    "template": "Given {input} and {output}, is it a hallucination?",
    "include_explanations": true,
    "use_function_calling_if_available": true,
    "classification_choices": {"hallucinated": 0, "factual": 1},
    "llm_config": {
      "ai_integration_id": "TGxtSW50ZWdyYXRpb246OTk5OTk6ZmFrZQ==",
      "model_name": "gpt-4o",
      "invocation_parameters": {},
      "provider_parameters": {}
    }
  }
}

Response 404:

{
  "status": 404,
  "title": "Not Found",
  "type": "https://arize.com/docs/ax/rest-reference/errors#resource-not-found",
  "detail": "LLM integration not found or not accessible to this space"
}