List built-in judge templates

GET

/api/v1/agent_version/judge_templates

const url = 'https://app.prefactorai.com/api/v1/agent_version/judge_templates';
const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}

curl --request GET \
  --url https://app.prefactorai.com/api/v1/agent_version/judge_templates \
  --header 'Authorization: Bearer <token>'

Authorizations

authorization

Parameters

Query Parameters

idempotency_key

One of:

string
Null

Optional idempotency key for ensuring request idempotency

string

Responses

200

Success

application/json

object

default_system_prompt_template

required

Liquid template wrapped around a judge criterion. The only variable is criterion.

string

status

required

string

Allowed values: success

templates

required

Array<object>

AgentVersion_JudgeTemplate

object

criterion

required

Rubric text inserted into the system prompt

string

description

required

What this template checks

string

key

required

Stable identifier for this template

string

name

required

Display name of the template

string

Example

{
  "default_system_prompt_template": "You are a judge.\n\n{{ criterion }}",
  "status": "success",
  "templates": [
    {
      "criterion": "Decide whether the agent completed every request that was still active at the end of the run.",
      "description": "Checks that the agent completed every request it was given.",
      "key": "completeness",
      "name": "Completeness"
    }
  ]
}

default

Error

application/json

One of:

object

code

required

string

Allowed values: bad_request bad_authtoken not_authenticated not_permitted not_implemented unknown not_found unexpected_ref_type unexpected invalid_action invalid_state alert_unknown alert_already_cleared invalid_value required_value idempotency_key_already_used temporarily_unavailable conflict

message

required

string

status

required

string

Allowed values: error

Example

{
  "code": "bad_request",
  "status": "error"
}