Retrieve Model Evaluation Run | DigitalOcean API (unofficial showcase)

3 parameters

eval_run_uuidstringrequired

UUID of the evaluation run.

pageintegeroptional

Page number for per-prompt results (defaults to 1).

per_pageintegeroptional

Number of per-prompt results per page (defaults to 50).

6 status codes

200A successful response.

linksobjectoptional

Links to other pages

metaobjectoptional

Meta information about the data set

resultsarray<object>optional

Paginated per-prompt evaluation results.

runobjectoptional

Model Evaluation Run Detail - full view returned when fetching a specific run.

401Authentication failed due to invalid credentials.

idstringrequired

A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."

messagestringrequired

A message providing additional information about the error, including details to help resolve it when possible.

request_idstringoptional

Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.

404The resource was not found.

idstringrequired

A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."

messagestringrequired

A message providing additional information about the error, including details to help resolve it when possible.

request_idstringoptional

Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.

429The API rate limit has been exceeded.

idstringrequired

A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."

messagestringrequired

A message providing additional information about the error, including details to help resolve it when possible.

request_idstringoptional

Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.

500There was a server error.

idstringrequired

A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."

messagestringrequired

A message providing additional information about the error, including details to help resolve it when possible.

request_idstringoptional

Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.

defaultThere was an unexpected error.

idstringrequired

A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."

messagestringrequired

A message providing additional information about the error, including details to help resolve it when possible.

request_idstringoptional

Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.

Error handling

A 401 is returned: Authentication failed due to invalid credentials. A 404 is returned: The resource was not found. A 429 is returned: The API rate limit has been exceeded. A 500 is returned: There was a server error. Any other status is an error: There was an unexpected error.

import { Client } from "@octridev/digitalocean";

const client = new Client({
  baseUrl: "https://api.digitalocean.com",
  auth: { bearer: process.env.API_TOKEN },
});

const result = await client.modelEvaluationRuns.get({
  evalRunUuid: "9b2f1c10-7a3e-4d54-8c2a-1f6b9e0d4a21",
  page: 1,
  perPage: 20,
});

{
  "links": {
    "pages": {
      "first": "example string",
      "last": "example string",
      "next": "example string",
      "previous": "example string"
    }
  },
  "meta": {
    "page": 123,
    "pages": 123,
    "total": 123
  },
  "results": [
    {
      "candidate_model_name": "example name",
      "candidate_model_uuid": "123e4567-e89b-12d3-a456-426614174000",
      "candidate_routed_task": "Coding & brainstorming",
      "epoch": 1,
      "ground_truth": "example string",
      "input": "example string",
      "metric_results": [
        {
          "error_description": "example string",
          "metric_name": "example name",
          "metric_uuid": "123e4567-e89b-12d3-a456-426614174000",
          "metric_value_type": "METRIC_VALUE_TYPE_UNSPECIFIED",
          "number_value": 123,
          "reasoning": "example string",
          "status": "EVALUATION_METRIC_RESULT_STATUS_UNSPECIFIED",
          "string_value": "example string"
        }
      ],
      "output": "example string",
      "row_number": 1
    }
  ],
  "run": {
    "candidate_inference_config": {
      "max_tokens": 123,
      "reasoning_effort": "low",
      "stop_token": "example string",
      "system_prompt": "example string",
      "temperature": 123
    },
    "candidate_model_name": "example name",
    "candidate_model_source": "CANDIDATE_MODEL_SOURCE_SERVERLESS",
    "candidate_model_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "completed_at": "2023-01-01T00:00:00Z",
    "created_at": "2023-01-01T00:00:00Z",
    "dataset_name": "example name",
    "dataset_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "epochs": 1,
    "error_description": "example string",
    "eval_preset_name": "example name",
    "eval_preset_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "eval_run_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "judge_model_name": "example name",
    "judge_model_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "metrics": [
      {
        "associated_presets": [
          {
            "eval_preset_uuid": "\"12345678-1234-1234-1234-123456789012\"",
            "name": "\"My evaluation preset\""
          }
        ],
        "category": "METRIC_CATEGORY_UNSPECIFIED",
        "custom_eval_config": {
          "created_at": "2023-01-01T00:00:00Z",
          "deleted_at": "2023-01-01T00:00:00Z",
          "requires_ground_truth": true,
          "scoring_prompt": "example string",
          "updated_at": "2023-01-01T00:00:00Z"
        },
        "description": "example string",
        "evaluation_scope": "EVALUATION_SCOPE_UNSPECIFIED",
        "inverted": true,
        "is_metric_goal": true,
        "metric_name": "example name",
        "metric_rank": 123,
        "metric_type": "METRIC_TYPE_UNSPECIFIED",
        "metric_uuid": "123e4567-e89b-12d3-a456-426614174000",
        "metric_value_type": "METRIC_VALUE_TYPE_UNSPECIFIED",
        "range_max": 123,
        "range_min": 123,
        "source": "EVALUATION_METRIC_SOURCE_UNSPECIFIED"
      }
    ],
    "name": "example name",
    "progress": {
      "candidate_rows_evaluated": 100,
      "judge_rows_evaluated": 95,
      "total_rows": 100
    },
    "result_summary": {
      "end_time": "2023-01-01T00:00:00Z",
      "epoch_summary": {
        "avg_at_k_percent": 123,
        "cons_at_k_percent": 123,
        "epochs": 123,
        "pass_at_k_percent": 123,
        "per_epoch": [
          {
            "epoch": 1,
            "overall_score_percent": 123,
            "rows_scored": 123
          }
        ],
        "rows_excluded": 123,
        "rows_scored": 123,
        "score_stddev_percent": 123
      },
      "epochs": 1,
      "metric_summaries": [
        {
          "avg_at_k_percent": 123,
          "cons_at_k_percent": 123,
          "description": "example string",
          "fail_count": 123,
          "fail_percent": 123,
          "metric_name": "example name",
          "metric_uuid": "123e4567-e89b-12d3-a456-426614174000",
          "pass_at_k_percent": 123,
          "pass_count": 123,
          "pass_percent": 123,
          "skip_percent": 123,
          "skipped_count": 123
        }
      ],
      "overall_score_percent": 123,
      "per_model_summaries": {
        "summaries": [
          {
            "metric_summaries": [
              null
            ],
            "model_name": "example name",
            "performance_metrics": {
              "candidate_latency": null,
              "token_usage": null
            },
            "prompt_count": 123
          }
        ]
      },
      "per_task_summaries": {
        "summaries": [
          {
            "metric_summaries": [
              null
            ],
            "performance_metrics": {
              "candidate_latency": null,
              "token_usage": null
            },
            "prompt_count": 123,
            "task_name": "example name"
          }
        ]
      },
      "performance_metrics": {
        "candidate_latency": {
          "avg_e2e_latency_ms": 123,
          "max_e2e_latency_ms": 123,
          "min_e2e_latency_ms": 123,
          "p50_latency_ms": 123,
          "p90_latency_ms": 123,
          "p95_latency_ms": 123
        },
        "token_usage": {
          "total_candidate_input_tokens": "12345",
          "total_candidate_output_tokens": "12345",
          "total_candidate_tokens": "12345",
          "total_judge_input_tokens": "12345",
          "total_judge_output_tokens": "12345",
          "total_judge_tokens": "12345"
        }
      },
      "pricing": {
        "currency": "example string",
        "judge_model_pricing": {
          "input_cost": 123,
          "output_cost": 123,
          "total_cost": 123
        },
        "per_candidate_model_pricing": [
          {
            "model_name": "example name",
            "model_uuid": "123e4567-e89b-12d3-a456-426614174000",
            "pricing": {
              "input_cost": null,
              "output_cost": null,
              "total_cost": null
            },
            "prompt_count": 123
          }
        ],
        "total_cost": 123
      },
      "star_metric_summary": {
        "metric_name": "example name",
        "metric_uuid": "123e4567-e89b-12d3-a456-426614174000",
        "threshold": 123
      },
      "start_time": "2023-01-01T00:00:00Z",
      "total_duration_seconds": 123
    },
    "star_metric": {
      "metric_uuid": "123e4567-e89b-12d3-a456-426614174000",
      "name": "example name",
      "success_threshold": 123,
      "success_threshold_pct": 123
    },
    "started_at": "2023-01-01T00:00:00Z",
    "status": "MODEL_EVALUATION_RUN_STATUS_UNSPECIFIED"
  }
}

Request

Response

Errors