DigitalOcean API (unofficial showcase)
POST/v2/gen-ai/custom_evaluation_metrics

Create a custom evaluation metric

Creates a custom LLM-as-judge metric for model evaluation. Use metric_name, description, and config to define the metric and the judge instructions; prompt and model response are included in the judge context. Set requires_ground_truth in config when each evaluation row must provide ground truth.

  • IdempotentThe SDK sends Idempotency-Key, so a retried request is only applied once.

3 body fields

Custom metric details, including its name, description, and optional LLM-judge configuration.

configobjectoptional
Configuration for a custom model-evaluation metric scored by an LLM judge. Prompt and model response are always included in the judge context.
descriptionstringoptional
A description of the custom evaluation metric.
metric_namestringoptional
The name of the custom evaluation metric.

6 status codes
200Returns the created metric and its details, including its name, description, evaluation scope, metric type, and UUID.
metricobjectoptional
401Returned when authentication fails because the credentials are invalid.
idstringrequired
A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."
messagestringrequired
A message providing additional information about the error, including details to help resolve it when possible.
request_idstringoptional
Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.
404Returned when the requested resource is not found.
idstringrequired
A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."
messagestringrequired
A message providing additional information about the error, including details to help resolve it when possible.
request_idstringoptional
Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.
429Returned when the API rate limit has been exceeded.
idstringrequired
A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."
messagestringrequired
A message providing additional information about the error, including details to help resolve it when possible.
request_idstringoptional
Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.
500Returned when the server encounters an error.
idstringrequired
A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."
messagestringrequired
A message providing additional information about the error, including details to help resolve it when possible.
request_idstringoptional
Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.
defaultThere was an unexpected error.
idstringrequired
A short identifier corresponding to the HTTP status code returned. For example, the ID for a response returning a 404 status code would be "not_found."
messagestringrequired
A message providing additional information about the error, including details to help resolve it when possible.
request_idstringoptional
Optionally, some endpoints may include a request ID that should be provided when reporting bugs or opening support tickets to help identify the issue.

Error handling

A 401 is returned when credentials are invalid. A 404 is returned when a requested resource is not found. A 429 is returned when the API rate limit is exceeded. A 500 is returned when the server encounters an error. Any created_at, deleted_at, or updated_at value in config must use date-time format.