Установка

$

npx skills add https://github.com/openai/plugins/tree/5fd93af4cd0c623e020d0cc7e9ce178b4ac1f70f/plugins/plugin-eval/skills/metric-pack-designer

Ставит скилл в текущий проект - CLI спросит, для каких агентов. С флагом -g - в домашнюю папку, для всех проектов.

Описание

Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations.

SKILL.md

Исходник на GitHub

Metric Pack Designer

Use this skill when the user wants to extend plugin-eval with a local rubric.

Workflow

Clarify the custom rubric categories and target kinds.
Define the smallest useful checks[] and metrics[] payload.
Create a metric-pack manifest plus a script that prints JSON to stdout.
Run the pack through plugin-eval analyze <path> --metric-pack <manifest.json>.

Design Rules

Keep IDs stable across runs so comparisons stay meaningful.
Emit only checks[], metrics[], and optional artifacts[].
Do not try to overwrite the core score or summary.
Prefer deterministic local signals over subjective text generation.

Reference

../../references/metric-pack-manifest.md