Plugin Eval

Plugin Eval

plugin-eval is both:

• a local Node.js CLI
• a Codex plugin bundle

It helps engineers evaluate a local skill or plugin, understand why it scored that way, see what to fix first, explain token budgets, measure real usage, and decide what to do next without having to memorize a command sequence first.

What This Plugin Contains

• scripts/plugin-eval.js: the CLI entrypoint exposed as plugin-eval
• .codex-plugin/plugin.json: the Codex plugin manifest
• skills/: the plugin's chat-facing skills

The plugin is designed to feel chat-first in Codex, while still routing to explicit local commands you can run yourself.

Install As A CLI Tool

Requirements

• Node.js >=20

This package is currently marked "private": true, so the expected install path is from a local checkout rather than the public npm registry.

Run It Without Installing Globally

From the plugin root (plugins/plugin-eval in this monorepo):

node ./scripts/plugin-eval.js --help

You can use that form for every command in this README.

Examples:

node ./scripts/plugin-eval.js analyze ./skills/plugin-eval --format markdown
node ./scripts/plugin-eval.js analyze . --format markdown

Install A Global plugin-eval Command

From the plugin root (plugins/plugin-eval in this monorepo):

npm link

After that, plugin-eval should be available on your PATH:

plugin-eval --help
plugin-eval analyze ./skills/plugin-eval --format markdown

If you prefer not to create a global link, keep using node ./scripts/plugin-eval.js ... directly.

CLI Usage

Start From Chat

start is the chat-first router:

plugin-eval start <path> --request "<chat request>" --format markdown

Examples:

plugin-eval start ~/.codex/skills/game-dev --request "Evaluate this skill." --format markdown
plugin-eval start ~/.codex/skills/game-dev --request "Why did this score that way?" --format markdown
plugin-eval start ~/.codex/skills/game-dev --request "What should I fix first?" --format markdown
plugin-eval start ~/.codex/skills/game-dev --request "Measure the real token usage of this skill." --format markdown
plugin-eval start . --request "Help me benchmark this plugin." --format markdown

plugin-eval start keeps the workflow chat-first: