> ## Documentation Index
> Fetch the complete documentation index at: https://docs.selfbench.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# CLI: Evaluate

> Run models and harnesses on approved tasks, and follow comparisons.

`R` is `--repo OWNER/NAME`, or another [scope](https://docs.selfbench.dev/cli/overview.md#conventions).

| Command | Does |
| - | - |
| `catalog R` | Models available for evaluations |
| `catalog R --harness H` | Options a [saved harness](https://docs.selfbench.dev/cli/harnesses.md) can set on agent `H` |
| `compare R --model ID [--model ID ...]` | Run each model on every approved task |
| `comparisons R` · `comparison show\|resume\|cancel ID R` | Follow, resume, or stop comparisons |
| `runs R` · `run show\|cancel\|delete EVALUATION_ID R` | One model's evaluation, with every trial |

## Compare Flags

| Flag | Default |
| - | - |
| `--harness H` (repeatable): `pi`, `codex`, `claude-code`, `mini-swe-agent`, `terminus-2`, `muse-code` | `pi` |
| `--thinking LEVEL`: `off`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max` | the model's default |
| `--credential ID` | managed model |
| `--sandbox managed\|e2b\|modal\|daytona` and `--sandbox-credential ID` | managed sandbox |
| `--task RUN_ID/TASK_ID` (repeatable) | every approved task |
| `--skip-completed` | rerun everything |
| `--judge-credential ID` | required for tasks graded by an LLM judge |
| `--saved-harness ID` (repeatable): a [saved harness](https://docs.selfbench.dev/cli/harnesses.md) | none |
| `--environment ID` (repeatable): a repository [environment](https://docs.selfbench.dev/cli/environments.md) | none |
| `--no-baseline` | the tasks as they are run too |
| `--selection JSON\|@FILE` | a full [comparison body](https://docs.selfbench.dev/api/evaluate.md#request-body) in place of the flags |

Every task runs again with each saved harness, in each environment, and with each harness in each environment (at most 12), beside the tasks as they are; see [Harnesses and Environments](https://docs.selfbench.dev/api/evaluate.md#harnesses-and-environments). The older `--saved-condition`, `--condition`, and `--harness-option` flags still work.

A harness with a subagent model runs each model as the orchestrator and the subagent model for delegated work, on the credential the harness names; see [Subagent Models](https://docs.selfbench.dev/api/evaluate.md#subagent-models).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.