CLI

Every hunch command and flag, what each line of output means, and the exit codes.

hunch <command> <path> [flags]
hunch init [--list] RECIPE [DIR]

path is a spec file or a folder of specs. Use hunch --help to choose a command and hunch run --help (or another command) for its own flags. Commands that evaluate specs lint the project first: warnings go to stderr; an error exits 2. show reads a stored table even if the source file is no longer present.

​

Commands

Command	What it does	Asks the engine?
lint	Checks the specs and prints lint: ok (N judgments)	No
compile	Shows one real request and what run would ask and cost	No
run	Answers every row, caches the answers, writes one table per judgment	Yes, missing answers only
show	Lists the last materialized run and opens one row by key	No
test	Scores the answers against gold and checks the tests: thresholds	Yes, missing answers only
review	Walks you through the rows that need a human verdict	Yes, missing answers only
diff	Compares this spec with an older or other version on today’s rows	Yes, missing answers only
suggest	Asks an LLM to rewrite one question and keeps a rewrite only if it wins	Yes (writer and new answers)
ask	Answers one question for every row of a CSV, and writes the spec for it	Yes, missing answers only
init	Copies a starter recipe into a folder	No
plugins	Lists the engines hunch can use: built in, and from installed plugins	No
install	Installs engine plugins into hunch’s own environment	No
skill	Installs the skill that teaches a coding agent to use hunch	No
hook	Guards a coding agent’s shell commands: install wires it in, run is what the agent calls	Yes, per command
distill	Trains a small local model per question on the answers already in the store	No
docs	Writes a page people can read and search: each judgment’s status, numbers and lineage	No

Commands ask only for answers the store lacks. --max-cost 0 guarantees no spend: if anything is missing, the command stops before asking. For suggest, the cap also covers each writer call at its worst case.

​

hunch ask

hunch ask "Would this command destroy data?" commands.csv                     # yes or no
hunch ask "Which team should handle this?" messages.csv --options billing,technical,sales
hunch ask "Is this urgent?" messages.csv --columns subject,body --name urgent --max-cost 0.05

Writes <name>.yml in the current folder (the name defaults to the question’s first four words that aren’t filler, command_destroy_data above), runs it like run, prints how the answers split and the ten rows it was least sure of, and writes every row’s answer to <name>.answers.csv, in columns named as run’s tables name them: <name>, <name>_p, <name>_route. The key is the id column, else the first; the model sees --columns, else every column but the key and gold_*. Each question gets act: 0.9, so “unsure” means below 0.9. If <name>.yml exists and differs from what ask would write (another question or columns, or your own edits), it stops: pass --name, or hunch run the spec. A spec that fails lint isn’t left behind.

​

hunch plugins and hunch install

hunch plugins                          # built-in engines, and each plugin: prefix, package and version, ok or why not
hunch install hunch-engine-ollama      # into the environment hunch runs in

Engine plugins are Python packages, named hunch-engine-<name> on PyPI. hunch is usually a tool with an environment of its own, so pip install elsewhere doesn’t reach it; install does. In a uv tool install, it rebuilds the tool with the plugin added (uv tool install hunch-ai --with … --with hunch-engine-ollama, keeping plugins added before), since a package put beside the tool would be dropped by the next uv tool upgrade. Packages installed from a folder or git are kept as they were; if one was installed editable, it prints the command to run instead. Anywhere else, it installs into hunch’s interpreter with uv pip, or pip without uv. plugins also shows a plugin that failed to load, and one ignored because it claimed a built-in prefix. See Engines from plugins.

​

hunch init

hunch init --list               # the recipes, one line each
hunch init agent-commands       # copies the recipe into ./agent-commands
hunch init agent-eval evals/    # copies into evals/

The destination folder must not exist. Available recipes:

Recipe	What it tests	Example cost
agent-commands	A guard for 38 real coding-agent shell commands; explained in Guard your coding agent	About $0.001 for the first run; about $0.003 including the wording change
agent-eval	Whether a coding agent’s trace supports its claim of success	Depends on the trace
rag-answers	Whether 900 annotated answers say anything their passages do not support	About $0.03
tickets	Routing 40 made-up support tickets with choice, yes/no and score questions	See hunch compile

​

hunch skill

hunch skill          # into this folder
hunch skill ~/       # for every project on this machine

Writes the hunch skill to .claude/skills/hunch/SKILL.md (Claude Code) and .agents/skills/hunch/SKILL.md (Codex; Cursor reads both). It tells the agent how to run the loop (lint, compile, run with a cost cap, test, diff) and what not to do: spend without --max-cost, write *.reviews.csv, or edit .hunch/. Each file prints installed, updated or up to date; an existing copy is replaced, so re-run after upgrading hunch and commit the result.

​

hunch hook

hunch hook install                   # Claude Code and Codex, in this folder
hunch hook install --agent claude    # one agent: claude, codex or both
hunch hook run hunch/command_guard.yml   # what the agent calls, with the event on stdin (--deny for Codex)

install copies the command guard (spec, 38 labelled commands, NOTICE) into hunch/, keeping files already there, adds .hunch/ to .gitignore, and adds a PreToolUse hook with matcher Bash and a 30-second timeout to .claude/settings.json and .codex/hooks.json, keeping the rest of each file. It writes nothing if a settings file can’t be read. The hook command finds the spec through $CLAUDE_PROJECT_DIR (Claude Code) or the git root (Codex), and ends in || echo a warning, so a missing or older hunch lets commands through instead of blocking them.

run reads the agent’s event, judges the command with request (the last message the person typed, from the Claude Code or Codex transcript; empty if there is none), cwd, description and command, and logs the row, redacted by the spec’s rules, for --traffic. When any answer is yes or routes to review, it replies ask, or deny with --deny (Codex, which cannot ask yet), with the reasons; otherwise it prints nothing. It never replies allow. On any error it prints a systemMessage warning and lets the command through. HUNCH_MAX_COST defaults to 0.01 per command. See Guard your coding agent.

​

hunch distill

hunch distill intent.yml                                  # from the answers for the spec's own rows
hunch distill intent.yml --source banking77_unlabeled.csv # or for these rows
hunch distill hunch/command_guard.yml --traffic           # or for rows logged by judge(..., log=True)

Needs the distill extra. Asks nothing: it reads the store’s answers for --node (default: the last judgment) and writes distilled/<judgment>@<hash>/ next to the spec (meta.json and weights.npz). Per question it trains a linear layer on a frozen MiniLM encoder, class-balanced, on the row’s gold where there is some and the engine’s answer otherwise, holding out a fifth of the gold rows by a hash of their text, and fits a temperature so its confidence is a calibrated probability. It prints each question’s training size and rarest answer, warning under 30 examples, and the lines to measure and use it. See Distill a decision.

​

hunch docs

hunch docs support/
hunch docs support/ --open   # generate, then open the local HTML

Writes two files beside the results file (see below), named after the spec or folder: <name>.html, one self-contained page with no server or network needed, and <name>.manifest.json. It reads the specs, the results of the last test of the same path and the store’s run log, and never asks the engine. See Share what your judgments decide.

​

The manifest

Field	Meaning
version	Manifest format version (1). Field names are stable within a version.
git_sha, project	The commit, and the spec or folder name. No timestamp, so an unchanged project writes an identical file.
target	The --target it ran with, else null. Under a target, each judgment’s model is the engine that target uses.
judgments.<name>	The spec as written, every key it sets (a multi question stays one question), plus file (the spec’s path), spec_hash (the hash test records; the page marks a judgment stale when they differ) and upstream (the judgments it reads).

​

Flags

Flag	Used by	Default	What it does
--version	alone		Prints the installed version and exits.
--node NAME	run, show, test, diff, review, compile, suggest, distill	all (run, test), required when show or review has several judgments	One judgment in a project. run --node X runs X and the judgments it reads from.
--against REF	diff, review	none	diff: the old version, a spec/folder path or git:REF (for example git:HEAD~1); with --model and no --against, the same specs on their own engine. review: put rows where that version answers differently first
--source CSV	evaluation commands	the spec’s source	Run the root judgments on this CSV instead (a holdout set, for example)
--traffic	evaluation commands	off	Run the root judgments on rows logged by judge(..., shadow=...). Not with --source
--model ID	all	the spec’s model	Use this engine for every judgment. Its tables get a suffix (intent__deepseek_deepseek_flash), so the spec’s own tables are untouched
--max-cost USD	commands that can ask the engine	$HUNCH_MAX_COST, else no cap	The most one set of asks may be charged: a judgment’s missing answers, and separately its escalations and test’s order check. Above the estimate, nothing is asked; while asking, a request is sent only if its worst case still fits, so the charge stays under it
--receipt	test	off	Also write the numbers beside what was tested: <spec>.results.json, or results.json in a folder (with the engine’s suffix under --model). Only what the answers decide, no time or cost, so a re-run on cached answers changes nothing: commit it with a battery
--sample N	compile, run, test, diff	all rows	Judge only N rows of each root source, the same N every time
--verbose	run, test, diff	off	Show detailed diagnostics; test includes calibration bins and the full threshold dial
--target NAME	all	the spec as written	Run as the specs’ targets entry of that name says: its engine, sample, store and cost cap. A flag given here wins over the target. diff and review --against run the old side on it too, if the old specs define it; otherwise the old side runs as written
--allow-change	run	off	Accept a changed spec under on_change: freeze
--list	review	off	Print the queue instead of prompting
--limit N	review, show	no limit for review; 20 for show	Limit the review session or row list
--id KEY	show	row list	Print every stored field for one row whose source key equals KEY
--open	docs	off	Open the generated HTML report in a browser
--audit N	review	30	Random rows to spot check per question, counting ones already reviewed
--reviewer NAME	review	your login name	Name saved with each verdict
--question QID	suggest	required if the spec has several questions	The question to rewrite
--n N	suggest	3	Rewrites to try
--writer MODEL	suggest	deepseek:deepseek-flash	The LLM that writes rewrites

--sample picks rows by a hash of the key, so a repeated sample comes from the store and a later full run asks only the rest. run --sample caches its answers but does not replace any table, since downstream readers would take the sample for the whole.

​

Exit codes

Code	Meaning
0	Done. For test: no blocking configured check failed; the result may be measured or not assessed rather than passing
1	test: at least one FAIL line. Any command: stopped with a message (above --max-cost, on_change: freeze refused, unknown judgment)
2	Lint error in a spec, or bad arguments

test exits 1 both when a check fails and when --max-cost stops it from asking (for example, order_stability needs new answers). Read the last line of output to tell them apart.

​

Reading the output

​

compile

# intent: request for row 1 (one request per row, all questions read once)
{
  "model": "jev-1.13.0",
  "state": {
    "text": "I still have not received my new card, I ordered over a week ago."
  },
  "questions": { "intent": { "type": "choice", ... } }
}

# intent: 770 rows in; 770 answers planned, 770 cached, 0 to ask, ~0 input tokens, ~$0.00000 (jev-1.13.0)

Line	Meaning
JSON	The exact request for the first row, after redact and clip. For an LLM engine, the prompt.
# <judgment>: …	Rows in, answers planned, cached, to ask, estimated tokens and cost.
where keeps ≤ N, rows kept because the answers their where-clause needs aren't cached yet	Upstream hasn’t run; count and cost are upper bounds.
size warning	A row’s state is near the engine’s limit.
on_change: freeze and the spec changed	run will refuse without --allow-change.
# total	The whole project, with an expected cost when some rows are upper bounds.

​

run

run judges missing answers, caches them, and writes one table per judgment. Its summary starts with rows and answers, then names the table and store:

Run complete · lobbying.yml
  lobbying_review: 8 rows in → 8 written
    answers: 0 cached, 8 asked · $0.00015
  lobbying: yes 6, no 2
  Tables: lobbying_review · Store: .hunch/store.sqlite
  Run ID: 2026-09-27T19:50:41-f349b6
  Inspect: hunch show lobbying.yml
  Measure: hunch test lobbying.yml --max-cost 0

--sample N caches answers but does not replace a table. The summary says so and does not suggest show as if the sampled rows had been written. A repeated full run can use cached answers. For request timing and tokens, add --verbose.

​

show

hunch show lobbying.yml                # last materialized run, a short row list
hunch show lobbying.yml --id e06       # every stored field for one row
hunch show project/ --node lobbying_review --limit 50

show reads the last materialized table without asking the engine or reading the source file. It names the run ID and warns when the current spec differs from the spec that wrote the table. test and sampled run do not replace that table. The row list shows the source key, answer, confidence and route; --id shows the stored input fields as well. The HTML report below is intended for sharing and excludes input row text.

​

test

test measures answers against gold and evaluates acceptance checks you wrote under tests:. With no configured bar, it says measured rather than passed:

Test complete · lobbying.yml
  Measured; no acceptance checks configured
    lobbying: 6/8 agree with gold (75.0%)
      AUROC 1.000 · calibration error 0.198
      2 disagreements with gold; first: e06, e05
  Answers: 8 cached, 0 asked · $0.00000
  Results: .hunch/target/lobbying.json
  More: hunch test lobbying.yml --max-cost 0 --verbose
  Browser report: hunch docs lobbying.yml --open

A configured check appears as PASS, WARN, or FAIL with its measured value and limit. If it needs gold that is absent, it appears as NOT ASSESSED; the result cannot be called passing. test exits 1 for a failed check with normal severity. --verbose adds calibration bins, the full threshold dial, baseline comparison, and diagnostic examples. Accuracy estimated from reviewed rows keeps its 95% interval and basis in both modes. See How the numbers work.

​

The results file

When test finishes, it also writes what it printed as JSON, for CI, dashboards and agents (its schema and what may change between versions: Files hunch writes): .hunch/target/<tested path>.json, beside the store, with the path of the tested spec or folder relative to the store’s folder (hunch test examples/tickets/triage.yml writes .hunch/target/examples/tickets/triage.json). With --model, the engine’s suffix is added, as for its tables (triage__deepseek_deepseek_flash.json), so another engine’s results never replace the spec’s own. With --target, its name is added ([e-mail]), and the file goes in target/ beside the target’s store. test removes the file when it starts, so a run stopped by --max-cost leaves none rather than an old one.

Field	Meaning
version	Results format version (2 for new tests; version 1 receipts remain readable).
command, at, git_sha	test, when it ran (UTC), the commit it ran on.
passed	No blocking configured check failed (the exit code was 0). This can be true when nothing was assessed. Read assessment for the meaning.
assessment	failed, warning, unassessed, passed, measured, or no_gold; separates a completed measurement from acceptance.
sample	N if test ran with --sample N or a target’s sample, else null.
target	The --target it ran with, else null.
cost	Spent by this run, in USD.
judgments.<name>	spec_hash, model, rows, and questions (and multi, for type: multi).
questions.<q>.gold	Rows with gold: rows, and by origin source, review, excluded.
questions.<q>.accuracy	value, ci ([low, high], or null when basis is gold: plain accuracy against gold, no estimate), basis (estimate or gold), reviewed (per group: reviewed of of rows).
questions.<q>.calibration_error, auroc	As printed; auroc for yes/no questions with both answers in gold.
questions.<q>.act	With act in the spec: threshold, automated (share of all rows acted on), judged (acted-on rows with gold), wrong (share of those that are wrong; null if none has gold).
questions.<q>.dial	Per threshold: automated and wrong (shares); yes/no questions have them under yes and no.
questions.<q>.mistakes	total, and most_confident: id, got, p, gold.
questions.<q>.checks	Only configured acceptance checks. Each has check (the test key), passed, value, limit, severity. Measurements remain in their named fields even without a check.
questions.<q>.unavailable_checks	Configured checks that could not run, usually because suitable gold or reviews are absent.
questions.<q>.spot_checks, needs_context	Random spot checks, and how many of them needed more context.
questions.<q>.recall	Yes/no questions with gold-yes rows: yes_rows; with act for no, at_act; with min_recall, lowest_act (each threshold, value, ci, read). See tests.
questions.<q>.baseline	With baseline: rule, rows, said_yes, the rule’s accuracy, auroc, recall, precision, the same under model, and model_ahead (the model’s accuracy is above the rule’s). See baseline.
judgments.<name>.metrics.<m>	rule, rows, fired, rate (with weights, weighted, plus ci and effective_rows); with gold, gold (basis, rows), and gold_rate, missed, false_alarms (each count, of, rate, ci, with weights effective_rows); checks. See metrics.
judgments.<name>.metrics.<m> with by	Also by, no_group (rows with no value), groups (per value: rows, fired, rate, ci, with weights effective_rows, with gold gold_rate), and with a higher test, higher (groups, difference, ci, and with gold in both groups, gold: difference, ci). See per group.
judgments.<name>.examples	Per example: name, passed, and answers (question, expected, got, p, passed). See examples.

​

diff

diff compares two versions on the same rows. It leads with changed answers; --verbose adds probability movement, the full gold comparison, and more row examples.

Diff · intent.yml against intent_bare.yml
  intent: 77/770 shared rows changed answer; 15 near the decision boundary
    76: activate_my_card → card_linking
    255: fiat_currency_support → exchange_via_app
    268: exchange_rate → exchange_via_app
    72 more changed rows
    Gold on changed rows: 58 fixed, 8 broken; paired p=0.000 (significant).
  Answers: 1540 cached, 0 asked · $0.00000

The count includes rows whose top answer changed, while newly entered or removed rows are named separately. “Near the decision boundary” means one side’s answer margin was under 0.10; read those changes cautiously. The gold line uses today’s reviews on both sides. If the spec names exposures, the summary also counts which consumers can see the changed answers. A changed prompt does not automatically mean better answers: compare fixed and broken rows, then review those without gold.

​

review

See Review for the screen and keys. --list prints one row per item:

review queue: 112 disputed, 30 audit
  disputed  #  281 intent     card_arrival 1.00                    gold=card_delivery_estimate   my card was not in the mail again can you advise?

Columns: kind, row id, question, model answer and confidence, answer key (- if none), start of the row.

Verdicts saved to <judgment>.reviews.csv (columns qid,row_id,state_hash,verdict,label,reviewer,at,kind):

Verdict	Saved when
model_right / key_right	On a disputed row, you picked the model’s answer / the answer key
against_right / spec_right	On a shadow row, you picked the --against spec’s answer / this spec’s
confirmed	On a spot check, you picked the marked answer
labeled	You picked any other option
both_ok	b: two answers are acceptable; both count as right
ambiguous	a: no single right answer; left out of scoring
needs_context	c: the text shown is not enough to judge; left out of scoring and counted in test’s context line

kind says why the row was reviewed: disputed, audit (a random spot check), uncertain, shadow, or any other name you write when importing verdicts (for example flagged). Only audit rows feed the accuracy estimate; every verdict counts as gold. test also marks a row same_text when it takes its gold from another row with identical text: gold, but not a spot check.