Introduction

When a model’s answer becomes a rule in your software, hunch helps you run it, test it against reviewed cases, and see which answers a change would flip.

One Enron email proposes comments on a regulator’s draft rule. Another reports that a rate relief bill passed. Both concern public policy. Which belongs in a review of lobbying efforts? The quickstart lets you ask a model, then check its answers against published judgments.

The harder job is applying that distinction to thousands more emails, then changing the question without losing track of what moved. A probability for one answer cannot tell you the rule’s error rate. Reviewed cases and a record of what changed can.

​

Why not call Jev directly?

For a one-off question, call Jev directly. Jev already gives you typed answers, probabilities and several questions in one call. hunch is useful when the answer becomes part of a workflow you need to run again, measure, review and change safely.

Calling Jev directly	With hunch
You write the prompt, schema and input loop in code.	The prompt, typed answers, inputs and examples live in one YAML spec in git.
You decide how to batch, retry, cap spend and store results.	hunch run handles batches, max cost and a content-addressed store, so unchanged rows can reuse previous answers. See 100,000 reviews.
You can keep probabilities, but you still need reviewed cases to know the error rate.	hunch test compares answers with gold or reviewed cases and reports whether the rule meets the checks you set. See document review.
You build your own review queue when labels are missing or disputed.	hunch review presents a local queue of uncertain and useful rows. If review happens in your app, you wire that handoff and import the verdicts. See the review guide.
A prompt edit may silently change downstream decisions.	hunch diff shows the rows whose answers would flip before the edit ships. See the session cookbook.
Your app calls the model from request code.	hunch.judge() calls the same spec and store your batch runs, tests and docs use.

You usually do not need hunch for a single request-path call with no reviewed data, no reruns and no plan to compare changes. The direct API is simpler there.

​

If you know dbt

dbt keeps SQL transformations in versioned models with tests. A hunch spec is the versioned judgment definition. hunch run applies it, hunch test measures it, and hunch diff compares an edit before you adopt it. You can also call the same spec from an app or DuckDB query.

​

Where decisions live

The same shape, a question, a typed answer, a bar for acting and a person for the rest, turns up across a whole company:

Where	Decisions like
Coding and AI agents	Is this command safe to run? Did the agent really fix it? Should it stop or go on? Did this change leave a comment wrong?
Data pipelines	Turn text into columns you can query: which calls mention churn, which reviews are about quality, which contracts auto-renew. Which records in two lists are the same thing?
Product	Does this post break the rules? Is this user eligible? Which requests are urgent?
Operations and security	Is this alert worth paging for? Is this email phishing? Should this access request be approved?
Risk and compliance	Is this expense within policy? Does this document meet the requirement? Is this claim covered?
AI output checks	Is the answer supported by its source? Does it leak personal data? Did it follow the instructions?

The same workflow can serve each case: define the answer, inspect the evidence, and decide what may happen without a person.

​

Worked examples

Each cookbook starts with a real dataset, shows the question and its results, and explains how those results were measured.

A guard for an agent's shell commands

Over 1,315 real commands, it stopped 153 for a person to check, about 1 in 9. In a random sample of the commands it let run, 1 in 91 should have been stopped.

Agents say they fixed it

An agent claimed a fix in 160 of 200 runs; 66 of those failed the tests. Of the 55 patches the model gave under a 20% chance, 52 failed.

Did that turn work?

The developer’s next message says whether a coding agent’s turn worked. Read by a model, it was right about 9 times in 10, and diff caught a “clearer” wording that did worse.

100,000 reviews in one run

96,000 new requests cost $1.55; 4,000 answers were already cached. A fully cached re-run took 4.1 seconds and cost nothing.

​

How it works

Write a spec, run it on rows, review a sample, and test the answers. Before changing the spec, use diff to see which rows would change. Concepts shows the decision and review paths.

Here is the first Enron quickstart spec. It reads emails and asks one yes/no question, without an action threshold yet:

lobbying.yml

judgment: lobbying_review
model: jev-1.13.0
source: emails.csv
key: id
state: [subject, body]

questions:
  lobbying:
    type: noul
    instructions: Does this email describe an attempt to influence a law, regulation, or public policy?
    gold: gold

hunch run asks the question for each email and stores each answer under its exact input. hunch test compares the answers with reviewed labels and reports the mistakes. Then you can edit the wording and use hunch diff to see which emails would change before the new wording ships. Inside your app, hunch.judge() answers one row at a time from the same spec and store. A Pydantic AI agent on a decision model can be recorded and tested on past rows. hunch docs makes a project report for whoever signs off on the policy.

Quickstart

Start with Enron emails, run a small lobbying judgment, test it and inspect what a wording change would flip.