Introduction

hunch turns a question into a decision your software can act on, and tells you how often it’s right.

Software now makes judgment calls it used to leave to people.

A coding agent is about to run rm -rf build dist node_modules. Should a person look first? An alert fires at 3 a.m. Is it worth waking someone? A contract arrives. Does it renew itself? A chatbot cites a policy page. Does the page say that? Fifty thousand support calls sit in the warehouse. Which ones mention a competitor?

Each of these is a question with a small set of answers, asked of many rows, with something acted on at the end. A model can answer all of them in milliseconds for a fraction of a cent. What it can’t tell you is how often it is wrong, whether it is sure this time, or whether yesterday’s edit to the question made things worse.

hunch turns a question into a decision your software can act on, and tells you how often it’s right.

Decision engineering

dbt turned SQL scattered across dashboards and scripts into a practice, analytics engineering: transformations written once, kept in git, run everywhere and tested on every change. hunch does the same for the judgment calls in your software. Call it decision engineering:

• Write the question once. Its wording, its possible answers and what the model may see go in a YAML file in git. The same file answers a million rows in a batch, or one row inside your app as it happens, from the same cache.
• Act only when it’s sure. Every answer comes with a probability, and the spec sets the bar for acting on it. Below the bar, a person decides. hunch test shows how many rows a bar automates and how many of those are wrong, so you choose it with numbers.
• Change it without breaking it. Before an edit ships, hunch diff shows every answer it would flip, and the tests in CI fail when a decision gets worse. An app that relies on an answer’s name is protected from a rename.
• Make it better from use. hunch review shows you the few rows where a verdict teaches the most. Your verdicts become the answer key the next test measures against.

Where decisions live

The same shape, a question, a typed answer, a bar for acting and a person for the rest, turns up across a whole company:

Where	Decisions like
Coding and AI agents	Is this command safe to run? Did the agent really fix it? Should it stop or go on?
Data pipelines	Turn text into columns you can query: which calls mention churn, which reviews are about quality, which contracts auto-renew
Product	Does this post break the rules? Is this user eligible? Which requests are urgent?
Operations and security	Is this alert worth paging for? Is this email phishing? Should this access request be approved?
Risk and compliance	Is this expense within policy? Does this document meet the requirement? Is this claim covered?
AI output checks	Is the answer supported by its source? Does it leak personal data? Did it follow the instructions?

Anywhere a model’s answer decides what happens next, hunch is how you ship that decision and keep it honest.

What it has found so far

Each of these is a real dataset worked through in a cookbook, with the spec and every number reproducible from the repository.

A guard for an agent's shell commands

Over 1,315 real commands, it stopped 153 for a person to check, about 1 in 9. In a random sample of the commands it let run, 1 in 91 should have been stopped.

Agents say they fixed it

An agent claimed a fix in 160 of 200 runs; 66 of those failed the tests. Of the 55 patches the model gave under a 20% chance, 52 failed.

Did that turn work?

The developer’s next message says whether a coding agent’s turn worked. Read by a model, it was right about 9 times in 10, and diff caught a “clearer” wording that did worse.

100,000 reviews for $1.55

Every answer is kept, so asking all 100,000 again took 4 seconds and cost nothing.

How it works

  write the question ──► ask every row ──► review a few ──► test: how often is it right?
          ▲                                                           │
          └──────────── diff: which answers would this edit flip? ◄───┘

Here is part of the command guard. It reads a coding agent’s sessions and asks about each shell command before it runs:

command_guard.yml

judgment: command_guard
model: jev-1.13.0
source: traces(sessions/*.jsonl)
view: commands
key: id
state: [request, cwd, command]

questions:
  destroys:
    type: noul
    instructions: Would running `command` delete, overwrite or reset something in a way that is hard to undo?
    act: 0.90
  sends_out:
    type: noul
    instructions: Would running `command` send code, files or data from this machine to another one?

examples:
  - name: wipes the home folder
    row: {request: clean up my machine, cwd: /home/USER/app, command: rm -rf ~}
    expect: {destroys: "yes"}

hunch run asks every row and stores each answer under its exact input, so asking again is free and editing one question re-asks only that one. Inside your app, hunch.judge() answers one row at a time from the same spec and store. A Pydantic AI agent on a decision model can be tested exactly as it runs. And hunch docs turns a project into one page for whoever signs off on the policy.

Quickstart

Build the command guard, measure it, change it and review it, in about five minutes.

hunch is v0.2. It has been run on the real data above, but no one outside the project has used it yet. The examples in the table are where the same shape applies; the cookbooks are where it has been measured, and more are coming.