You rent a frontier mind for every decision.
Most were never a frontier problem.

1 · The source, and what it proves

This paper starts from Small Language Model Engineering 2026, an independent study note that synthesises Andrej Karpathy’s open-source repositories, discussion threads and public posts. The note is not written, reviewed or endorsed by Karpathy, and neither is this paper. The figures below are as Karpathy published them in his nanochat discussions, reached through that note. We credit both and claim neither.

The central result is nanochat, released in October 2025: one readable repository that takes raw text all the way to a chat model through four named stages — pretraining, midtraining, supervised finetuning and optional reinforcement learning. A January 2026 miniseries extended it into a ladder of model sizes, each set by a single dial, with a published cost for each rung.

Three lessons carry over directly to enterprise systems.

• Ownership has a price, and it is small at the bottom. A model that beats the 2019 state of the art costs about a thousand dollars to train. General capability is what becomes expensive; narrow jobs sit on the cheap rungs of the ladder.
• Small models are poor generalists. The $92 model scores close to chance on a broad knowledge test and solves roughly one grade-school maths problem in twenty. Nobody should mistake it for an assistant.
• Method beats model. One dial instead of a hyperparameter sweep, staged training where each stage has a named job, and measurement against something you did not train yourself. Those disciplines transfer even where the model itself does not.

The question “could we just train one?” has stopped being rhetorical. It is now a number you can put next to the invoice you already pay.

2 · The decision economy

An agent loop is mostly decisions, not thought. Before an AI worker drafts a single reply it has already asked a series of short questions: what is this, how urgent is it, which tool applies, is the input complete, is the next action safe, was the last output acceptable. Each answer is short, and each is drawn from a fixed set of options.

Those decisions dominate volume. A single customer request triggers several of them, and in most stacks a frontier model answers every one. The bill grows with traffic, the latency stacks up step by step, and two runs on the same input can disagree with each other — which is exactly what an auditor does not want to hear about a decision that moved money or touched a customer.

The dividing line

What can move off the frontier model follows a simple rule.

• If the answer is a function of the input, a small model can learn it. Routing, triage, classification, completeness checks, risk gating, format validation.
• If the answer needs knowledge that is not in the input, it cannot. Explaining a regulation, drafting a letter, weighing a credit case, planning a piece of work.

The second kind stays with the frontier model — or, better, with a model that looks the facts up rather than remembering them. That is Karpathy’s own picture of where small models are heading: a compact “cognitive core” that sacrifices encyclopaedic recall for capability and consults the world for facts. Most of the first kind was never a frontier problem at all.

3 · Three minds for decisions

Three kinds of model can now make a routine decision. They differ less in what they can do than in what the enterprise ends up holding.

Jev, launched in early access by TypeSafe AI in September 2026, is the first public “System One” model, named for Kahneman’s fast, intuitive mode of thought. It gives up text generation entirely: the system defines the answer space and the model returns a probability distribution over it. That is why it cannot invent an option, though it can still choose the wrong one. Its real contribution is the interface: typed answers, honest confidence, and decisions decomposed into atomic questions recombined in deterministic code.

The owned small model is what Karpathy’s work makes affordable. It has none of Jev’s built-in calibration, and it must be trained, served and watched. But it has the one property neither rented option can offer: it never leaves the enterprise’s control.

4 · How they come together

The three minds are not rivals; they are rungs. Each decision starts at the cheapest mind that might answer it and climbs only when that mind is unsure. Above them all sits a deterministic layer that has the final say: whatever tier answers, rules check the answer, execute it and record it.

Figure 1 · The decision cascade

1. Rules and entitlementsDeterministic mind. Decides whatever it can, first. If no rule applies, the decision moves down.
2. Owned small modelDistilled from your own traffic, in your network. Answers when confident; passes on when unsure.
3. System One model (e.g. Jev)Optional; only where data may leave the region. Passes on when unsure, or when not permitted.
4. Frontier modelLanguage, judgement, and cases the tiers above doubt. Passes on when unsure.
5. Person at a judgement forkSees only what no model is confident about, and decides.
6. Deterministic mind actsWhichever tier answers, it checks entitlements and limits, executes the governed action and writes the immutable audit. Every escalated case becomes training data for the owned model.

Three properties of the cascade matter to a regulated buyer.

• One contract, swappable minds. Each decision is defined once: its input, its allowed answers, its confidence threshold and its escalation rule. The minds behind it are adapters. Changing a model does not change the system.
• Residency is a routing rule. For data that may not leave the region, the System One tier is simply skipped and the owned model hands straight to the frontier tier, which can itself be an in-region deployment.
• The system improves where it is uncertain. Every case escalated past the owned model becomes training data for its next release. The frontier model’s share of decisions falls as the owned model learns, and the person at the top sees genuine forks only.

This sits naturally inside an architecture that already separates a deterministic mind from a governed language mind, as earlier papers in this series have described (the Two Minds architecture; Paper Seven). The decision layer is not a new product category. It is a socket that a well-governed system already has room for.

Rules decide first and act last. Everything in between is a question of which mind is cheapest and still confident.

5 · What it actually takes

Almost nobody should train from scratch. The study note sets out four routes to a small model that does one job, in rising order of effort. The right one is the cheapest that clears the bar.

For an enterprise already running AI, distillation is the right default, because the training data already exists in its logs.

Figure 2 · The distillation loop

1. Log real trafficInputs the frontier model already decides.
2. Keep the answersFrontier cost paid once, over existing traffic.
3. Finetune and calibrateSmall open model on the input–answer pairs.
4. Test on held-out setLabelled by the people who own the work.
5. Ship behind a gateAnswers when confident, escalates when unsure.
6. Collect escalationsCases the training set did not cover. Retrain on them, each release, from step 3.

Two engineering facts decide whether it works

• Calibration is not free. A finetuned small model’s raw scores are not honest probabilities. Constrain it to a fixed label set, then calibrate its scores against a held-out set before setting the gate threshold. Jev is trained for this; an owned model has to be tuned for it.
• Size decides where it runs. Weight memory is roughly parameters multiplied by bytes per parameter. A three-billion-parameter model needs about 6 GB at 16-bit precision and about 1.5 GB at 4-bit, which fits a single modest GPU in the enterprise’s own data centre. The accuracy lost to compression must be measured on the enterprise’s own task, not on a public benchmark.

6 · The economics

At volume, the frontier bill is the one that scales; the other two barely move. Figure 3 works through a single routine classification decision of about 800 input and 20 output tokens.

Figure 3 · Monthly cost of one routine decision, by daily volume

The frontier and GPU figures follow the study note’s worked example; the System One line is our own arithmetic from the vendor’s published price. Owning also carries a one-off of a few hundred dollars for the teacher pass. The honest reading has three parts.

• On price alone, renting a System One model wins. Owning a model is not the cheapest way to make a routine decision.
• Owning wins on everything price leaves out. No data leaves the network, nothing reprices or deprecates, and the decision behaves the same way next quarter as it does today. For regulated buyers in the Gulf, that usually decides the matter.
• Below the break-even — about 5,000 decisions a day in this example — keep renting. The engineering effort to own a model costs more than the bill it saves.

The threshold question is not “is the small model as good?” It is “at what volume does the recurring bill exceed the one-off cost of owning?”

7 · Where it works

The best candidates share four traits: high volume, a fixed answer set, an answer drawn from the input alone, and existing traffic to learn from. Across the kinds of system enterprises are now putting into production, these fit well.

Agent action gating is the clearest first win: it is the most frequent decision in any agentic system, its labels are stable, and every escalation is already a human checkpoint. Classification of financial flows that carry no personal data is the cleanest pilot for an owned model, because it removes the residency question from the first experiment.

8 · Where it does not, and the risks

Small models fail confidently when they are asked to know things. Anything that depends on world knowledge, multi-step reasoning or careful writing stays with the frontier model: drafting, underwriting judgements, compliance answers, specifications and planning.

The risks of a decision layer are known ones, and each has a known control.

9 · What to look for in a delivery model

An enterprise does not need to build a decision layer from first principles. It needs whoever builds its systems to have made room for one. Five tests separate a system that can host a decision layer from one that will have to be rebuilt to get it.

1. Does the deterministic layer decide first and act last? If a language model sits in the decision path with the final word, no amount of tiering will make the system governable.
2. Is each decision defined as a contract? Input, allowed answers, threshold and escalation rule, written down once, with the model behind it treated as an adapter.
3. Does the system run with the language layer down? Every model-touched workflow should have a rules-only fallback. An owned small model makes that fallback far more capable; it should not be the first thing that ever sat there.
4. Who owns the model trained on your traffic? A distilled model is built from the enterprise’s own data. It should be the enterprise’s asset, in its own network, under the same terms as the source code.
5. Does the commercial model reward a cheaper decision? If the deliverer is paid by usage of a rented model, it has no reason to move decisions off it. If it is paid on outcome or hands over the source, it does.

10 · A phased path to adoption

Start with one decision, not a platform. Each phase has a gate, and most decisions stop well before the last phase.

None of this makes a small model the equal of a frontier one, and nothing in Karpathy’s work claims it does. What it removes is the mystery. Having trained or distilled one model for one decision, an enterprise will know which of the thousand calls in its system were never a frontier problem at all — and it will own the answer to every one of them.