What you can compute, don’t train

A bank transaction: 688,135 forints left the account, and the note says “this item is a 50% instalment of the gross amount of an invoice”. The open invoices include one for exactly 688,135 forints and one for 1,376,270 forints, from the same partner. Which one did we pay?

Anyone who reads the note to the end picks the larger one, because the amount paid is half of it. Our language model picked the smaller one, with 67% confidence. The item comes from our own test and is hard on purpose, but the behaviour is no exception: on items like this, which need a calculation, across three different tasks, the model found the right candidate in only 28–59% of cases.

Since mid-October we have spent four rounds of measurement looking for a remedy. We expected the answer to be some kind of training, and I admit there was an unspoken assumption in that: if the model decides badly, it needs to be taught better. In the end a few lines of code solved the item above. And we learned something about training that made the whole exercise worthwhile: a control test that ran in minutes showed that our best-looking result measured something other than what we thought.

What do we want from the machine?

DocAI processes company documents, and many of its tasks follow the same pattern. There is a case (a bank transaction, a fault ticket, an expense claim) and a few options (open invoices, possible fault causes, policy clauses), and the right one has to be chosen. For this we use a large language model: a program that has learned from huge amounts of text how a text continues, and that answers our question with a single letter, the label of the chosen option. The model under study is Qwen3.6-35B-A3B, running on our own hardware.

The choice alone is not enough. It also needs a number: how sure the model is. If that number is above a preset threshold, the system closes the case with no human check. If it is below, a person looks at it. Two measures follow from this, and the whole article is about them: what share of cases the machine closes correctly (coverage), and how many of its closures are wrong (the error rate). Raising the threshold means fewer errors, but more work left for people.

There is not always a right answer on the list. We distinguish four situations: a suitable option exists; none is suitable; a piece of data needed for the decision is missing; or the request does not belong to the task at all. The last three are the weak spot of these models, and we ran into the same problem around the “none of these” answer in earlier articles.

Two working methods are worth explaining, because every number rests on them. The test data is synthetic: the texts were written by a different language model (Muse-Glimmer-30B) from a detailed plan, so in every case we know exactly what the right answer is, from the plan rather than from a model’s opinion. The family of the model under study neither wrote nor labelled any data. And we pre-register: the question, the success criterion and the analysis code are locked and marked with a checksum before we look at any test result. That way we cannot pick a yardstick afterwards on which we look good.

A small checker next to the big model

The idea is simple. While the model answers, millions of numbers change inside it: these are its hidden states, a kind of working notes. If the model misjudges its own confidence, perhaps those numbers can reveal whether it is wrong right now. A separate, small model can be trained for this, which the field calls a head; here I will call it the checking head. We do not touch the large model or change its answer. Only the small checker learns.

The first round asked whether this works in general: one head, trained on many kinds of task, measured on an unseen one. It did not. On none of the three held-out task groups did it meet the pre-registered criterion, which fitted what we already knew about the general adapter.

The second round was smaller and more specific: a few hundred examples from the given task, with the head learning only whether an answer can be trusted. That is where the surprise came. We chose the threshold on a separate sample so that at least 95% of the closures would stay correct. With its own confidence the model could not close a single case on those terms, because no such threshold existed. With the head, after 500 training examples, it closed 66.6% of the cases correctly on average across three tasks, with a 2.9% error rate.

From zero to two thirds. That sounded too good to believe.

The first twist: most of the success was one rule

Luckily we had required, at pre-registration, that the gain be broken down. The breakdown showed this: for requests outside the task the model consistently answered “none of these”, and very confidently. On all three tasks it said so for all 100 such items. By our yardstick that is an error, because the right answer there is not that no invoice fits but that this is not an invoice-matching question. Those confident errors landed at the top of the ranking, which is why no usable threshold existed.

If you sort the decisions by confidence, ideally the errors end up at the end of the queue. Measured by the improvement in that ordering, 72% of the gain came from a single rule with no training at all: move the “none of these” answers to the back. In the number of cases closed the rule was worth less, but the direction was clear. Much of what the head had learned was whether deciding was justified in the first place.

So in the third round we gave the model words. The prompt, the instruction text given to the model, now said it could answer in four ways: choose a candidate, or say that none is suitable, that a piece of data is missing, or that the request is out of scope. On a fresh test, with no training, the model closed 36.9% on its own. The head added to that, up to 64.8%, but did not beat the head used with the old prompt.

So the head adds real value, but a well-written prompt delivers a good part of it, and that is the thing to write first. It did not work everywhere, of course: on the bank task the model never once flagged missing data, and on its own it closed nothing there.

The mistake the model is surest about

Up to this point nearly all errors came from the model not recognising that no right answer exists. The real test is when a right answer exists and it still picks the wrong one. For that we built items like the one in the introduction: the right answer needs a calculation step (partial payment, unit conversion, net versus gross, currency), and among the candidates sits a decoy that matches on the surface.

We ran the same head, with the same threshold, on them:

The checker did not recognise this kind of error. The model’s own confidence behaved even worse: it was higher on the decoy than when the model decided correctly. On the bank task the head favoured the decoy even more than the model did. The most likely explanation is that in the training data the right invoice’s amount always matched the transaction, so it learned that an exact amount is a good sign. With partial payments that is exactly what misleads it.

Then came a simple test: the same items, with one extra line at the end of the text giving the result of the missing step, calculated by code. The share of correct answers rose from 29% to 100% on the bank task, from 52% to 94% on troubleshooting, and from 24% to 67% on expense claims. This was an exploratory test and an upper bound on the effect, but it showed the direction. This was not a failure of understanding; it was a skipped calculation.

A few lines of code before the model

In the fourth round this became a pre-registered procedure that we call the chain. The idea is that the model should not calculate, only quote and decide, with code doing everything in between.

The model first receives only the text of the item, not the candidates. It collects the data from it and quotes the source verbatim for each value. This is extraction. The code checks whether the quote really is in the text and whether the value can be read from it; whatever fails is discarded. Then the code calculates: full amount from an instalment, unit conversion, net from gross, currency. Finally it compares every candidate with the result and rules out only what provably contradicts it. If exactly one candidate’s invoice number appears in the payment note and everything else matches, the code binds the invoice itself: this is the chain’s only independent decision. Everything else goes to the model, together with the calculated values.

The most important property of the chain is that it states only what it can prove. It draws no conclusion from missing data, and if something does not add up it discards the information instead of making something else up.

In a pre-registered measurement the share of correct answers on the trap items rose from 28–59% to 97.5–98.9%. The code never removed the right answer and never bound a wrong invoice. The model’s extraction barely hurt: with flawless data the result would have been only 0.5–1.1 points better.

One condition, however, was not met. We speak of a false pass when an extraction error makes the chain pass on something different from what it would with flawless data, and the item still does not go to a person. Our target was at most 2%, and on troubleshooting it came to 3.3%. We went through the cases one by one. During extraction the model sometimes converted a value of its own accord (writing 1200 for 1.2 kOhm, when 1200 is not in the text), and at other times it quoted without accents, which the strict comparison counted as a mismatch. After two small fixes we repeated the measurement, separately pre-registered and on fresh items: the share of correct answers was 98.6–100%, and false passes were 0, 0 and 0.5%.

The chain is no miracle cure. On a test with a natural mix, where trap items are rare, it did not increase coverage by itself. On the bank task it bound 363 items out of 1000 without error, and on the other two there is nothing for it to bind. Its value lies in the hard items and in safety.

The good result that failed the control test

That leaves the error that cannot be calculated: when the text does not settle the question at all. This was the head’s original promise, because here code cannot help. We built a pre-registered test for it: every case has two versions, one with the deciding sentence and one without, where the right answer is that a piece of data is missing. Can the head tell them apart?

The measure is AUROC: if you pick a decidable and an undecidable item at random, how likely is the decidable one to get the better score? 0.5 is a coin flip, 1.0 is perfect separation.

The result was clean. The model’s own confidence scored 0.738, the head 0.924, a difference of +0.186 with the uncertainty band well above zero. Under the protocol that counts as “improves materially”. On another pre-registered question, recognising a wrong choice, there was no detectable advantage, but on this one the answer seemed to be in.

Then we ran a control we should have run earlier. We trained a simple program on the same task, one that counts the words and letters of the text and understands nothing of it (in the field, a TF-IDF classifier). The answer came within minutes: 1.000. The word counter separated the two perfectly, better than the head.

The flaw was in the test. The undecidable version was simply the text without the deciding sentence, so it was shorter and more clipped than the other. You do not need to understand whether the question can be decided; it is enough to notice whether there is one more sentence. The pre-registered number stayed correct. It just measured something other than what we thought.

Two more separately pre-registered controls followed, with the heads left untrained. In the first, the undecidable version also got a non-deciding sentence, so length was fine, but the topic still gave it away: the word counter reached 0.998 and the head’s edge was 0.144. The protocol had said in advance that this would not count as evidence. In the second we wrote minimal pairs. Both texts have the same words and the same structure, and both contain a sentence that looks deciding; in one it is about the amount claimed, in the other about an earlier, different claim. The word counter fell to 0.699 and the head’s edge came to +0.075.

Under the pre-registered reading the claim about the head stands, in a narrowed form: on this task type, with this construction, with these heads. The edge is real, and it shrank to roughly a third to a half. In practice, though, it is not usable: above the threshold the head would close 77.7% of the undecidable items, the model’s own confidence 37.9%. It ranks better but lets more wrong closures through. We claim no product benefit from this.

What is left on real bank transactions?

So far every number has been synthetic. We also tried the chain on 262 real bank transactions from a contributing company, from June to September 2026, for evaluation only: the items never left our own machines and only aggregate numbers appear here. I labelled all 262 blind, seeing exactly what the model saw. These are development numbers: the sample and the labels are known and the threshold was chosen on them, so they are optimistic.

The first lesson was embarrassing. Extraction, which barely erred on synthetic data, proved weak here. On 93 items a candidate’s invoice number appeared verbatim in the payment note; the model brought out 30 of those, and the chain could bind 22 items. Real payment notes are short: 91 of the 93 were under 40 characters. The fix is worthy of this article’s title: we simply searched the note for the candidates’ invoice numbers, with no model involved. That gave 70 bindings with no error (the upper bound for 0 / 70 is about 5%).

The second lesson: here the chain holds the 2% target, not the model. Of the model’s own closures 2 out of 56 were wrong, which is 3.6%, and both were items I could not decide either from the data shown. Chain and model together made 126 closures with 2 errors, 1.6%. On real items the head added nothing over the model’s own confidence (0.910 and 0.911 respectively at separating right from wrong choices). And one honest figure: at this company the existing matcher already solves nearly everything, so compared with it only 1–3 new closures appeared. Here the benefit is safety, not volume.

What this measurement does not prove

All the main numbers come from synthetic data, from a single generator model. The trap items are hard on purpose and measure what the system does with a known type of mistake, not how common that mistake is in real life. So about the chain working we claim only the synthetic result. The real data is one company, four months, 262 known items and a single labeller: a direction, not proof.

The repair round was built for known extraction errors. The fresh sample measures that, but two kinds of error remain (80 mOhm instead of 800, and “40%” read as an amount), and the samples are small: in the first measurement one item moves the figure by 0.5–1.1 points. The labels are not flawless either: in the random blind audit we found 2 errors out of 30 on the bank task and 3 out of 30 on troubleshooting. The faulty units were removed, but only the audited items were checked.

The claim about the head is narrow and small. One task type, one construction, one base model. Whether a different training sample would give more we did not measure, so we do not claim it.

We will check the real-data numbers on a fresh sample of about 150 items with a pre-registered threshold. The research continues: our goal remains a decision model that knows, even on an unseen task, when it cannot decide. Until then, what can be computed we do not leave to the model.

---

All four rounds ran under pre-registered protocols (internal numbering 05a: 29 October 2026; 06: 31 October; 07: 1 November; 08: 4 November, the repair round 5 November, the two controls 6 and 8 November); the text-only control and the test with the computed line are post-hoc, exploratory analyses. Model: Qwen3.6-35B-A3B, extraction in FP8 under vLLM 0.30 at temperature 0, hidden states and scores in BF16, on NVIDIA GB10. Test: 1000 items per family in natural composition, 381 trap items, 627 in the repair round, 348 minimal-pair units, in Hungarian and English. Reported intervals are template-group cluster bootstrap with 10,000 replicates, at 95%. The protocol, the code and the aggregate results are in the docai-evals repository.