Chinese words in the Hungarian answer: how much may you take from a model to make it stop?

There is a class of failure that is hard to take seriously until it happens in your own system: the model is working in Hungarian on a Hungarian document, and in the middle of the process it writes Chinese words. A token or two, typically a conjunction. Then it finds its way back to Hungarian as if nothing had happened.

For us it showed up in the nightly agent loop, on Hungarian invoices and contracts, sampling at temperature 0.6. There is a known recipe for it: scale down the rows of the Chinese tokens in the model's output matrix. We tried it, and it worked. What was missing was the number: how much may you take away, what does it cost, and is the intervention well shaped. A pre-registered measurement round answered that: twelve model variants, six instruments, three days. This is the short version; the long one, with methodology and limitations, is in the Research section (in Hungarian).

Two terms before we go on. CJK is the shared Unicode range of Chinese, Japanese and Korean writing, and Han is the name of the Chinese characters within it. Han risk is the number of Han tokens to expect per million generated positions. It does not count the Chinese words actually written, because there are too few of those to measure; at every position it adds up the probability the model gave to Han tokens after sampling. We write it as /M. Zero means that at no position was a Chinese token among the candidates the sampler could pick.

In short:

• The dose–response is not a curve but two cliffs. Chinese capability is intact down to a factor of 0.7 (36/36), halves at 0.6 and is practically gone at 0.5. The Han risk is exactly zero from 0.8 down and still leaks at 0.9. One dose sits in the window: 0.7.
• The Hungarian cost is zero, and not by luck. The intervention only touches Chinese rows, which never win on Hungarian text, so the greedy Hungarian output of the patched variants is byte-identical to the original on 150 tasks. The whole cost is in Chinese.
• The 0.7 variant generated not a single Han token over ~1.09 million positions, across several servers and seeds. The original model generated thousands on the same runs.

Where we intervened, and why there

The last layer of a language model, the lm_head, is surprisingly simple: a dictionary with exactly one token per row. The product of a row and the model's internal state is the token's logit, its raw score, which then becomes a probability. Qwen3.6 has a vocabulary of 248 thousand entries, and 55 424 of those rows contain a Han character. Multiply these rows by a number below 1 and the logits of the Chinese tokens shrink, while the model's internals are not touched at all. It is not a prompt the model can forget. It is not training, which touches everything. It is not a decoding blocklist that has to be present on every call. The property lives in the model file itself: one of the checkpoint's 55 files changes.

Two forms competed. One is scaling, the smoothie-qwen recipe. The other is a row replacement known from Korean work, which points the target rows against the mean internal state on Hungarian text. And two masks: the raw one, containing every token with a Han character, and the Korean-style refined one, which keeps the frequent single-character ideographs.

The question nobody had measured: how much?

This picture is the round in one frame. On top, the cost: a Chinese probe of 36 short tasks on which the original model is flawless. Below, the benefit: the Han risk at three temperatures, on Hungarian agent runs.

The two axes break in two different places. Chinese collapses between 0.7 and 0.6, and it is a cliff, not a slope: at 0.5, 97% of the answers contain no Chinese character at all, the model cannot speak. The Han risk disappears between 0.9 and 0.8. The window exists, and exactly one measured dose sits in it: 0.7. It sits right at the edge of the cost cliff, with no margin downward.

The Korean row replacement did not turn out cheaper: it kills Chinese already at half strength. The refined mask fails in both directions: the ideographs it keeps do not hold Chinese at a usable level, but they do let the Han risk in. What has to be kept for Korean can be dropped for Hungarian.

The Hungarian cost: zero, and we know why

The protocol's primary cost instrument was a Hungarian trap corpus of 150 tasks. After the first three variants it turned out that this instrument is blind to this intervention by construction. The patch only changes the logits of Chinese rows, and on Hungarian text those never win: the greedy output is bit-identical. Under sampling it can differ only where a Han token is among the twenty top-k candidates, that is, exactly at the positions that are the risk itself.

The non-target-language cost is bounded above by the non-target-language risk. If the patch changed a Hungarian position, the original model could have written Chinese there.

Eight variants confirmed this byte for byte, including the strongest 0.1 factor. Three variants landed in a server state for which we have no run of the original model; there it is a derivation, not a measurement. So the whole cost lives in the language being suppressed, and the trap corpus became a negative control: it proves the measurement sees no phantom effect.

Re-measured on the distribution the model actually samples from

The Han risk in the grid above was computed from the model's raw candidate list. A reader pointed out that this is not enough: vLLM applies the presence penalty before the top-k cut, pushing down tokens already seen, so Han candidates further back can enter the actual twenty. We measured again with the server returning the probabilities after the full sampling chain.

The raw method had underestimated the original model's Han risk by about an order of magnitude: 73/M instead of 5.5–12/M, from three cases, with wide error bars. On the same actual distribution the 0.7 variant is exactly zero on the production settings. On the retry path, where the system retries a suspected loop at temperature 0.9 with a stronger presence penalty, it is 0.45/M: not zero, with no event observed, and that figure is a lower bound.

After closing the study we recounted every run, including those that never reached a final answer. Across three fresh servers, 15 seeds, concurrent load and 150 synthetic documents, ~1.09 million positions in total, the 0.7 variant generated not a single Han token. The original model generated 65 on the same seeds, plus another 6 593 in a single runaway generation on the retry path.

What this measurement does not prove

The patch removes the Chinese word, not the loop. On the very seed where the original model ran away, the 0.7 variant ran away too: without Chinese, but with 65 thousand tokens of word salad. The loop guard stays.

We measured one model, one quantisation, one engine. The mask depends on the vocabulary; nothing transfers to another model without re-measuring. On the cost side there was only a Chinese probe; Japanese kanji and Korean hanja fall in the same range, so we expect the same loss, but we did not measure it and do not claim it. The Han risk figures come from three agent cases. No dose between 0.6 and 0.7 was measured.

And the patch only works with a truncating sampler. 96% of the targeted rows have a negative logit on Hungarian text, and scaling pulls those towards zero, so the raw Chinese probability mass rises with the patch. The zero comes from the top-k cut. If you sample without top-k and top-p, this model does not help you.

Along the way we also learned about the instrument: batch size and a fresh server start both flip the greedy output on a few tasks, by as much as the effect we wanted to measure. That is a story of its own, in chapter 7 of the study.

What we are doing now

The 0.7 variant goes to a low-traffic, reversible production trial, tracking Han occurrences and task quality. The decision rule was written before the data, and all three of its conditions held. The model is available on Hugging Face as Qwen3.6-35B-A3B-FP8-cjk-damped-S07, under Apache-2.0, with the full dose–response table and a warning: this is not a better model, it is one from which we deliberately cut a capability. If you need Chinese, Japanese or Korean output, it is damage.

What to take away if you do not run Qwen

1. If a token class is your problem, the lm_head is the cheapest place. But measure the dose, do not guess it: our window was one step wide.
2. The non-target-language cost is bounded above by the non-target-language risk. Do not look for the cost in your own language; it is in what you suppress.
3. The raw candidate list is not the actual distribution. If you compute risk from logprobs, ask for the processed ones (--logprobs-mode processed_logprobs).
4. Count what never reached an answer, too. Our largest flood of Chinese was in a run the calculator had skipped.

Measurements were made on a DGX Spark (GB10) with Qwen3.6-35B-A3B-FP8 (Apache-2.0) under vLLM. The extraction corpora and the agent traces contain customer data and are not public; every number in this article is an aggregate. The synthetic trap corpus, the Chinese probe, the code and every variant's output on the public instruments are in the docai-evals repository. The intervention builds on the dnotitia/smoothie-qwen recipe; the closest prior work is ThakiCloud/Qwen3.8-27B-ko-cjk-suppressed.