Teaching S1-mini Swedish.

Babbla is my dictation app for the Mac: hold a key, talk, and the text lands in whatever app is in front, without the audio or the text leaving the machine. After the recognizer there is a cleanup pass — a small language model that deletes the ums, fixes the punctuation, and, if it is the right model, writes 'tjugotretusen fyrahundrafemtio kronor' as '23 450 kr'. For English that model has been Superwhisper's S1-mini since August: a 0.6B Qwen3 fine-tune that does one job, normalizing a speech transcript, in about a fifth of a second. For Swedish it was not, and the way it was not is the whole reason this post exists.

Measured on August 29, S1-mini left 28 of 33 spoken Swedish fillers in place and returned 9 of 18 Swedish dictations byte-identical. The dangerous half of that sentence is 'byte-identical'. Babbla has a retention guard between every model and the paste buffer — an output that drops too many of the input's words is rejected and the raw transcript is used instead, because a model that rewrites your sentence into its own is worse than no model. A no-op retains every input word. The guard waved it through, the user got no cleanup and no signal, and the only thing that stopped it in production was a language gate in the planner that sent Swedish to the big model instead: Qwen3 4B, five times slower, and — under the 'keep every other word, change nothing else' anchor that keeps it from hallucinating — structurally unable to write out a number, a date or a list.

A model that fails loudly is a bug. A model that fails by doing nothing is a feature nobody asked for, and it is the one kind the guards cannot see.

So the goal was not speed, although speed comes with it. It was capability: Swedish numbers, dates, clock times, amounts, addresses and lists, which no stage in the app could do at all. And the decision that shaped everything else was to continue training from S1-mini itself rather than start a Swedish model from scratch. S1-mini already knows the task shape — a fixed system prompt, a control line of three axes, transcript in, clean text out — and that knowledge is the most valuable thing available for free. The licence is Apache 2.0 plus a naming clause: derivatives must keep calling it 'S1-mini' by 'Superwhisper', which is why the attribution in Babbla's settings now reads 'S1-mini by Superwhisper, Swedish by Babbla'.

Measure the store before believing the plan

The plan said: build the release gate from 80 to 100 real Swedish dictations in my own history store, which holds 457 records. The first script I wrote copied the store — the database and its write-ahead log, opened read-only, never the original — and counted. Thirty-nine of the 457 are Swedish. I dictate to coding agents in English all day; the Swedish ones are messages to my family and to Winston about a ski trip. There were never 80 real Swedish dictations to hand-correct, and no amount of planning would have conjured them.

The survey paid for itself anyway, because it also said what Swedish speech looks like once the recognizers are done with it, and that is what the training inputs had to imitate. KB-Whisper punctuates and capitalizes 96 percent of the time. Fillers almost never survive it — 4 percent of Swedish records contain one, against 23 percent for the English engine. Dates come out as digits ('4 december') about a tenth of the time and as words otherwise. Questions are common, lists are rare, compound splits are rarer. Every one of those numbers became a probability in the noise generator, and the biggest of them is the one people forget: the most common real input is a clean transcript that must be left exactly alone.

The golden set became 38 real dictations — every Swedish one with three or more words — plus 86 I wrote by hand to the recognizer's habits, covering the tags the real ones cannot: amounts, times, lists, emails, self-corrections, English commands inside Swedish sentences. The eval reports the two sources separately, so the real-dictation number is never diluted by my own cases. Then baselines. Original S1-mini: 0 of 32 ITN cases, 0 of 16 lists, 0 of 17 emails. Qwen3 4B under its instruction frame: 1 of 32, 0, 0 — and 21 percent on fillers, 10 percent on punctuation, 44 percent on questions. Those three became the relative gate: the new model has to beat the model it replaces on the things that model could do.

Generate the data backwards

There is no audio anywhere in this project. The model is text to text, so the training pairs are generated backwards: start from a clean written sentence, render it the way someone would say it, and the pair (spoken, written) teaches inverse text normalization. The written side needs a convention, and the convention is Svenska skrivregler with one exception I chose on purpose: clock times take a colon. The rulebook writes 14.30, but '14.30' reads as a sentence boundary to every downstream tool that sees it, so Babbla writes 'kl. 14:30'. Small counts in prose stay words, 13 and up are digits, anything with a unit or a currency is digits at any size, thousands take a space, decimals a comma. 'halv tre' is 14:30 when the sentence says 'på eftermiddagen' and 2:30 when it does not.

The spoken side is a small library of Swedish inverse-ITN rules, with a test per rule: cardinals ('tjugotretusen fyrahundrafemtio'), ordinals for dates ('femtonde mars'), years both ways ('tjugohundratjugosex' and 'tvåtusentjugosex'), the whole colloquial clock ('kvart över två', 'fem i halv tre', 'tjugo i tre'), öre, procent, phone numbers digit by digit or grouped, and spelled addresses ('anna punkt svensson snabel-a example punkt se'). On top of that sits the noise: fillers where the recognizers actually leave them, false starts ('vi borde, vi borde'), self-corrections whose written form keeps only the value the speaker settled on, lists dictated flat with commas and 'och', emails spoken as one run with the greeting and sign-off inline, and a minority of bare, unpunctuated variants.

Four layers of source text feed it. Riksdagen's parliamentary protocols, which turn out to be the best Swedish corpus for this by a distance because they are transcribed speech — people saying numbers, dates and amounts out loud, already in written convention. Swedish Wikipedia, which needed a filter because the shard I pulled is four-fifths bot-written species stubs ('Eucossonus ingår i familjen vivlar'). About 2 100 spoken-style scripts across Babbla's app categories — chat, email, notes, instructions to a coding agent — written by Claude through the headless CLI on my Claude Code subscription, in resumable batches so a rate-limit pause loses nothing. And an English replay layer, which is the load-bearing one for not breaking what worked: English inputs from Simple English Wikipedia and my own store, with targets that are the original S1-mini's own greedy outputs. The original is the only authority on what the original does.

Every pair is emitted in mlx-lm's chat format with the system prompt and control line copied verbatim from the Swift source, so the tokenized training turn is byte-identical to what the app sends at run time:

system: You are a text normalizer for speech-to-text transcripts. The input begins with a
        control line specifying the styling, structure, and context settings; clean the
        transcript to match those settings and output only the cleaned text.
user:   [Styling: semi-formal] [Structure: lists] [Context: general]
        Eh, vi behöver köpa mjölk, smör, ost och, öh, bröd.
assistant:
        Vi behöver köpa:
        - Mjölk
        - Smör
        - Ost
        - Bröd

The final corpus is about 82 000 pairs. The number matters less than the shape: a quarter of the sentences are identity pairs, because the store said so.

Every generator fix came from a golden miss

The first real run was LoRA, rank 32 on every linear layer, 2 500 iterations, 45 minutes. It scored 73 percent exact on the golden set and 92 percent on the real dictations, and it already beat Qwen3 4B on fillers, punctuation and questions by a wide margin. ITN was at 69 percent. What I want to record is not the number but the method, because I did not touch a hyperparameter until the very end. Each run's misses were read one by one, and each one pointed at a hole in the generator, never at the model.

• Self-corrections resolved numbers but not words: 'blå, nej vänta, till grön' came back with both colours, because the generator had only ever corrected numbers and weekdays. Names, colours and small counts joined the sites.
• 'git checkout main, git pull, git status' came back as 'Git Pull, Git Status'. The list layer had taught it that comma-separated items get capitals. Lowercase command lines became identity pairs of their own.
• A 170-word real dictation fell into a repetition loop — the same sentence eleven times. Nothing in the corpus was longer than one sentence, and Babbla does not chunk below a thousand words. Two to five source sentences run together became a layer.
• 'klockan sju på kvällen' came back as 'kl. 7'. The 12-hour/24-hour decision needs the half of day stated in the same sentence, so frames that state it are drawn three times as often, with the hour range chosen to agree with it.
• 'Vad betyder felet cannot find module i loggen?' came back as two sentences with a capital C. English error messages quoted inside Swedish had no frame. Now they are identity pairs.
• The worst one was mine. A frame taught 'alltså' as a restatement — 'Tåget går 9:40, alltså tjugo i tio' — while the golden convention treats 'alltså' as a correction marker. Two of the five remaining ITN misses were that contradiction. You cannot train your way out of training data that disagrees with the test.

The generator is the model. The weights just cache it.

The full fine-tune that quantization erased

The plan said LoRA for iteration, full fine-tune for the release candidate — adding a language shifts more of the network than LoRA reaches comfortably. So run full1: every parameter, learning rate 1e-5, 6 000 iterations, two and a half hours. In bf16 it was excellent: 85.5 percent exact, lists at 94, self-corrections at 93, zero guard rejections. I quantized it to 4-bit like the original ships, ran the eval, and got 37 percent — within three points of the untrained original. Zero ITN. English agreement equal to the original's. The fine-tune had vanished.

It took a while to believe it, because the fused bf16 weights were demonstrably trained. The mean absolute change per weight against the base turned out to be about 3e-5. Affine 4-bit quantization with a group size of 64 has a step around 1e-3 on weights this size. The fine-tune had moved every weight a little, coherently, and the quantizer rounded every one of those moves back to where the original sat. LoRA's deltas had survived in run1 because a low-rank update concentrates its change in fewer, larger moves. A full fine-tune spreads it thin, and thin is exactly what a quantizer throws away.

Three ways out. Raise the learning rate so the deltas are bigger: full2 at 5e-5 survived 4-bit partly (73 percent) but cost English. Ship 8-bit: the step is sixteen times finer, and full1 at 8-bit scored 84.7, within a point of bf16. Or go back to LoRA for the release: its deltas survive, but a 6 000-iteration LoRA at batch 8 peaked at 24.6 GB and Metal killed it beside the other things this Mac runs. I shipped 8-bit. It costs 300 MB of download and about ten milliseconds per dictation next to a recognizer that takes two seconds, and it keeps the whole fine-tune.

The final model is full2 continued twice — 2 500 iterations at 2e-5 from the previous weights after each round of generator fixes, an hour each — and at 8-bit it passes every gate:

golden set, 124 cases, exact match after whitespace normalization, 8-bit, greedy

  all                111/124   89.5 %      guard rejections   0
  real dictations     36/38    94.7 %      mean pass          0.23 s
  itn                 30/32    93.8 %      (gate ≥ 85 %)
  list                14/16    87.5 %      (gate ≥ 80 %)
  self-correction     15/15   100.0 %
  email               16/17    94.1 %
  mixed-language      keep    100.0 %      (gate = 100 %)
  fillers / punct / question   84 / 67 / 85 %   (Qwen3 4B: 21 / 10 / 44 %)

Two gates I had to rewrite, and one I did not

The English regression gate was written as '95 percent exact agreement with the original on 830 held-out pairs'. Before trusting a number like that, measure the ceiling: I ran the original's own 4-bit build against the original's own bf16 greedy outputs. They agree 82 percent of the time. Quantization alone costs eighteen points of exact match on English, so no 4-bit or 8-bit artefact of anything could ever have met 95. The gate became 'within five points of that measured ceiling', and the shipped model lands at 78.

The retention guard rewrote a gate in the other direction. Two of my authored cases — 'Priset är fyrahundra, nej förresten, fyrahundrafemtio kronor' → 'Priset är 450 kr.' — were answered correctly by the model and rejected by the app, because the correct answer keeps two of seven content words and the floor is 30 percent. The guard was doing its job on a shape it had never seen. I lengthened the cases rather than lowering the floor, and wrote down that ITN-heavy Swedish output will eventually deserve a smarter floor than a word count.

The gate I did not rewrite is the one that runs through the real app. The Python eval scores the model alone; the shipping gate is a Swift test that reads the same JSON fixture and runs every case through the planner, the runner, the guard and the deterministic passes, and asserts the same thresholds. And because a text eval cannot tell me what happens after a microphone, there is a second Swift test that takes six WAVs synthesized with macOS's own Swedish voice — say -v Alva — through the KB-Whisper and Parakeet router and the cleanup chain, and checks what would be pasted:

decode sv-amount [kb-whisper/sv]: Fakturan hamnade på 23 450 kronor och ska betalas senast den 15 mars.paste  sv-amount: Fakturan hamnade på 23 450 kr och ska betalas senast den 15 mars.decode sv-time   [kb-whisper/sv]: Vi ses klockan halv tre på torsdag eftermiddag utanför stationen.paste  sv-time:   Vi ses kl. 14:30 på torsdag eftermiddag utanför stationen.decode sv-list   [kb-whisper/sv]: Äh, vi behöver köpa mjölk, smör och ost och bröd.paste  sv-list:   Vi behöver köpa:\n- Mjölk\n- Smör\n- Ost\n- Bröddecode sv-fillers [kb-whisper/sv]: Vi borde. Vi borde skicka rapporten på måndag.paste  sv-fillers: Vi borde skicka rapporten på måndag.

Notice the first decode line: KB-Whisper had already written '23 450 kronor' as digits. Part of what I thought was the model's job is the recognizer's, and the synthesized-speech test is what told me. It also told me something less flattering: the sentence I had chosen for the mixed-language case, 'kör git pull på main', reaches KB-Whisper from the Swedish voice as 'jeet pull på mig nån puss'. That is not a cleanup problem, and the fixture now uses a product name instead.

What changed in the app

Less than you would think, which was the point of continuing from S1-mini. The planner's language gate is gone: every dictation that is not a translation takes the fast stage — Swedish, English, and the short ones the language detector cannot place, which used to be the silent tax of the old gate, because a two-word dictation always went to the slow model. Qwen3 4B remains for translate mode only, so an install that never translates never needs the 2.3 GB download. The fast stage's download is pinned to a Hugging Face commit now, both stages are, because until this week a push to either repo would have changed what users ran with no release in between. And an upgraded install re-fetches the bilingual weights at launch, under a new repo directory, then deletes the English-only one.

That last piece is where the agents earned their keep. The whole project ran the way everything I ship runs now — Claude Code doing the typing against a design document, me supplying the judgment — and the reviews were the part I would not skip. The second review pass caught that my launch migration would start an unrequested 630 MB download for anyone who had switched to Apple Intelligence. The third caught that the revision marker I had just added applied to Qwen3 4B as well, whose pinned revision is also a commit, so every existing install would have read its 2.3 GB of Qwen weights as 'not downloaded' after upgrading. Neither showed up in a test. Both would have shipped.

The model is on Hugging Face as johrell/S1-mini-sv, weights, model card and Superwhisper's licence and nothing else — no dataset, no eval set, and no dictation of mine, because every real Swedish one I had is in the held-out set. The tooling is in Babbla's repository: the store survey, the golden-set builder, the ITN rules with their tests, the noise generator, the labeller, the corpus assembler, the training scripts and the eval. If you want to teach a text normalizer your language, that is the whole recipe, and the recipe is mostly measurement.