Plainsay is signed and notarised. Then, on first run, it downloads a few hundred megabytes of Core ML from the internet and executes it. Until v0.2.27, nothing verified that file — and neither of the libraries we depend on verifies it either. This is what we found when we went to look, and what we did about it.
An external reviewer raised it as the last item on a list of otherwise small findings, and rated it honestly: high precondition, high impact. An attacker needs to be in a position to serve modified model files. That is not a low bar.
But it is the one place unsigned code enters an otherwise tight chain. Everything else that runs on your Mac is signed by us and notarised by Apple. The component that actually does the transcribing arrives over the network and is compiled and executed. For an app whose entire pitch is that your audio never leaves the machine, that is the wrong gap to leave open.
We read both rather than assuming. The answers were not what we expected:
If-Range, to resume an interrupted transfer.Both also resolve from a moving branch, and neither exposes a revision parameter through the API we call. So there was nothing to reuse, and pinning a commit was not reachable without forking. That left verifying it ourselves.
The tempting approach is to ask the server for the file's hash and check the download against it. That proves the transfer was not corrupted — which TLS already tells you. It proves nothing about whether the file is the one you meant to run, because the hash and the file come from the same place over the same connection.
So the expected digests for every model Plainsay can download are compiled into the app’s own binary, and are recomputed locally after download and before the model is loaded. Someone who controls what the model host serves cannot also change what we expect it to be, because changing that means changing a notarised binary.
The sentence above used to read differently. The digests were shipped as a file inside the app bundle — except they never were. Our packaging script copies resources from an explicit list, and nobody added the new one, so the file the app looked for was in no release we ever published.
It should have failed loudly on the first build. It did not, because of how the lookup works: it also checks the build directory recorded when the app was compiled. On the machine we build on, that directory exists. The app found the file, verified models, wrote healthy logs, and behaved perfectly — on exactly one Mac in the world.
Everywhere else it crashed on startup, every time, from v0.2.28 to v0.2.30. Three releases. We did not find it. A user did, twice, with a stack trace precise enough to fix from — and we are naming that because it is the only reason this section exists.
The fix was not to repair the lookup but to delete it. The digests are now generated as source and compiled in: no file to find, no packaging step to forget. That is also a better answer to the original problem, since the expected hashes now sit inside the notarised executable rather than beside it. Fixed in v0.2.31; the three affected releases are marked on the releases page.
We are telling you this on the page that exists to argue we take supply-chain integrity seriously. A verification feature that stops the app from starting is not integrity, it is theatre with a crash in it. The lesson we actually took is the one written into the code now: a safeguard must never be able to take the app down with it.
We want to be precise about what this buys, because overstating it would be worse than not having it.
The pins record what the model repositories served at the moment they were taken — exactly as a Package.resolved records what a dependency resolved to. They cannot tell you the upstream model was honest that day. What they do is make any later change visible, which is the realistic attack: a repository, or an account with push access to one, quietly starting to serve something different than it did when a human last looked.
The manifest is generated from Hugging Face's tree API without downloading the models — a few hundred kilobytes of JSON rather than several gigabytes of weights. Large files carry a real SHA-256; smaller ones carry git's blob SHA-1. Both are recomputable on your Mac, and each pin records which applies.
Version 0.2.27 verified the directory FluidAudio returned, but the model files were actually in a sibling directory. It also treated every pinned repository path that FluidAudio had never downloaded as tampering. The result was the exact failure the tests were meant to prevent: Parakeet was rejected, deleted, downloaded again, and rejected again on the next launch.
Version 0.2.28 resolves the directory that actually contains the model and matches the check to FluidAudio's real subset. Its current int8 download contains 23 of the 88 pinned repository files; those 23 are verified, the other 65 are reported as absent, and any downloaded file whose contents differ is still refused. Version 0.2.38 trims the Parakeet pins to exactly those 23 files, so nothing is reported absent on a normal install. This page originally described the stricter rule as deliberate. It was wrong, and the correction belongs in the same account as the fix.
Synthetic tests prove the digest functions but not the pins. Wrong pins would pass every one of them while every real user hit a hard failure on first run — looking for all the world like an app that simply cannot download its own model. That is the worst place to be wrong, so it was checked end to end:
changed: AudioEncoder.mlmodelc/weights/weight.bin.git hash-object itself rather than a constant — that framing is easy to get subtly wrong, and it fails in the direction that looks like stale pins rather than a broken implementation.The pin is only as good as the moment it was taken. If a downloaded pinned file is republished with different contents, the check fails closed and a human has to look at what changed and re-pin deliberately. A moved or no-longer-downloaded path is reported but is not itself treated as tampering. We did not wire the upstream check into CI: it would then fail on every unrelated change, and the correct response to a changed pin is someone reading the diff — which is exactly what a permanently red build trains people to stop doing.
We would rather say that plainly than imply the problem is now permanently solved.
All of the above is in the open: the verification code, the pinned manifest, the generator, and the tests including the single-bit tamper case. Plainsay's Mac client is MIT licensed and public. If this page was worth reading, the diff is worth reading.
github.com/conrader/plainsay — inspect the MIT-licensed Mac client