Multilingual Readability: 17 Formulas, 86 Languages
A site translated into nine languages is written well in one and badly in eight. Nobody notices, because the only readability number the team has was computed for English. Multilingual readability means measuring each language with the formulas that language actually has.
Code: ReadSightRS, MIT licensed.
The crate readsight carries 86 languages across 19 writing systems and 17 formulas, with all data compiled into the binary.
The universal five travel everywhere
Gunning Fog, SMOG, Coleman-Liau, ARI and LIX count letters and syllables, so they work on any script. A Thai page, an Arabic page and a Polish page all get a grade from the same five formulas.
That is enough to catch a translation that turned a clean sentence into a paragraph of legal prose. It is not enough to say how hard the text is for a native reader, because the threshold that means “easy” differs by language.
The other twelve belong to a language
Flesch Reading Ease and Flesch-Kincaid carry published coefficients for twelve languages: German uses Amstad, Russian uses Oborneva, Dutch uses Douma, Turkish uses Ateşman. German has its own Wiener Sachtextformel in four variants, Italian has Gulpease, Polish has FOG-PL, Arabic has OSMAN, Spanish has a family of four.
The engine hands each language its slice: nine formulas for English, eleven for Spanish, eight for German, five for Thai. A metric designed for English syllables never lands on a Thai sentence.
One score per language, not one score for the site
A content dashboard that averages a German and a Spanish grade produces a number with no meaning. The useful view is a table: language, page, grade, threshold for that audience.
Two decisions make that table possible. The formula set per language is explicit, so nothing silently changes when a page moves between languages. And the arithmetic is identical across runtimes, checked against the same reference values, so a Rust batch job and a PHP admin panel report the same grade.
Why the data lives inside the binary
The crate compiles 86 language files and 86 hyphenation pattern files into the binary with include_dir. No filesystem, no network, no cache directory to provision.
For a batch that scores fifty thousand pages on a build agent, that is the difference between a job that works and a job that fails on a missing data volume. Config::from_dirs() exists for the opposite case, when the data has to be editable without a rebuild.
What batch scoring is for
Three jobs earn their keep. A migration check that compares the old and new text of a page before a redesign ships. A monthly report over a catalogue, sorted by grade, so the worst hundred pages get edited first. And a gate in a translation pipeline that refuses a target text more than two grades harder than its source.
The last one is the least obvious and the most valuable: it catches the translation that is technically correct and unreadable, before a reader finds it.
What it does not know
The crate cannot judge structure, argument or tone, and its syllable count follows TeX hyphenation points, which is exact for line breaking and approximate for phonetics. Slavic Cyrillic languages use a heuristic counter instead, and the readme names that trade.
A readability grade is a filter over a catalogue. It finds the pages that need an editor, and it will never find the page that is easy to read and says nothing.
Related: content readability for the formulas themselves, and readability check for the single-page workflow that starts this work.