Word error rate
Also known as: WER, CER
Word error rate measures how often generated speech gets the words wrong, by transcribing the audio and comparing it to the script. A character-level version (CER) is stricter. It catches dropped, added, or mangled words that a naturalness score would miss.
Why it matters
A voice can sound beautiful and still say the wrong thing, and word error rate is the metric that catches it. It matters most exactly where narration is easiest to get wrong: numbers, names, acronyms, and loanwords, where a single dropped syllable changes the meaning.
It is also objective in a way listener scores are not. You transcribe the output and compare it to the script, so there is a hard number rather than an impression.
In practice
Run speech to text on the generated audio and compare it against your script. Watch the short function words and the proper nouns, which is where errors cluster, and treat a rising error rate after any processing step as a warning that the step is damaging speech.