What is Word Error Rate (WER) in Speech?

Word Error Rate (WER) is the standard metric utilized in computational linguistics and machine learning to quantify the accuracy of automatic speech recognition (ASR) systems.

Derived from the Levenshtein edit distance algorithm, WER calculates the minimum number of word-level editing operations required to transform an ASR model’s generated transcript into an authoritative human reference transcript.

The mathematical formula for WER

Word Error Rate is defined by the formula:

$$\text{WER} = \frac{S + D + I}{N} = \frac{S + D + I}{S + D + C}$$

Where:

• $S$ is the number of substitutions (words replaced with an incorrect word, e.g., “there” instead of “their”).
• $D$ is the number of deletions (words spoken in reference but omitted by the ASR model).
• $I$ is the number of insertions (hallucinated or extra words added by the model that were never spoken).
• $C$ is the number of correct words matched.
• $N$ is the total number of words in the ground-truth reference transcript ($N = S + D + C$).

Worked example calculation

Consider a spoken reference sentence compared against an ASR output:

• Reference: "We agreed to ship the local Markdown vault today." ($N = 9$)
• ASR Output: "We agreed to *deploy* [S] the local Markdown vault *on* [I] today."

Here:

• $S = 1$ (“deploy” substituted for “ship”)
• $D = 0$
• $I = 1$ (“on” inserted)
• Total errors = $1 + 0 + 1 = 2$
• $\text{WER} = \frac{2}{9} \approx 22.2%$

Nuances and limitations of WER in modern note-taking

While WER remains the scientific standard, modern developers recognize several practical nuances:

1. Text normalization: Differences in capitalization, punctuation, and numeric representation (“4” vs “four”, “October 9th” vs “10/9”) can artificially inflate WER even when the transcript conveys identical meaning.
2. Filler words: A model that automatically filters out conversational filler words (“um”, “uh”, “you know”) will be penalized under strict WER despite producing a more readable meeting summary.
3. Semantic error weight: Substituting “not” reverses an entire business decision, whereas omitting “the” is harmless. Standard WER treats both errors identically.

Benchmark accuracy in Oyma

In Oyma on macOS:

• We deploy Whisper models evaluated against rigorous conversational speech corpora.
• Text normalization cleans raw outputs into polished, punctuated prose.
• Transcripts achieve benchmark-leading accuracy locally on Apple Silicon, capturing technical jargon, participant names, and complex decisions reliably.