ASR evaluation

Transcription accuracy

• WER(Word Error Rate)
• SER(Sentence Error Rate)

Semantic accuracy

Concept Accuracy(based on domain concepts)

Reference:

• ASR evaluation

Tool:

• NIST Scoring Toolkit - sclite

SCTK has at its core, sclite (Score-Lite), which is a flexible Dynamic Programming alignment engine used to “align” errorful hypothesized texts, such as output from an ASR system, to the correct reference texts. After alignment, sclite generates a veriety of summary as well as detailed scoring reports.