What does BLEU mean?
BLEU (Bilingual Evaluation Understudy) is an algorithm for evaluating the quality of text which has been machine-translated from one natural language to another. It provides a numerical score indicating how similar the machine output is to one or more high-quality human reference translations.
Core answer
Answer
BLEU calculates scores primarily based on modified n-gram precision: comparing overlapping sequences of words (n-grams) between the machine translation and the reference texts. It incorporates a brevity penalty to penalize overly short translations. Key considerations include that it operates best at the corpus level, not sentence level; requires high-quality reference translations; and correlates well with human judgment when averaging results over many test sentences. It primarily measures adequacy over fluency and does not assess semantic meaning directly.
BLEU is widely used as a standard metric in research and development for machine translation systems to quickly compare performance between versions or algorithms. It provides an objective, automated benchmark during system development, helping researchers identify improvements. However, its value is comparative and should always be complemented with human evaluation for a complete assessment of translation quality.
Related keywords