Skip to content

Commit 97d48ce

Browse files
committed
master: NMT model evaluation.
1 parent 0f8cddc commit 97d48ce

File tree

3 files changed

+30
-1
lines changed

3 files changed

+30
-1
lines changed
Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,30 @@
1-
# Evaluating NMT Models
1+
# Evaluation for Machine Translation
2+
3+
## BLEU Score
4+
5+
### What is BLEU Score?
6+
7+
1. The BLEU Score *(Bilingual Evaluation Understudy)* evaluates the quality of machine-translated text by comparing a candidate texts translation to one or more reference translations. <br><br>
8+
9+
2. The closer the BLEU score is to one, the better your model is. The closer to zero, the worse it is.
10+
11+
3.To get a BLEU score, the candidates and the references are usually based on an average of uni, bi, try or even four-gram precision.<br>
12+
<img src="./images/28. BLEU Score Calculation.png" width="40%"></img><br><br>
13+
14+
### Disadvantages of BLEU
15+
16+
1. BLEU doesn't consider the semantic meaning of words.
17+
2. BLUE doesn't consider the sentence structure of the sentence.
18+
19+
## ROGUE Score
20+
21+
### What is ROGUE Score?
22+
23+
1. ROGUE stands for *Recall Oriented Understudy for Gisting Evaluation*, which is a mouthful. But let's you know right off the bat that it's more recall-oriented by default. This means that it's placing more importance on how much of the human created reference appears in the machine translation.
24+
2. It was orignially developed to evaluate text-summarization models but works well for NMT as well.
25+
3. ROGUE score calculates the precision and recall between the generated text and human-created text.
26+
<img src="./images/29. ROUGE Score Calculation.png" width="40%"></img><br><br>
27+
28+
### Disadvantages of ROUGE
29+
30+
1. Low ROUGUE score doesn't necessarily mean that translation is bad.
408 KB
Loading
292 KB
Loading

0 commit comments

Comments
 (0)