Maybe the BLEU is wrong

[puzzledTao]

In the BLEU.py, function modified_precision(references, hypothesis, n):

def modified_precision(references, hypothesis, n):
# Extracts all ngrams in hypothesis.
counts = Counter(ngrams(hypothesis, n))
if not counts:
return Fraction(0)
# Extract a union of references' counts.
max_counts = reduce(or_, [Counter(ngrams(ref, n)) for ref in references])
# Assigns the intersection between hypothesis and references' counts.
clipped_counts = {ngram: min(count, max_counts[ngram]) for ngram, count in counts.items()}
return Fraction(sum(clipped_counts.values()), sum(counts.values()))

function Fraction() returns reduction of a fraction. Such as value = Fraction(3, 6), so value.numerator = 1,
value.denominator= 2. However, when we statistics the number of n-gram, we should add 3 not 1. Looking forward to your reply.

[cshanbo]

Hmm, thank you for pointing out this, I will check it. Normally we use BLEU from nltk. A PR will be welcomed.