Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About pearson score #3

Open
joewellhe opened this issue Dec 23, 2017 · 1 comment
Open

About pearson score #3

joewellhe opened this issue Dec 23, 2017 · 1 comment

Comments

@joewellhe
Copy link

I read your paper "Better Summarization Evaluation with Word Embeddings for ROUGE".
I'm very interested in your work. I try Rouge-score in the data the same with your, but the pearson score
not good as your.
e.g. pearson score of rouge2 with Pyr is 0.59 (computed by the matlab script provided by TAC)
however, in your paper, this score is 0.96.
Why you can get such a high score. If you do the pre-process in TAC data, Could you tell me how you do pre-process.

@Lukecn1
Copy link

Lukecn1 commented Jul 16, 2020

I have the exat same issue, I am not able to reproduce the high correlation scores between ROUGE and the human evaluations reported in the paper.

I get very similar scores to the one provided by OP.

Did you do any preprocessing and if so, is it possible to see this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants