Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different results on demo and trained model checkpoint #11

Open
MM-Vianai opened this issue Jul 30, 2020 · 0 comments
Open

Different results on demo and trained model checkpoint #11

MM-Vianai opened this issue Jul 30, 2020 · 0 comments

Comments

@MM-Vianai
Copy link

MM-Vianai commented Jul 30, 2020

I have followed the instructions to run the checkpoint model, and been able to successfully run it. However, I get different "answer"s on the demo and the trained model checkpoint.

Demo:

  • Passage: The family of a cancer victim that had won a $4.4 million judgment against Laboratory Corp. of America over a botched Pap smear test settled with the company Monday just before the parties were set to start a new trial on noneconomic damages in Florida federal court.
  • Question: Who was the injured?
  • Answer: cancer victim

Trained Model Checkpoint:

  • input.jsonl :
    {"passage": "The family of a cancer victim that had won a $4.4 million judgment against Laboratory Corp. of America over a botched Pap smear test settled with the company Monday just before the parties were set to start a new trial on noneconomic damages in Florida federal court.", "question":"Who was the injured?"}
  • output.jsonl:
    {"question": "Who was the injured?", "passage": "The family of a cancer victim that had won a $4.4 million judgment against Laboratory Corp. of America over a botched Pap smear test settled with the company Monday just before the parties were set to start a new trial on noneconomic damages in Florida federal court.", "predicted_ans": "against Laboratory Corp.", "answer": "against Laboratory Corp.", "inputs": [{"name": "question", "tokens": ["Who", "was", "the", "injured", "?"]}, {"name": "passage", "tokens": ["The", "family", "of", "a", "cancer", "victim", "that", "had", "won", "a", "$", "4.4", "million", "judgment", "against", "Laboratory", "Corp.", "of", "America", "over", "a", "botched", "Pap", "smear", "test", "settled", "with", "the", "company", "Monday", "just", "before", "the", "parties", "were", "set", "to", "start", "a", "new", "trial", "on", "noneconomic", "damages", "in", "Florida", "federal", "court", "."]}, {"name": "numbers", "tokens": ["4.4", "0", "100.0"]}, {"name": "dates", "tokens": ["-1/-1/-1"]}, {"name": "composed_numbers", "tokens": ["0.0", "8.8", "95.6", "104.4", "200.0"]}, {"name": "year_diffs", "tokens": ["0"]}, {"name": "count", "tokens": ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]}], "program_nested_expression": [{"name": "span", "identifier": 4}, [{"name": "relocate", "identifier": 3}, [{"name": "filter", "identifier": 2}, {"name": "find", "identifier": 1}]]], "program_lisp": "(span (relocate (filter find)))", "program_execution": []}

==> cancer victim vs. against Laboratory Corp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant