Question about "ignore handling"

[hwijeen]

Hi, thanks for the great library!

I see from the paper that some categories are ignored from the metric calculation.

I'm a bit confused as the provided gold data seems to include ignore field, which is set False for every fields. (https://github.com/opendatalab/OmniDocBench/blob/4f530f0d4b664b0e7f3a6bab88daae5ab2c00403/demo_data/omnidocbench_demo/OmniDocBench_demo.json#L6482).
And there's this code that does the mentioned filtering.

Should I manually set ignore to True for some categories to reproduce the values reported in the paper? Or is it not necessary?

[ouyanglinke]

Thank you for following our work.
You don't need to set the ignore field in the dataset. The ignore field is currently not in use, so it is all set to false.
The process of ignoring headers, footers, and similar elements is conducted automatically during evaluation by default, which is exactly at this line of code you mentioned.