Why the OCR data is not consistent with the paper?

[Road2Redemption]

In the paper you claimed that there are 50K web_cc OCR data for each language, so that there should be 500K data in total, but the released version of PangeaInstruct have only 300K data in total, is there a size mismatch? Or there are data that you keep as private?
Thank you!

[yueqis]

Hi, thanks for pointing this out! We only used 300k OCR data in training so the previously released data contain only 300k OCR data. We now uploaded the 500k data to huggingface, please check: https://huggingface.co/datasets/neulab/PangeaInstruct/blob/main/ocr/webui_multilingual_ocr/data-500k.json

[Road2Redemption]

Thank you very much!