add dataset SciQA

KGQA · May 16, 2024 · ffffd9a · ffffd9a
1 parent 4c5bb04
commit ffffd9a
Show file tree

Hide file tree

Showing 4 changed files with 18 additions and 0 deletions.
diff --git a/frontend/static/entries.txt b/frontend/static/entries.txt
@@ -64,4 +64,5 @@
 /datasets/other/WC2014QA - 2 Hop
 /datasets/other/WC2014QA - Total
 /datasets/other/Spider4SPARQL
+/datasets/other/SciQA
 /systems/
diff --git a/other/$SciQA.md b/other/$SciQA.md
@@ -0,0 +1,2 @@
+## References
+<a name="myfootnote1">[1]</a> Auer, S., Barone, D.A.C., Bartz, C. et al. "The SciQA Scientific Question Answering Benchmark for Scholarly Knowledge". https://doi.org/10.1038/s41598-023-33607-z
diff --git a/other/SciQA.md b/other/SciQA.md
@@ -0,0 +1,8 @@
+---
+    name: SciQA
+    datasetUrl: https://zenodo.org/doi/10.5281/zenodo.5845197
+---
+
+|       Model / System       | Year | Precision | Recall | F1 | Language |         Reported by              |
+|:--------------------------:|:----:|:------:|:----:|:----:|:--------:|:---------------------------------------------------------------------:|
+|       JarvisQALcs        | 2023 |  0.138  |  0.138  |  0.138  |   EN     | [Auer et al.](https://doi.org/10.1038/s41598-023-33607-z) |
diff --git a/other/^SciQA.md b/other/^SciQA.md
@@ -0,0 +1,7 @@
+# SciQA
+
+**SciQA** [[1]](#myfootnote1)</sup> is scientific QA benchmark for scholarly knowledge. leveraging ORKG.  The dataset os created by manually developed a set of 100 complex questions that are answerable with ORKG. Based on those questions, eight question templates are adopted to automatically generated further 2465 questions. In total the dataset is composed of 2565 questions, along with the SPARQL queries, answers, etc. The dataset is available on [huggingface](:https://huggingface.co/datasets/orkg/SciQA) and [zenodo](https://zenodo.org/doi/10.5281/zenodo.5845197). 
+
+
+
+
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		## References
		<a name="myfootnote1">[1]</a> Auer, S., Barone, D.A.C., Bartz, C. et al. "The SciQA Scientific Question Answering Benchmark for Scholarly Knowledge". https://doi.org/10.1038/s41598-023-33607-z
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,7 @@
		# SciQA

		SciQA [[1]](#myfootnote1)</sup> is scientific QA benchmark for scholarly knowledge. leveraging ORKG. The dataset os created by manually developed a set of 100 complex questions that are answerable with ORKG. Based on those questions, eight question templates are adopted to automatically generated further 2465 questions. In total the dataset is composed of 2565 questions, along with the SPARQL queries, answers, etc. The dataset is available on [huggingface](:https://huggingface.co/datasets/orkg/SciQA) and [zenodo](https://zenodo.org/doi/10.5281/zenodo.5845197).