diff --git a/dataset.html b/dataset.html index 7a8fb01..b4723ae 100644 --- a/dataset.html +++ b/dataset.html @@ -112,22 +112,21 @@

Dataset
-

R2R-IE-CE v1Available upon acceptance

+

R2R-IE-CE v1To be released

This dataset is used to benchmark the task of Detection and Localization of Instruction Errors, presented in our paper. - You will find the dataset explanation here, as well as the Habitat Integration to use it. + You will find the dataset explanation here, as well as the Habitat integration tools to use it.

-
Val Unseen
-

Here we show how our json.gz dataset are formed:

+
Format of {split}_bertidx.json.gz

 {"episodes": [
 {
-  "episode_id": "//integer number",
+  "episode_id": "integer number representing an episode id",
   "trajectory_id": "integer number representing the gt trajectory",
   "scene_id": "string representing the scene",
   "start_position": "list of float representing the start position",
@@ -143,10 +142,10 @@ 
Val Unseen
"error_type": "integer representing the error type", "token_swapped": [ { + "old_word": "string representing the old word", "new_word": "string representing the new word", "token_id" : ["list of bert token id of this word"], "token_id_position": ["list of integer representing the index of the swapped words in the new instruction"], - "old_word": "string representing the old word", }] } "old_episode_id": "integer number representing the episode id from which this episode was generated" @@ -157,7 +156,7 @@
Val Unseen
]}
-
Habitat Integration
+
Habitat integration

In Progress

diff --git a/index.html b/index.html index 6a1c241..bd264b6 100644 --- a/index.html +++ b/index.html @@ -137,7 +137,7 @@

- Code (upon acceptance) + Code (to be released) @@ -145,7 +145,7 @@

- Dataset (upon acceptance) + Dataset (to be released) @@ -236,9 +236,9 @@

Scenario


Effect of Direction error

-

We reported the scenario depicted in the Fig.1 of the paper (also reported in the Scenario - Section above).

- The videos shows the full trajectories of the agent following the perturbed and correct instructions. +

We show the full episode for the example reported in the Scenario + section above (also Fig.1 in the paper).

+ The videos shows the agent following the perturbed and correct instructions respectively.
@@ -329,14 +329,13 @@

IEDL for detecting errors -

In our paper, we answered the following question: What if we apply IEDL on the R2R-CE +

We answer the following question: What if we apply IEDL on the R2R-CE dataset?

-

Our method, IEDL, detected 8 episodes from R2R-CE standard Val Unseen dataset that should be - deleted.

-

Here, we report the video showing the trajectories of these particular episodes.

+

Our method helped to detect 8 episodes from R2R-CE Val Unseen dataset that present different issues.

+

Here we show the BEVBert trajectories for these episodes, as well as the detected issue.


- Note: we report the instruction as given in the dataset. + Note: any typos in the instructions are reported verbatim from the dataset.
diff --git a/rxr_videos.html b/rxr_videos.html index 22bd99e..5b86f98 100644 --- a/rxr_videos.html +++ b/rxr_videos.html @@ -120,10 +120,11 @@

-
- -

We report also the episode to be removed from the validation split of RxR-CE.

- +
+

Our method helped to detect 10 episodes from RxR-CE Val Unseen dataset that present different issues.

+

Here we show the BEVBert trajectories for these episodes, as well as the detected error.

+
+ Note: any typos in the instructions are reported verbatim from the dataset.