This research is about improving learning. More specifically, it looks at how learners can fill in knowledge gaps when they lose track. In face-to-face lectures, for example, they can ask the teacher directly for clarification, but may be reluctant to interrupt. With recorded video lectures, there is no such inhibition threshold, but the replay may not address the root causes of misunderstandings.
Recent advancements in technology provide an opportunity to address this challenge, notably:
-
The latest large language models demonstrate impressive abilities to manipulate language (rephrasing, simplification, elaboration).
-
Millions of educational videos are available on video sharing platforms, including their transcripts.
Combining the two can provide a simple and practical solution for improving comprehension in educational videos (see image): When watching an online lecture, the learner can signal the need for additional clarification, for example by pressing a button. The video will pause and, based on the transcript, a large language model will provide an explanation by rephrasing or elaborating on what was just said. When the ambiguity is resolved, the presentation continues.
B. Ruf and M. Detyniecki, "The «Huh?» Button: Improving Understanding in Educational Videos with Large Language Models", 18th IEEE International Workshop on Multimedia Technologies for E-Learning (MTEL), 2024.