DLOT: An Open-Source Application to Assist Human Observers

Abstract: Adaptive intelligent educational systems are gaining popularity, offering personalized learning experiences to students based on their individual needs and styles. One crucial feature of such systems is real-time personalized feedback. However, identifying real-time learning processes impacting student performance remains challenging due to data volume constraints. Current research often relies on labor-intensive human observation, which is time-consuming and not scalable. To efficiently collect real-time data, an observation tool is essential. Qualitative/Mixed Method research explores participant experiences in education, social science, and healthcare, utilizing methods like focus groups and observations. However, these methods can be labor-intensive, particularly in maintaining observation time intervals. Existing tools lack comprehensive support for education-focused focus groups and observations. To address these issues, this paper introduces the Data Logging and Organizational Tool (DLOT), a flexible tool designed for qualitative studies with human observers. DLOT offers customizable time intervals, cross-platform compatibility, and data saving and sharing options. The tool empowers observers to log timestamped data and is available on GitHub. The DLOT was validated through two studies. The first study predicted students' affective states using real-time annotations collected via DLOT, observing 30 students in each class. The second study created multimodal datasets in a computer-enabled learning environment, observing 38 students individually. A successful usability test was conducted, offering a potential solution to challenges in real-time learning process identification and labor-intensive qualitative research observation.