skip to main content
10.1145/3649165.3690098acmconferencesArticle/Chapter ViewAbstractPublication PagessigcseConference Proceedingsconference-collections
research-article

Predicting Student Performance Using Sequence Models in XLogoOnline

Published: 05 December 2024 Publication History

Abstract

This study investigates the efficacy of sequence modelling in predicting student performance. Over eight months, user interactions were recorded as students solved navigation tasks in the XLogoOnline programming environment. On this data, three sequence models were trained to learn the temporal dependencies for several performance features. We compared the models' predictive capabilities and found the Transformer architecture to perform best in making multi-step predictions. Although prediction quality declines for multi-step forecasts due to the accumulation of error, we show that the quality of long-term forecast becomes closer to those of shortterm forecasts, as the input length increases. Our results provide valuable insights for the development of more effective teaching tools that can monitor student learning in real time.

References

[1]
2023. XLogoOnline. https://xlogo.inf.ethz.ch/release/latest.
[2]
Toufique Ahmed, Noah Rose Ledesma, and Premkumar T. Devanbu. 2023. SynShine: Improved Fixing of Syntax Errors. IEEE Trans. Software Eng. (2023).
[3]
Sahil Bhatia and Rishabh Singh. 2016. Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural Networks. CoRR abs/1603.06129 (2016).
[4]
Yuriy Brun and Michael D. Ernst. 2004. Finding Latent Code Errors via Machine Learning over Program Executions. In ICSE.
[5]
Hao Cen, Kenneth R. Koedinger, and Brian Junker. 2006. Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement. In ITS.
[6]
Albert T. Corbett and John R. Anderson. 1995. Knowledge Tracing: Modelling the Acquisition of Procedural Knowledge. User Model. User Adapt. Interact. (1995).
[7]
Wallace Feurzeig and Seymour Papert. 2011. Programming-languages as a conceptual framework for teaching mathematics. Interact. Learn. Environ. (2011).
[8]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Comput. (1997).
[9]
Ken Kahn and Niall Winters. 2021. Constructionism and AI: A history and possible futures. British Journal of Educational Technology (2021).
[10]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
[11]
Qi Liu, Shuanghong Shen, Zhenya Huang, Enhong Chen, and Yonghe Zheng. 2021. A Survey of Knowledge Tracing. CoRR abs/2105.15106 (2021).
[12]
John Neter, Michael H Kutner, Christopher J Nachtsheim, William Wasserman, et al. 1996. Applied Linear Statistical Models. (1996).
[13]
Philip I. Pavlik, Hao Cen, and Kenneth R. Koedinger. 2009. Performance Factors Analysis - A New Alternative to Knowledge Tracing. In AIED.
[14]
Roy D Pea. 1987. Logo Programming and Problem Solving. (1987).
[15]
Tung Phung, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adish Singla, and Gustavo Soares. 2023. Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models. In EDM.
[16]
Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J. Guibas, and Jascha Sohl-Dickstein. 2015. Deep Knowledge Tracing. In NIPS.
[17]
Eddie Antonio Santos, Joshua Charles Campbell, Dhvani Patel, Abram Hindle, and José Nelson Amaral. 2018. Syntax and sensibility: Using language models to detect and correct syntax errors. In SANER.
[18]
Jacqueline Staub. 2021. Logo Environments in the Focus of Time. Bulletin of EATCS (2021).
[19]
Jacqueline Staub, Zaheer Chothia, Larissa Schrempp, and Pascal Wacker. 2021. Encouraging Task Creation Among Programming Teachers in Primary Schools. In ISSEP.
[20]
Jacqueline Staub and Jeremy Marbach. 2024. Student Trajectories within the XLogoMini Environment.
[21]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NIPS.
[22]
Chao Wen, Ahana Ghosh, Jacqueline Staub, and Adish Singla. 2024. Task Synthesis for Elementary Visual Programming in XLogoOnline Environment. In AIED Track on Late-Breaking Results.
[23]
Chao Wen, Jacqueline Staub, and Adish Singla. 2024. Program Synthesis Benchmark for Visual Programming in XLogoOnline Environment. CoRR abs/2406.11334 (2024).

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGCSE Virtual 2024: Proceedings of the 2024 on ACM Virtual Global Computing Education Conference V. 1
December 2024
292 pages
ISBN:9798400705984
DOI:10.1145/3649165
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. knowledge tracing
  2. programming education
  3. sequence modeling
  4. turtle graphics
  5. xlogoonline

Qualifiers

  • Research-article

Conference

SIGCSE Virtual 2024
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE TS 2025
The 56th ACM Technical Symposium on Computer Science Education
February 26 - March 1, 2025
Pittsburgh , PA , USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 64
    Total Downloads
  • Downloads (Last 12 months)64
  • Downloads (Last 6 weeks)52
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media