Characterization and Effects of CS2 Learning with GenAI, Visualization, and Human Support
Generative AI (GenAI) is becoming a widely adopted learning support tool for both students and instructors, as it offers benefits such as personalized tutoring and scaffolded learning. However, recent research highlights potential drawbacks such as over-reliance and metacognitive issues, especially in novice programmers. Most prior work focuses on introductory programming courses, and important questions remain about the underlying mechanisms behind the negative effects of GenAI and if findings can be generalized when students learn more advanced computer science concepts. To address this gap, we conducted a mixed-methods study comparing student interactions with GenAI to two traditional learning supports in a second-year algorithms course: namely algorithm visualization (AV) and human live tutoring (LT). Twelve students participated in three 90-minute study sessions focusing on sorting, tree, and graph algorithms. We recorded gaze and interaction data, and each session concluded with a test assessing their conceptual understanding of the topic. Our analysis classifies when during the problem-solving process participants sought help and compares the interaction patterns across the three learning supports. Although GenAI produced a larger increase in self-efficacy compared to live tutoring, it was associated with noticeably lower results in learning outcomes. We found that participants did not use algorithm visualizations effectively, faced usage barriers when using GenAI to learn advanced topics, and live tutoring yielded the highest learning outcomes results.
Fri 14 AugDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:45 - 14:35 | GenAI beyond CS1Research Papers at Main conference room Chair(s): James Prather Abilene Christian University | ||
13:45 25mTalk | Characterization and Effects of CS2 Learning with GenAI, Visualization, and Human Support Research Papers Quinton Yong University of Victoria, Miguel A. Nacenta University of Victoria, Anthony Estey University of British Columbia | ||
14:10 25mTalk | Evaluating LLM-Generated Contextualized Algorithm Design Problems Research Papers Erica Goodwin University of Chicago, Katherine Braught University of Illinois at Urbana-Champaign, Jonathan Liu University of Chicago, Dip Kiran Pradhan Newar pc, Yael Gertner University of Illinois Urbana-Champaign, Seth Poulsen Utah State University, Diana Franklin University of Chicago | ||