Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks
Learning to communicate with code-generating AI models is an emerging skill for novice programmers. One recent pedagogical approach, Prompt Problems, allows students to practice solving computational tasks through natural language prompting. Little is known about the mistakes students make when interacting with AI or what strategies they use to recover. In a CS1 course, we studied attempts by more than 900 students to solve dialogue-based Prompt Problems. We analyzed student reflections, unsuccessful prompts, and reported debugging strategies. Compared to traditional coding tasks, students generally found prompting easier, more enjoyable, and better targeted at developing problem-solving skills. The most common mistakes related to omission of key details, suggesting both a failure to acknowledge their importance and over-reliance on AI to infer them. When prompts failed, students focused more on clarifying their intent and reflecting on the provided problem details, than on tracing generated code or examining test cases.
Thu 13 AugDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
09:10 - 10:25 | GenAI InteractionsResearch Papers at Main conference room Chair(s): Leo Porter University of California San Diego | ||
09:10 25mTalk | Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks Research Papers Victor-Alexandru Padurean Max Planck Institute for Software Systems, Kaitlin Riegel University of Auckland, Gweneth Barbre Abilene Christian University, Musa Blake Abilene Christian University, Paul Denny The University of Auckland, Alkis Gotovos Max Planck Institute for Software Systems, Juho Leinonen Aalto University, Stephen MacNeil Temple University, James Prather Abilene Christian University, Adish Singla Max Planck Institute for Software Systems Link to publication DOI | ||
09:35 25mTalk | Scaffolded AI-Verification: Assessment Patterns for Resource-Constrained Environments. Research Papers Kehinde Aruleba University of Leicester, Kikelomo Ladipo University of Leicester, UK, Ismaila Temitayo Sanusi University of Eastern Finland, Solomon Oyelere University of Exeter | ||
10:00 25mTalk | When AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code Research Papers Victor-Alexandru Padurean Max Planck Institute for Software Systems, Kaitlin Riegel University of Auckland, Alkis Gotovos Max Planck Institute for Software Systems, Jyotika Mahapatra Max Planck Institute for Software Systems, Ahana Ghosh Max Planck Institute for Software Systems, Paul Denny The University of Auckland, Juho Leinonen Aalto University, James Prather Abilene Christian University, Adish Singla Max Planck Institute for Software Systems Link to publication DOI | ||