Student-AI Interaction: A Case Study of CS1 students

Generative artificial intelligence tools (Generative AI), such as ChatGPT, allow users to interact with them in intuitive ways (e.g., conversational) and receive (mostly) good-quality answers. In education, such systems can support students’ learning objectives by providing accessible explanations and examples even when students pose vague queries. But, they also encourage undesired help-seeking behaviors, such as by providing solutions to the students’ homework. Therefore, it is important to better understand how students approach such tools and the potential issues such approaches might present for the learners. In this paper, we present a case study for understanding student-AI collaboration to solve programming tasks in the CS1 introductory programming course. To this end, we recruited a gender-balanced majority non-white set of 15 CS1 students at the University of Houston, a large public university in the US. We observed them solving programming tasks. We used a mixed-method approach to study their interactions as they tackled Python programming tasks, focusing on when and why they used ChatGPT for problem-solving. We analyze and classify the questions submitted by the 15 participants to ChatGPT. Additionally, we analyzed user interaction patterns, their reactions to ChatGPT’s responses, and the potential impacts of Generative AI on their perception of self-efficacy. Our results suggest that, in about a third of the cases, the student attempted to complete the task by submitting the full description of the tasks to ChatGPT without making any effort on their own. We also observed that few students verified their solutions. We discuss the potential implications of these results.