other · ACM Transactions on Computing Education · la publicación, 12 nov 2026 · gratis
Los estudiantes dejaron de pedir el código y empezaron a preguntar por qué fallaba
Un curso de Python en Japón siguió a 248 alumnos durante tres años con un asistente de IA conectado. Las calificaciones no cambiaron de forma significativa, y el estudio no puede decir que la herramienta sea la causa.
Versión breve · la versión detallada sigue, unos 7 min
Pregunte al chat de weeklyAI
Pregúnteme sobre la IA en su vida: le señalo las notas que weeklyAI ha publicado sobre el tema, esta semana y antes.
Las conversaciones se guardan mientras exista weeklyAI, para mejorar la publicación. Se responde en el idioma en que usted escribe.
- El estudio, de un vistazo
- Quiénes
- Estudiantes de un curso de introducción a Python
- Cuántos
- 248 estudiantes
- Dónde
- Japón
- Cuándo
- 2023 a 2025
- Tipo de estudio
- analysis of what people did
- Quién lo hizo
- Universidad de Kyushu y Universidad de Kyoiku de Osaka
- El límite que importa
- Compara grupos de años distintos; no puede establecer causa y efecto.
Porcentaje de los mensajes que los estudiantes escribieron al asistente, clasificados a mano en una muestra de las conversaciones; cada año es un grupo distinto de alumnos, no los mismos estudiantes seguidos.
Calificaciones de tareas semanales: curso sin IA frente a los tres años con IA
Prácticamente iguales: 96.50 frente a 96.28
Prácticamente iguales: 96.50 frente a 98.18
Prácticamente iguales: 96.50 frente a 98.43
El dato que más pesa aquí no es cuántos usan el asistente, sino para qué lo usan.
En un curso de introducción a Python en una universidad nacional de Japón, los alumnos tuvieron acceso a un asistente de inteligencia artificial basado en GPT. El curso se repitió en 2023, 2024 y 2025. Los alumnos lo usaban desde una página web propia del curso. Cada conversación quedó registrada. El curso se dio tres veces, en 2023, 2024 y 2025, con 248 estudiantes en total. Casi todos cursaban el primer año y 9 de cada 10 se describían como principiantes.
En el primer año, los alumnos pedían sobre todo que el asistente les escribiera el código: el 42.67% de sus mensajes eran de ese tipo. Las conversaciones solían durar poco. Muchas terminaban con una sola respuesta, como quien consulta un motor de búsqueda.
En 2024 el patrón se invirtió. La petición más común pasó a ser revisar el código propio: el 52.25% de los mensajes. Los diálogos se hicieron más largos y encadenados.
En 2025 los alumnos alternaron entre leer los mensajes de error (el 36.18% de los mensajes), comprobar sus arreglos y volver a escribir. Fue el año con las conversaciones más entrelazadas de los tres.
Las calificaciones de las tareas y las notas finales se mantuvieron parecidas a las de un curso anterior sin IA. En las tareas no hubo diferencia significativa. Conviene mirar cómo se calificaba: los alumnos podían entregar varias veces hasta que sus programas pasaran todas las pruebas. En un curso con un solo intento, el resultado podría ser otro.
El estudio compara grupos de años distintos; no controla lo que cambió entre uno y otro. Los modelos de IA mejoraron, y los compañeros se influyen entre sí. Cualquiera de esas cosas pudo mover las cifras tanto como la costumbre de usar la herramienta.
El registro cubre solo el uso dentro de la plataforma. No dice cuánto preguntaban a compañeros, ayudantes o internet, ni si dejaron de hacerlo.
Cuando el asistente se equivocaba y además no servía, fue poco frecuente: entre el 0% y el 3.69% de las respuestas revisadas.
En su propio curso, fíjese en una cosa: si las preguntas de los alumnos pasan de "escríbeme esto" a "por qué falló".
Qué significa para usted
En su propio curso, vale la pena mirar qué preguntan los alumnos cuando algo no funciona: si pasan de "escríbeme el código" a "por qué falló", algo está cambiando en su manera de aprender. El estudio no puede decir que la herramienta haya mejorado las calificaciones, así que conviene observar también cómo se califica cada entrega.
Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769
Quién pagó: El artículo no indica quién financió el estudio, si algún financiador tuvo influencia, ni quién proporcionó la herramienta basada en GPT o la interfaz.
Versión detalladaLos pasajes copiados del artículo, las ilustraciones y cada fuente con cuánto leímos de ella · 1306 palabras · unos 7 minLeerla →Cerrar
De pedir el código terminado a revisar el propio: lo que muestra un curso de Python observado tres años
Un estudio siguió a tres generaciones de estudiantes de programación con un asistente de inteligencia artificial en el aula. La forma de usarlo cambió; las calificaciones, casi no.
Un aula de primer año, catorce clases de Python, un asistente de inteligencia artificial disponible en todo momento desde una página web hecha para el curso. Cada estudiante podía escribirle una pregunta y recibir una respuesta al instante. El estudio siguió ese arreglo durante tres generaciones seguidas, de 2023 a 2025, en una universidad nacional de Japón, y sumó a 248 estudiantes en total: 62 en el primer año, 126 en el segundo y 60 en el tercero1. Casi todos cursaban su primer año de carrera y nueve de cada diez se describían como principiantes en Python.
Para saber si el asistente cambiaba algo, hacía falta un punto de partida. Los autores incluyeron una oferta anterior del mismo curso, impartida por el mismo profesor a principios de 2023, cuando la inteligencia artificial generativa todavía no se había vuelto costumbre entre los estudiantes de ese lugar; esa oferta tuvo 39 alumnos2. Lo que sigue compara esa referencia y las tres generaciones entre sí.
La familiaridad con estas herramientas subió año con año. En 2023, solo 5.2% de los encuestados decía conocerlas bien, más de una cuarta parte las conocía poco y la mayoría las conocía a medias3. Para 2025, todos reportaban al menos un conocimiento medio y 60% decía conocerlas bien4. El uso activo pasó de 36.8% a 91.7% en el mismo periodo, con apenas 8.3% que reportaba no usarlas en 20255.
Conviene detenerse en qué es el asistente del que se habla. No es un buscador ni un libro de texto: es un programa entrenado con enormes cantidades de texto que, ante una pregunta escrita, produce una respuesta nueva palabra por palabra, como si continuara una conversación. Por eso puede explicar un concepto, escribir una función o interpretar un mensaje de error, y por eso también puede equivocarse con total seguridad. En este curso, además, todo lo que los estudiantes le escribieron quedó registrado, y los investigadores clasificaron una parte de esas conversaciones a mano.
Ahí está el cambio más interesante. En 2023, los estudiantes le pedían sobre todo que escribiera el código: las preguntas de implementación eran 42.67% del total6. En 2024, la proporción se invirtió: las preguntas para verificar código llegaron a 52.25% y las de implementación cayeron a 7.87%7. En 2025 el patrón se volvió más equilibrado, con las preguntas sobre mensajes de error subiendo a 36.18% y una mezcla de verificación (23.27%) e implementación (17.28%) todavía presente8. Los autores lo resumen así: se pasó de un estilo centrado en implementar, a uno centrado en verificar, y luego a un patrón más balanceado de depurar, verificar e implementar9.
Piense en lo que eso significa en la práctica. Pedir el código es una sola pregunta y una sola respuesta: se copia, se entrega y se acabó. Interpretar un mensaje de error es otra cosa: hay que leer lo que el programa dice, preguntar por qué, probar un cambio, volver a preguntar. La conversación se alarga. Los propios autores notaron que en 2023 los estudiantes hacían menos peticiones por semana, 3.05 en promedio, y que en 2024 y 2025 subieron a 4.68 y 4.75. No es que preguntaran más por preguntar: la estructura de sus diálogos se volvió más enredada, con más idas y vueltas entre entender, escribir, corregir y comprobar.
Y las calificaciones, ¿subieron? No de forma apreciable. Los puntajes de las tareas semanales fueron 96.50 en la oferta de referencia, 96.28 en 2023, 98.18 en 2024 y 98.43 en 2025; la prueba estadística no encontró diferencia significativa entre generaciones10. Las calificaciones finales también se mantuvieron parecidas. Los autores concluyen que tener acceso a la inteligencia artificial, por sí solo, no cambió de manera sustancial el desempeño observable en las tareas ni en las notas11.
Hay que leer ese resultado con cuidado. Las tareas se calificaban con un esquema que permitía entregar varias veces hasta que todos los casos de prueba pasaran, así que el puntaje reflejaba sobre todo la corrección final, no el camino. Además, aun en la oferta sin inteligencia artificial los estudiantes podían pedir ayuda a un asistente de cátedra, a compañeros o a recursos en línea, lo que probablemente diluyó el efecto de tener o no tener el asistente. Los propios autores advierten que su diseño es observacional: no puede establecer causa y efecto, y señalan como ejemplos de ello las mejoras en los modelos o las costumbres que se contagian entre compañeros12. También señalan que los registros solo capturan el uso dentro de la plataforma y pueden perder otras vías de ayuda, como los compañeros o los profesores12.
Así lo lee weeklyAI. El dato que más pesa aquí no es cuántos usan el asistente, sino para qué lo usan. Una herramienta que responde al instante puede dejar a un principiante en la etapa de preguntar y recibir, sin dar el paso siguiente: dudar de la respuesta, comprobarla, explicarla con sus propias palabras. En este curso, el paso sí ocurrió, pero no solo: ocurrió dentro de una asignatura con ejercicios semanales, un profesor presente y la instrucción de citar las fuentes externas que se usaran. Esta lectura se equivocaría si los estudiantes recién llegados, sin ninguna guía del curso, pasaran igual de rápido a conversaciones largas de comprobar y corregir su propio código.
Esa es la pregunta que un docente puede llevarse a su salón: ¿mis alumnos cierran la conversación con la primera respuesta o siguen preguntando? Un ejercicio sencillo es pedirles que anoten una segunda pregunta antes de dar una tarea por terminada, o que expliquen por qué la respuesta de la máquina les sirve o no. El criterio se practica; no se supone. Y hay algo que este estudio no puede decir: si los estudiantes aprendieron a usar mejor la herramienta por el curso, por la costumbre de sus compañeros o porque los modelos mejoraron. Las tres cosas iban juntas en el tiempo.
Otros trabajos que se han hecho sobre el tema apuntan en la misma dirección. Un experimento aleatorizado con 185 estudiantes universitarios encontró que el desempeño en una prueba fue más alto cuando los alumnos trabajaban con textos y con el asistente que cuando trabajaban solo con el asistente, y que la frecuencia de interacción no se relacionó con el resultado; lo que sí se asoció fue el tipo de preguntas: profundizar ayudaba, aceptar sin más la respuesta de la máquina perjudicaba, según el resumen de ese estudio, lo único disponible de él131415.
Ese mismo resumen sugiere que el valor educativo de estas herramientas puede depender menos del acceso a la tecnología que de cómo el estudiante se compromete con ella16. Es una idea que conviene tener presente cuando se discute si prohibir o permitir el asistente en clase: la pregunta útil no es si está ahí, sino qué se hace con él.
La conclusión de los autores va en esa línea: en la era de la inteligencia artificial, el reto central de la enseñanza es menos si los estudiantes usan estas herramientas y más cómo los cursos redefinen prácticas de aprendizaje productivas sin quitarles agencia a los estudiantes17.
Vuelva al aula del principio: catorce clases, un asistente en pantalla, estudiantes que al principio le pedían el código y tres años después lo usaban para revisar el suyo. Lo que cambió no fue la nota; fue la conversación. Si usted enseña programación, o si su hijo o hija acaba de entrar a una carrera donde se programa, la pregunta que puede llevar a la siguiente junta o a la siguiente tarea es sencilla: ¿qué está haciendo el estudiante con la respuesta que recibe, y cómo se lo estamos enseñando?
De dónde sale cada dato de contexto, y cuánto leímos de cada documento
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "The study spanned from 2023 to 2025 and consisted of three offerings of the same course, with a total of 248 students."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "To provide a pre-GenAI point of reference for RQ3, we also include a separate baseline offering taught by the same instructor in early 2023 (n=39), when students in our context had not yet widely adopted GenAI."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "In 2023, only 5.2% of respondents reported high familiarity, while the majority fell into the medium category (68.4%) and over a quarter reported low familiarity (26.3%)."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "By 2025, all respondents reported at least medium familiarity, and 60% reported high familiarity, suggesting that GenAI had become a more common and better understood tool before entering the course."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "By 2025, GenAI use became nearly universal: 91.7% reported active use and only 8.3% reported no use."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "In 2023, students’ prompts were dominated by Code Implementation prompts (42.67%), indicating an implementation-driven pattern in which learners primarily asked the AI to help write or complete code."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "In 2024, this pattern changed markedly: Code Verification became the most prevalent prompt type (52.25%), while Code Implementation dropped to 7.87%."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "By 2025, students’ help-seeking behaviors shifted again toward debugging-oriented use. Error Message Interpretation rose to 36.18%, while Code Verification (23.27%) and Code Implementation (17.28%) remained substantial."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Overall, the distributional changes indicate that student–AI interaction moved from an implementation-driven style (2023), to a verification-driven style (2024), and then to a more balanced debug–verify–implement pattern (2025)."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "A Kruskal–Wallis test found no statistically significant differences across cohorts (H = 3.23, p = 0.357). Final-grade means were also broadly similar across years, suggesting that overall performance levels did not shift dramatically despite substantial changes in GenAI adoption and interaction patterns."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Taken together, these results suggest that AI access alone did not substantially change observable performance on course assignments or overall grades."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Second, our cohort-level observational design cannot establish causality (e.g., model improvements or peer norms). Third, dialogue logs capture only in-platform GenAI use and may miss other support channels (e.g., peers or TAs)."
- Dietrich L, Fink L, Urhahne D, Kremer K. (2026). What Makes Learning with GenAI Effective? Text Support and Deeper Engagement are Associated with Higher Performance in an Ecology Context. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-026-10374-w — solo el resumen - no se pudo obtener el texto completo — el pasaje: "Test performance was highest in the text-only and text + ChatGPT conditions and lowest in the ChatGPT-only condition, yielding a robust overall condition effect."
- Dietrich L, Fink L, Urhahne D, Kremer K. (2026). What Makes Learning with GenAI Effective? Text Support and Deeper Engagement are Associated with Higher Performance in an Ecology Context. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-026-10374-w — solo el resumen - no se pudo obtener el texto completo — el pasaje: "Although ChatGPT-only students produced substantially more interaction turns than students with access to both texts and ChatGPT, interaction frequency was unrelated to test performance."
- Dietrich L, Fink L, Urhahne D, Kremer K. (2026). What Makes Learning with GenAI Effective? Text Support and Deeper Engagement are Associated with Higher Performance in an Ecology Context. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-026-10374-w — solo el resumen - no se pudo obtener el texto completo — el pasaje: "Instead, specific interaction patterns were differentially associated with performance: deepening questions were positively associated, whereas direct acceptance of AI-generated responses was negatively associated with test performance."
- Dietrich L, Fink L, Urhahne D, Kremer K. (2026). What Makes Learning with GenAI Effective? Text Support and Deeper Engagement are Associated with Higher Performance in an Ecology Context. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-026-10374-w — solo el resumen - no se pudo obtener el texto completo — el pasaje: "The findings suggest that, in science learning contexts, the educational value of GenAI may depend less on access to the technology than on learners’ epistemic engagement with it."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - el artículo del que trata esta nota — el artículo completo — el pasaje: "These changes suggest that, in the AI era, the central instructional challenge is less about whether students use AI and more about how courses redefine productive learning practices while maintaining student agency."
Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769
Quién pagó: El artículo no indica quién financió el estudio, si algún financiador tuvo influencia, ni quién proporcionó la herramienta basada en GPT o la interfaz.
other · ACM Transactions on Computing Education · the paper, 12 Nov 2026 · free
Fewer students asked the AI to write code as the years went on
In one Python course in Japan, requests shifted from finished code to checking and fixing. Grades stayed about the same, and the study cannot say why.
Short version · the longer version follows, about 7 min
Ask the weeklyAI chat
Ask me about AI in your life: I point you to the stories weeklyAI has published on it, this week and before.
Conversations are saved for as long as weeklyAI exists, to improve the publication. Answers come in the language you write in.
- The study at a glance
- Who
- undergraduates in an introductory Python course
- How many
- 248 students
- Where
- Japan
- When
- 2023 to 2025
- Kind of study
- analysis of what people did
- Who did it
- Kyushu University and Osaka Kyoiku University
- The limit that matters
- One course, one institution; year groups were compared, not randomly assigned, so cause cannot be shown.
Shares of coded student prompts in each year; each year's prompts were sorted into many categories, so these shares do not add to 100.
Average weekly assignment scores, before and during AI use
96.50 versus 96.28
96.50 versus 98.18
96.50 versus 98.43
The study compared year groups; it did not control them.
A first-year Python course at a national university in Japan ran three times, in 2023, 2024 and 2025. In all, 248 students took the course across the three years. Each time, the course gave every student a GPT-based assistant through a custom web page, and every conversation was logged.
In 2023, students mostly wanted code. Prompts asking the assistant to write or complete an implementation made up 42.67 percent of their questions, and many exchanges ended after a single answer.
Then the pattern flipped. In 2024, checking their own code became the most common request, at 52.25 percent of prompts, while requests for finished code fell to 7.87 percent. The logs show longer back-and-forth: students verified, revised, and verified again.
By 2025, students moved between several kinds of help. Interpreting error messages rose to 36.18 percent of prompts, alongside verifying fixes and writing code. Of the three years, these conversations were the most interconnected.
The grades did not move with them. Weekly assignment scores averaged 96.50 in a comparison course taught before AI was widely used, then 96.28, 98.18 and 98.43 across the three years. The difference was not statistically significant. The researchers give a few possible reasons the scores stayed flat. One is that students could resubmit assignments until all tests passed, so the scores mostly record eventual correctness.
The study compared year groups; it did not control them. Better AI models, or students arriving with different habits, could explain the shift as easily as growing skill with the tool. The logs also captured only the in-course assistant. Help from classmates, teaching assistants or websites went unrecorded.
The students themselves changed too. Those reporting high familiarity with AI rose from 5.2 percent in 2023 to 60 percent in 2025, and active use from 36.8 percent to 91.7 percent. Not every answer landed: between 5.30 and 7.82 percent of AI responses were correct but unhelpful, and up to 3.69 percent were both wrong and unhelpful.
In your own course, watch what students type into the box. When the questions turn from "write this for me" to "why did this fail," they are doing what this study describes.
What this means for you
What this means for a teacher is a question worth asking of the tool itself: when a student pastes in an error message, are they learning to read it, or handing it off? The study watched one course in one country and cannot say the tool changed anyone's grades, so watch the questions your own students ask and whether they come back to check the fix. That habit is the thing to notice.
Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769
Who paid: The article does not say who funded the study, whether any funder had a say, or who supplied the GPT-based tool or interface.
The longer versionThe passages copied from the paper, the pictures, and every source with how much of it we read · 1454 words · about 7 minRead it →Close
They Asked the Chatbot for Code. Three Years Later, They Asked It to Check Their Work.
One Japanese Python course tracked 248 students through the arrival of everyday AI. Their grades barely moved.
A student types a broken line of Python into a chat box and presses send. The reply comes back with an explanation of the error. The student fixes the code, runs it, and asks the assistant to check the result.
That small back-and-forth is what a three-year study of an introductory Python course set out to watch. The course ran at a national university in Japan, once in 2023, once in 2024 and once in 2025, with 248 undergraduates in all1. The authors are Boxuan Ma, Huiyong Li, Gen Li, Li Chen, Cheng Tang, Atsushi Shimada and Shin'ichi Konomi; the article lists Li Chen at Osaka Kyoiku University and the other six at Kyushu University.
Every student in the three years had access to a custom web interface that connected to a GPT-based assistant, offered as a supplement to lectures, slides and instructors. Every conversation was logged. The study also drew on questionnaires and course records, and it compared the three years against a separate earlier offering taught by the same instructor in 2023, before AI use was common, with 39 students2.
The first thing that changed was the students themselves. In 2023, 5.2 percent said they were highly familiar with generative AI, and about a quarter said they had low familiarity3. By 2025, every respondent reported at least medium familiarity and 60 percent reported high familiarity4. Active use followed the same path: 91.7 percent reported using these tools by the final year, against 8.3 percent who reported none5.
The conversations changed too, and this is the finding worth carrying away. In 2023, the most common request was for code: 42.67 percent of coded prompts asked the assistant to write or complete a program6. In 2024, the largest share by far, 52.25 percent, asked the assistant to verify code the student had already written, while requests to write code fell to 7.87 percent7. By 2025, interpreting error messages had risen to 36.18 percent, with verification at 23.27 percent and code writing at 17.28 percent8. The authors describe a move from asking for implementations, to checking them, to a mix of debugging, verifying and writing9.
Grades did not follow. Average assignment scores were 96.50 in the pre-AI comparison group, then 96.28, 98.18 and 98.43 across the three AI years. Final grades averaged 4.17, 4.28, 4.04 and 4.30. A standard test for differences across the years found none on assignment scores10. The authors read this as a sign that access to AI alone did not substantially change what showed up on assignments or in final grades11.
Consider what the grade actually certifies. The course allowed students to submit weekly exercises repeatedly until the automatic tests passed, so the score records eventual correctness rather than first attempts. The pre-AI comparison group could also ask teaching assistants, classmates or the internet for help, which would soften any difference AI made. The authors name both points themselves. None of this means the flat grades are wrong; it means they measure a narrow thing.
The study has limits the authors state plainly. It is one course at one institution, so the pattern may not hold in other countries, other programming languages or other grading schemes. Because it watched successive groups rather than randomly assigning students to conditions, it cannot say the assistant caused any of the shifts; better models and the habits of classmates could explain them just as well12. And the logs capture only what students typed into the course interface, not help sought from people12.
Other work on this question has gone further, and differently. A randomized experiment with 185 university students, working on task sheets about lake restoration, compared reading texts alone, using a chatbot restricted to those texts, and using both; test scores were highest with texts alone or texts plus chatbot and lowest with the chatbot only, and the frequency of interaction did not predict performance1314. What did predict it was the kind of question: deepening questions helped, and simply accepting the assistant's answers hurt15. According to the summary of that study, the only part available, the value of the tool depended less on access than on how hard the learner pushed on the material16.
A different design produced a different result. A quasi-experiment with 312 undergraduates, over eight weeks, compared ordinary teacher feedback on writing with structured AI feedback on a rubric, moderated by a teacher and paired with learning analytics; the AI-supported group's final writing score rose by 4.46 points, self-regulation rose, and teachers spent less time per feedback cycle17181920. According to the summary, the only part available, the authors present the combination of rubric feedback, teacher moderation and analytics as promising for scaling formative assessment21. The difference from the programming course is worth holding onto: there, the assistant answered an open request; here, it worked inside a rubric a teacher had set.
A third trial, with 362 nursing students in a 16-week anatomy course, compared a generative AI digital tutor combined with a knowledge graph against blended and traditional teaching; the AI-enhanced group scored higher on module tests and final comprehensive scores222324. According to the summary, the only part available, the authors report higher knowledge retention and stronger clinical reasoning in that group and propose the model for reforming foundational medical education2526. These are randomized comparisons with clear outcomes, and they sit alongside a three-year observation that found no grade change. Both can be true: a tool can lift learning when it is built into a task and change nothing when it is simply available.
One more number deserves a look: the quality of the assistant's answers. Coders judged between 89.39 and 94.67 percent of sampled replies both correct and helpful across the years. Correct but unhelpful replies, technically valid yet too advanced or off the course's intended approach, persisted at 5.30 to 7.82 percent. Incorrect and unhelpful replies reached 3.69 percent in 2025. The authors trace much of this to underspecified prompts and missing course context rather than to wholesale error.
Here is weeklyAI's reading. A course that lets students resubmit until the tests pass, and hands nearly everyone a score in the high nineties, teaches a quiet lesson: the right result is the point. A student who asked for finished code in 2023 and a student who debugged her own in 2025 may have walked out with the same transcript and very different abilities. The flat grades do not prove that happened. They do mean the grades cannot rule it out. This reading would be wrong if students from the early, code-requesting year performed as well as later students on an unassisted task or a delayed check of what they retained. The study did not run such a check, and that is the gap to watch.
A second thing weeklyAI expects, offered as an expectation and not as a finding: the jump from a minority to nearly everyone using these tools probably owes something to simple prevalence. When almost everyone in a class uses an assistant, using it stops being a decision and becomes the way the course is done. This reading would be wrong if later students chose the tool mainly from their own judgment of what it did for them, and if the heaviest users were also the ones most likely to check an answer against the slides or an instructor. A class can test this cheaply: ask why they use it, and listen for whether the answer is a reason or a crowd.
The practical turn for a teacher is the sequence the logs show. Verification and error-reading are learnable habits, and they can be assigned. Ask students to submit one answer they rejected and say why. Ask them to explain an error message before they ask for a fix. The assistant's confident wrong answers, small in share but present, are the raw material for that practice.
For a parent or a school leader in Latin America, the United States or Canada, the study says nothing about your course, your grading rules or your students. What it offers is a set of questions that travel. Does the grade rest on retries until tests pass? Is any work done without the assistant and checked? What does the class believe the assistant is for: producing answers, or testing them?
The student at the keyboard now asks the machine to check the code, not to write it. Whether that habit means the student learned more is a question the transcript cannot answer. What would you want to see a learner do, unaided, before you trusted the score?
Where each piece of context comes from, and how much of it we read
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "The study spanned from 2023 to 2025 and consisted of three offerings of the same course, with a total of 248 students."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "To provide a pre-GenAI point of reference for RQ3, we also include a separate baseline offering taught by the same instructor in early 2023 (n=39), when students in our context had not yet widely adopted GenAI."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "In 2023, only 5.2% of respondents reported high familiarity, while the majority fell into the medium category (68.4%) and over a quarter reported low familiarity (26.3%)."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "By 2025, all respondents reported at least medium familiarity, and 60% reported high familiarity, suggesting that GenAI had become a more common and better understood tool before entering the course."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "By 2025, GenAI use became nearly universal: 91.7% reported active use and only 8.3% reported no use."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "In 2023, students’ prompts were dominated by Code Implementation prompts (42.67%), indicating an implementation-driven pattern in which learners primarily asked the AI to help write or complete code."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "In 2024, this pattern changed markedly: Code Verification became the most prevalent prompt type (52.25%), while Code Implementation dropped to 7.87%."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "By 2025, students’ help-seeking behaviors shifted again toward debugging-oriented use. Error Message Interpretation rose to 36.18%, while Code Verification (23.27%) and Code Implementation (17.28%) remained substantial."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "Overall, the distributional changes indicate that student–AI interaction moved from an implementation-driven style (2023), to a verification-driven style (2024), and then to a more balanced debug–verify–implement pattern (2025)."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "A Kruskal–Wallis test found no statistically significant differences across cohorts (H = 3.23, p = 0.357). Final-grade means were also broadly similar across years, suggesting that overall performance levels did not shift dramatically despite substantial changes in GenAI adoption and interaction patterns."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "Taken together, these results suggest that AI access alone did not substantially change observable performance on course assignments or overall grades."
- Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769 - the article this story is about — the whole article — the passage: "Second, our cohort-level observational design cannot establish causality (e.g., model improvements or peer norms). Third, dialogue logs capture only in-platform GenAI use and may miss other support channels (e.g., peers or TAs)."
- Dietrich L, Fink L, Urhahne D, Kremer K. (2026). What Makes Learning with GenAI Effective? Text Support and Deeper Engagement are Associated with Higher Performance in an Ecology Context. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-026-10374-w — only the abstract - the full text could not be fetched — the passage: "Test performance was highest in the text-only and text + ChatGPT conditions and lowest in the ChatGPT-only condition, yielding a robust overall condition effect."
- Dietrich L, Fink L, Urhahne D, Kremer K. (2026). What Makes Learning with GenAI Effective? Text Support and Deeper Engagement are Associated with Higher Performance in an Ecology Context. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-026-10374-w — only the abstract - the full text could not be fetched — the passage: "Although ChatGPT-only students produced substantially more interaction turns than students with access to both texts and ChatGPT, interaction frequency was unrelated to test performance."
- Dietrich L, Fink L, Urhahne D, Kremer K. (2026). What Makes Learning with GenAI Effective? Text Support and Deeper Engagement are Associated with Higher Performance in an Ecology Context. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-026-10374-w — only the abstract - the full text could not be fetched — the passage: "Instead, specific interaction patterns were differentially associated with performance: deepening questions were positively associated, whereas direct acceptance of AI-generated responses was negatively associated with test performance."
- Dietrich L, Fink L, Urhahne D, Kremer K. (2026). What Makes Learning with GenAI Effective? Text Support and Deeper Engagement are Associated with Higher Performance in an Ecology Context. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-026-10374-w — only the abstract - the full text could not be fetched — the passage: "The findings suggest that, in science learning contexts, the educational value of GenAI may depend less on access to the technology than on learners’ epistemic engagement with it."
- Usyk DA, Unatlokov VK. (2026). Personalized Feedback Based on Artificial Intelligence in Higher Education: Impact on the Quality of Assignment Performance. Open Education. https://doi.org/10.21686/1818-4243-2026-5-14-23 — only the abstract - the full text could not be fetched — the passage: "the intervention was associated with a 4.46-point increase in the final writing score (95% C.I. 3.25–5.68; p<0.001; Hedges’ g=0.84)"
- Usyk DA, Unatlokov VK. (2026). Personalized Feedback Based on Artificial Intelligence in Higher Education: Impact on the Quality of Assignment Performance. Open Education. https://doi.org/10.21686/1818-4243-2026-5-14-23 — only the abstract - the full text could not be fetched — the passage: "a 0.41-point increase in self-regulation (95% C.I. 0.22–0.60; p<0.001; g=0.50)"
- Usyk DA, Unatlokov VK. (2026). Personalized Feedback Based on Artificial Intelligence in Higher Education: Impact on the Quality of Assignment Performance. Open Education. https://doi.org/10.21686/1818-4243-2026-5-14-23 — only the abstract - the full text could not be fetched — the passage: "Average teacher efforts per feedback cycle was lower in the intervention (7.07±2.44 min) compared to the control (18.28±4.02 min; p<0.001)."
- Usyk DA, Unatlokov VK. (2026). Personalized Feedback Based on Artificial Intelligence in Higher Education: Impact on the Quality of Assignment Performance. Open Education. https://doi.org/10.21686/1818-4243-2026-5-14-23 — only the abstract - the full text could not be fetched — the passage: "The control group received traditional teacher feedback; the intervention group received structured AI feedback on the rubric with mandatory teacher moderation and individualized learning analytics in the LMS."
- Usyk DA, Unatlokov VK. (2026). Personalized Feedback Based on Artificial Intelligence in Higher Education: Impact on the Quality of Assignment Performance. Open Education. https://doi.org/10.21686/1818-4243-2026-5-14-23 — only the abstract - the full text could not be fetched — the passage: "The combination of rubric-based AI feedback, teacher moderation, and learning analytics is a promising didactic tool for scaling up formative assessment: it improves the quality of academic writing and regulatory competencies while reducing subjective teacher workload and efforts."
- Zhao C, Zhu J, Liu J, Zhao W, Pang Y. (2026). Effectiveness of a generative AI-powered digital tutor integrated with a knowledge graph in anatomy education for nursing students: a randomized controlled trial. BMC Medical Education. 10.1186/s12909-026-09469-0 — only the abstract - the full text could not be fetched — the passage: "A three-arm parallel randomized controlled trial was conducted among 362 first-year nursing students at Cangzhou Medical College, of whom 301 completed the study."
- Zhao C, Zhu J, Liu J, Zhao W, Pang Y. (2026). Effectiveness of a generative AI-powered digital tutor integrated with a knowledge graph in anatomy education for nursing students: a randomized controlled trial. BMC Medical Education. 10.1186/s12909-026-09469-0 — only the abstract - the full text could not be fetched — the passage: "The AI-Enhanced Group achieved higher module scores (82.5 ± 7.3) compared with the Blended Group (77.6 ± 8.1) and Traditional Teaching Group (74.1 ± 8.5) (F = 28.64, p < 0.001)."
- Zhao C, Zhu J, Liu J, Zhao W, Pang Y. (2026). Effectiveness of a generative AI-powered digital tutor integrated with a knowledge graph in anatomy education for nursing students: a randomized controlled trial. BMC Medical Education. 10.1186/s12909-026-09469-0 — only the abstract - the full text could not be fetched — the passage: "Final comprehensive scores were also significantly higher in the AI-Enhanced Group (84.2 ± 6.8) (F = 32.16, p < 0.001)."
- Zhao C, Zhu J, Liu J, Zhao W, Pang Y. (2026). Effectiveness of a generative AI-powered digital tutor integrated with a knowledge graph in anatomy education for nursing students: a randomized controlled trial. BMC Medical Education. 10.1186/s12909-026-09469-0 — only the abstract - the full text could not be fetched — the passage: "Furthermore, the AI-Enhanced Group demonstrated higher knowledge retention rates (82.3%) and case-based inference accuracy (91.2%) than the comparison groups."
- Zhao C, Zhu J, Liu J, Zhao W, Pang Y. (2026). Effectiveness of a generative AI-powered digital tutor integrated with a knowledge graph in anatomy education for nursing students: a randomized controlled trial. BMC Medical Education. 10.1186/s12909-026-09469-0 — only the abstract - the full text could not be fetched — the passage: "This AI-enhanced instructional model may provide an effective approach for reforming foundational medical education and promoting structured knowledge learning."
Ma, B., Li, H., Li, G. et al. (2026). Three Years with Classroom AI in Introductory Programming: Shifts in Student Awareness, Interaction, and Performance. ACM Transactions on Computing Education. https://doi.org/10.1145/3862769
Who paid: The article does not say who funded the study, whether any funder had a say, or who supplied the GPT-based tool or interface.

