Weekly AI Watch

Every week, what changed and what it means for you. No hurry, no alarm.

Week of 20 November 2026 | Past editions15,959 papers found · 15,002 read · 5,264 belonged · 42 stories passed every check

Observatorio de la
Inteligencia Artificial

Cada semana, qué cambió y qué significa para usted. Sin prisa y sin alarma.

Semana del 20 de noviembre de 2026 | Ediciones anteriores15,959 estudios encontrados · 15,002 leídos · 5,264 correspondían · 42 notas pasaron todas las comprobaciones

weeklyAI · Week of 20 November 2026weeklyAI · Semana del 20 de noviembre de 2026

← Your rights← Sus derechos

experiment · BMC medical education · la publicación, 17 jun 2026 · gratis

Dos exámenes de práctica salieron casi iguales, y uno se escribió 5.6 veces más rápido

Un ensayo en una sola escuela de medicina de Canadá comparó preguntas hechas por ChatGPT con preguntas hechas por estudiantes. No dice nada sobre un examen real ni sobre preguntas de profesores.

Versión breve · la versión detallada sigue, unos 8 min

Pregunte al chat de weeklyAI

Pregúnteme sobre la IA en su vida: le señalo las notas que weeklyAI ha publicado sobre el tema, esta semana y antes.

Las conversaciones se guardan mientras exista weeklyAI, para mejorar la publicación. Se responde en el idioma en que usted escribe.

El estudio, de un vistazo
Quiénes
Estudiantes de primer año de medicina
Cuántos
258 estudiantes
Dónde
Canadá (Universidad de Columbia Británica, Vancouver)
Cuándo
8 y 9 de diciembre de 2024
Tipo de estudio
experiment
Quién lo hizo
Facultad de Medicina de la Universidad de Columbia Británica
El límite que importa
Fue un examen de práctica, con apuntes a la mano, en una sola escuela; no dice nada de un examen real.
Preguntas que separaron poco a los alumnos, según quién las escribió
Preguntas escritas por inteligencia artificial62.5%
Preguntas escritas por estudiantes42.9%

Porcentaje de preguntas que cayeron en la categoría de baja capacidad para distinguir a quien sabe de quien no sabe, en un examen de práctica con apuntes a la mano.

Preguntas hechas por inteligencia artificial frente a preguntas hechas por estudiantes avanzados

Preguntas hechas por inteligencia artificialfrente aPreguntas hechas por estudiantes avanzados

Cada pregunta de la máquina tomó 4.2 minutos; cada una de los estudiantes, 19.6 minutos.

Preguntas hechas por inteligencia artificialfrente aPreguntas hechas por estudiantes avanzados

Los alumnos calificaron los dos exámenes de forma parecida en claridad, dificultad, relación con el curso y valor educativo.

Preguntas hechas por inteligencia artificialfrente aPreguntas hechas por estudiantes avanzados

Las preguntas de los estudiantes separaron un poco mejor a quien sabía de quien no sabía; la diferencia fue pequeña.

Preguntas hechas por inteligencia artificialfrente aPreguntas hechas por estudiantes avanzados

Ninguno de los dos exámenes cambió de forma apreciable la sensación de los alumnos de estar preparados.

Cuando oiga que una escuela, un colegio profesional o una empresa usa inteligencia artificial para escribir o calificar exámenes, pregunte quién revisó las preguntas y si se compararon con preguntas escritas por personas.
Lectura de weeklyAI

En la Facultad de Medicina de la Universidad de Columbia Británica, en Vancouver, armaron dos exámenes de práctica. Cada uno tenía 112 preguntas de opción múltiple sobre los mismos temas del curso. Uno lo escribió ChatGPT-4, y otra inteligencia artificial, Google Gemini, revisó las respuestas. El otro lo escribieron estudiantes de cursos más avanzados, sin ayuda de ninguna máquina.

258 estudiantes de primer año hicieron uno de los dos exámenes. Fueron asignados al azar y no supieron cuál les tocó. Las calificaciones salieron parecidas: 73.4% en el examen de los estudiantes y 71.0% en el de la inteligencia artificial. La diferencia no fue concluyente.

La máquina ganó en tiempo. Cada pregunta le tomó 4.2 minutos, frente a 19.6 minutos de los estudiantes. Son 5.6 veces más rápido. Los alumnos calificaron los dos exámenes de forma parecida en claridad, dificultad y relación con el curso, sin diferencias claras.

Los estudiantes ganaron en otra cosa. Sus preguntas distinguieron un poco mejor a quienes sabían más de quienes sabían menos. La mayoría de las preguntas de la inteligencia artificial quedó en la categoría de baja capacidad para separar a los alumnos: 62.5%, frente a 42.9% de las escritas por estudiantes.

El examen era voluntario y se podía consultar el material mientras se respondía. No contaba para la calificación. Por eso no puede decir si las preguntas hechas por una máquina servirían para un examen real, para una nota o para una licencia profesional.

Tampoco es una comparación contra profesores. Del otro lado había estudiantes avanzados, no examinadores expertos. El estudio no muestra que la inteligencia artificial iguale a un examen escrito por profesionales.

Los autores registraron el ensayo después de haberlo empezado. No se puede comprobar de forma independiente que el plan del estudio se fijó antes de ver los datos, y por eso los hallazgos secundarios conviene tomarlos con cautela. Y los modelos cambian rápido: lo que se midió en diciembre de 2024 puede no aplicar a los de hoy.

Nadie salió lastimado. Los investigadores piden que una persona revise siempre lo que produce la máquina antes de usarlo.

Cuando oiga que una escuela, un colegio profesional o una empresa usa inteligencia artificial para escribir o calificar exámenes, pregunte quién revisó las preguntas y si se compararon con preguntas escritas por personas.

Qué significa para usted

Cuando en su país se hable de usar inteligencia artificial para hacer exámenes, conviene preguntar quién revisó cada pregunta y con qué preguntas se comparó. En este ensayo la máquina escribió mucho más rápido, pero sus preguntas separaron peor a los alumnos y el examen era de práctica, sin nota. Nada de esto dice todavía qué pasará en un examen real.

Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0

Quién pagó: El estudio no recibió financiación externa y los autores declaran no tener conflictos de interés; el artículo no menciona ningún proveedor de equipos o programas más allá de los modelos de IA utilizados (ChatGPT-4 y Google Gemini) y la plataforma REDCap alojada en la University of British Columbia.

No tome esto como consejo médico profesional.

Versión detalladaLos pasajes copiados del artículo, las ilustraciones y cada fuente con cuánto leímos de ella · 1502 palabras · unos 8 minLeerla →Cerrar

La IA escribió las preguntas del examen. Los alumnos no notaron la diferencia.

Un ensayo en una facultad de medicina de Canadá midió cuánto tarda una máquina en redactar un examen y cómo lo califican quienes lo presentan. Lo que no midió es lo que más le importa a un estudiante.

Imagine el cuadernillo de un examen de medicina: 112 preguntas de casos clínicos, cada una con su viñeta, cinco opciones y una explicación de por qué cada respuesta es correcta o no. Detrás de cada pregunta hay horas de alguien leyendo, redactando y corrigiendo. Un grupo de investigación de la facultad de medicina de la Universidad de Columbia Británica, en Vancouver, quiso saber qué pasa si una parte de ese trabajo se le entrega a un programa de inteligencia artificial.

El ensayo se hizo con 258 estudiantes de primer año de medicina, de los 328 inscritos en el curso. La mitad presentó un examen de práctica con preguntas hechas por IA; la otra mitad, con preguntas hechas por estudiantes de cursos superiores. Nadie sabía de dónde venían sus preguntas. El examen se aplicó el 8 y 9 de diciembre de 2024.

El resultado más claro está en el tiempo. Hacer una pregunta con el flujo de IA tomó 4.2 minutos en promedio; hacerla a mano tomó 19.6. Dicho de otra forma: por cada pregunta que un estudiante tardaba casi veinte minutos en pulir, la máquina sacaba más de cinco1. Un examen completo de 112 preguntas: unas 36 horas de trabajo humano contra menos de 8 del sistema.

La segunda parte era saber si los alumnos lo notaban. Calificaron ambos exámenes de manera parecida en claridad, dificultad, relación con el curso y valor educativo. Ninguna diferencia entre un examen y otro alcanzó a ser significativa tras los ajustes estadísticos, y todas fueron pequeñas2. En la práctica: nadie levantó la mano para decir que su examen sonaba a robot.

Donde sí hubo diferencia fue en una medida técnica: qué tanto cada pregunta separa a quien sabe de quien no sabe. Las preguntas escritas por estudiantes discriminaron un poco mejor que las de la IA3. Y el detalle incómodo: en el examen de IA, casi dos de cada tres preguntas cayeron en la categoría de baja discriminación, frente a menos de la mitad en el examen humano4.

Los puntajes finales fueron parecidos. En el análisis principal, el grupo con preguntas humanas promedió 73.4 por ciento y el de IA, 71.0; la diferencia no fue estadísticamente significativa. Y ninguno de los dos exámenes cambió de forma apreciable la sensación de los alumnos de estar preparados para el examen real5.

Hay tres límites que conviene tener presentes. Fue un examen de práctica, con apuntes a la mano, en una sola escuela y con un solo grupo de estudiantes de primer año; los propios autores advierten que esto no se puede trasladar sin más a un examen final con consecuencias6. La comparación fue contra estudiantes avanzados, no contra profesores expertos en redactar exámenes, así que no prueba que la IA iguale a un examinador profesional7. Y el ensayo se registró en el registro público de ensayos clínicos después de haber comenzado, no antes, lo que deja una puerta abierta al sesgo en cómo se reportaron los resultados8.

Para dimensionar el asunto, el artículo describe un estudio en el que desarrollar una sola pregunta de opción múltiple de buena calidad es un trabajo riguroso, de varias horas de redacción y edición. Los costos estimados van de más de 200 dólares por pregunta utilizable hasta más de 2,000 dólares por pregunta, y los bancos de preguntas de los exámenes de licencia requieren inversiones de millones9. Ahí está la tentación de automatizar.

Así funciona el sistema que se probó, en palabras llanas: alguien escribe la indicación con el objetivo del curso y el tema de la pregunta, y ChatGPT redacta el caso completo, las opciones y las explicaciones. Después, un segundo modelo, Gemini de Google, revisa si el dato es correcto y si la respuesta marcada como buena lo es de verdad. Si algo no cuadra, se vuelve a empezar. Ningún humano corrigió el texto final: solo se podían cambiar las instrucciones10.

Lo curioso es que este ensayo se presenta como el primero de su tipo con asignación al azar. El artículo describe otros trabajos que ya habían comparado preguntas hechas por IA con preguntas hechas por profesores, pero ninguno lo había hecho repartiendo a los estudiantes al azar entre un examen y otro11.

Hay otras piezas del rompecabezas que ayudan a ver el tamaño del fenómeno. Un estudio sobre cómo las escuelas de informática de la salud manejan la IA encontró que, de 48 programas revisados, 40 tenían al menos un documento con orientación sobre el tema, pero la mayoría eran recomendaciones y no políticas formales; además, las menciones a privacidad y propiedad intelectual eran menos frecuentes que las de integridad académica, según el resumen de ese estudio, lo único disponible de él121314.

Otro trabajo, también leído solo en su resumen, pone el dedo en algo más de fondo: la cuestión no es únicamente qué genera el sistema, sino cuándo los alumnos, los maestros y las instituciones empiezan a tratar lo que produce como razón suficiente para juzgar15. Ese mismo trabajo propone distinguir quién inicia, quién decide y quién responde cuando una máquina opina sobre el desempeño de alguien16.

Aquí está la lectura de weeklyAI. Cuando una institución quiere evitar que la acusen de decidir con prejuicios, tiende a entregar la tarea a un sistema automático y a presentar el resultado como imparcial, aunque el criterio de fondo lo siga poniendo alguien. Es probable que escuelas o entidades de licencias que adopten preguntas hechas por IA presenten el examen como más neutral que el hecho por personas, sin explicar quién revisó, con qué criterio ni qué pasó con los temas donde el sistema falló. Esta lectura se equivocaría si esas instituciones publican quién supervisa la generación, cómo se corrigen los temas flojos y reconocen por escrito que la imparcialidad no está garantizada por usar IA. Cuando le digan que un examen o una selección hecha con IA es objetiva, usted puede preguntar quién revisó las preguntas, con qué criterios y qué pasó con los temas donde el sistema rindió peor.

Hay un segundo ángulo que también conviene mirar. Los sistemas automáticos que reparten cosas que la gente siente que se merece —un cupo, una licencia, una nota— suelen operar como cajas cerradas: ni quien los programó puede explicar del todo por qué dieron ese resultado. Cabe esperar que, si un examen o una admisión se genera o califica con IA, el afectado no pueda obtener una explicación clara de por qué su pregunta era difícil, por qué falló un tema o cómo se fijó el puntaje de corte. Lo contrario se vería si la institución entrega, a quien lo pida, el criterio de cada pregunta, el registro de revisiones humanas y una vía de reclamo que responda con razones concretas. Antes de aceptar un resultado, usted puede pedir por escrito el criterio usado y el canal de reclamo; si le responden con generalidades, guarde esa respuesta como prueba.

Hay una advertencia que otros trabajos citados en el artículo agregan. Es una señal sobre lo que se puede esperar en otros lugares, no sobre lo que este ensayo encontró.

Los propios autores no piden desplegar la tecnología sin más. Lo que sostienen es un modelo con una persona en el circuito: la IA genera rápido, y un experto revisa, calibra dificultad y cuida que cada pregunta corresponda al curso17. Y piden repetir el ejercicio en otras instituciones, otros niveles y otros formatos, con versiones más estables de los modelos18.

La tercera pieza tiene que ver con lo que se pierde cuando se gana tiempo. Cuando las decisiones institucionales se entregan a cálculos automáticos, la discusión previa entre personas pierde peso y el resultado técnico pasa a decidir en lugar del acuerdo entre quienes son afectados. Es de esperar que, al usar IA para armar exámenes, se consulte menos a docentes y estudiantes sobre qué debe evaluarse, y que la discusión sobre el contenido quede reemplazada por la rapidez de generar preguntas. La expectativa fallaría si el proceso conserva espacios reales de deliberación con docentes y estudiantes, y si sus objeciones cambian efectivamente las preguntas antes de aplicarse. Usted puede preguntar en su escuela o colegio profesional si hubo revisión humana documentada y si las observaciones de docentes y estudiantes modificaron el examen; y pedir que esa revisión quede por escrito.

Ese cuadernillo de 112 preguntas, con sus viñetas y sus cinco opciones, ahora tiene dos posibles autores: un grupo de estudiantes que pasó horas en cada una, o una máquina que tardó minutos con la supervisión de otra máquina. La pregunta que queda para quien se siente a presentar un examen no es si la IA puede escribirlo. Es quién responde cuando una de esas preguntas sale mal.

¿Quién revisó las preguntas de su último examen, y con qué criterio?

De dónde sale cada dato de contexto, y cuánto leímos de cada documento

  1. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "LLM-assisted MCQ development achieved a 5.6-fold efficiency gain compared with student authorship (4.2 ± 1.9 vs. 19.6 ± 7.5 min per item; p  < 0.0001)."
  2. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Student perceptions of exam acceptability—including clarity, difficulty, relevance, and educational value—were comparable between AI-generated and student-generated exams (all Bonferroni-adjusted p  ≥ 0.12; all Cohen’s |d| ≤ 0.31)."
  3. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Student-generated items demonstrated slightly higher discrimination indices than AI-generated items, though the effect size was small, and distractor efficiency did not differ between protocols."
  4. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "A majority of AI-gen items fell in the poor-discrimination category (62.5%), compared with a plurality of Student-gen items (42.9%); correspondingly, nearly half of Student-gen items reached the good-to-excellent range (47.3%) versus approximately one-third of AI-gen items (33.9%)"
  5. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Neither exam meaningfully changed students’ perceived preparedness."
  6. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "In a formative, open-resource examination setting with student-generated comparators, LLMs can substantially accelerate MCQ development while producing assessments that are psychometrically comparable and acceptable to learners. These findings may not generalize to high-stakes summative or closed-book assessment settings."
  7. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Both question-generation teams consisted of senior medical students rather than faculty experts. Although this design reflected the student-generated comparator used in the present trial, it may underestimate the psychometric advantage achievable by experienced faculty item writers."
  8. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "This trial was retrospectively registered, which may introduce a risk of outcome reporting bias. Although the study protocol, outcome measures, and statistical analysis plan were pre-specified prior to data analysis, the absence of prospective registration means this cannot be independently verified"
  9. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Developing a single high-quality multiple-choice question (MCQ) is rigorous, often requiring multiple hours of drafting and editing [ 3 – 5 ]. Consequently, cost estimates range from over US$200 (inflation-adjusted) per usable MCQ [ 3 ] to more than US$2,000 per MCQ [ 4 ], with licensure exam question banks requiring multi-million dollar investments [ 5 ]."
  10. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "AI-generated items were produced via a dual-model workflow (ChatGPT for generation; Google Gemini for validation); student-generated items were authored by senior medical students."
  11. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "However, few studies [ 8 – 12 ] have directly compared the use of LLM-generated and faculty-generated examination questions in medical education and none have used a randomized controlled trial (RCT) study design."
  12. Eldredge C, Dolezel D, Jones J, Vooppala N, Bronsburg SE. (2026). Institutional approaches to artificial intelligence policy and guidance in health informatics and information management education: emerging trends and inconsistencies. Journal of the American Medical Informatics Association. 10.1093/jamia/ocag150 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "Forty programs (83%) had at least 1 qualifying document, 8 programs had no qualifying documents."
  13. Eldredge C, Dolezel D, Jones J, Vooppala N, Bronsburg SE. (2026). Institutional approaches to artificial intelligence policy and guidance in health informatics and information management education: emerging trends and inconsistencies. Journal of the American Medical Informatics Association. 10.1093/jamia/ocag150 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "Most documents provided guidance rather than formal policies; neither policy type nor audience differed by delivery mode."
  14. Eldredge C, Dolezel D, Jones J, Vooppala N, Bronsburg SE. (2026). Institutional approaches to artificial intelligence policy and guidance in health informatics and information management education: emerging trends and inconsistencies. Journal of the American Medical Informatics Association. 10.1093/jamia/ocag150 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "References to privacy, intellectual property, and regulatory concepts were present but less common than academic integrity-related themes."
  15. Du Y. (2026). When linguistic assistance becomes authoritative judgement: a multilevel framework for generative AI in higher-education language practices. Educational Linguistics. https://doi.org/10.1515/eduling-2026-0009 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "The central issue is not only what these systems generate, but when learners, teachers, and institutions begin to treat their outputs as reasons for judging language."
  16. Du Y. (2026). When linguistic assistance becomes authoritative judgement: a multilevel framework for generative AI in higher-education language practices. Educational Linguistics. https://doi.org/10.1515/eduling-2026-0009 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "The framework differentiates the initiator, subject, decision-maker, and accountable actor, and represents the source of authorisation, the subject’s degree of meaningful choice, and the durability of system embedding as co-occurring configurational conditions rather than alternative routes."
  17. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Although small differences persist, these findings support the integration of LLM-assisted item generation within a human-in-the-loop framework, combining AI efficiency with expert oversight to preserve psychometric quality."
  18. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Future studies should apply this framework across institutions, training levels, disciplines, and assessment formats, while evaluating newer and more stable AI models as they become available."

Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0

Quién pagó: El estudio no recibió financiación externa y los autores declaran no tener conflictos de interés; el artículo no menciona ningún proveedor de equipos o programas más allá de los modelos de IA utilizados (ChatGPT-4 y Google Gemini) y la plataforma REDCap alojada en la University of British Columbia.

No tome esto como consejo médico profesional.

Los hallazgos de otros estudios que aquí se mencionan provienen de este documento; esos estudios no se consultaron uno por uno.

experiment · BMC medical education · the paper, 17 Jun 2026 · free

AI Wrote a Medical School Practice Exam in a Fifth of the Time. Students Couldn't Tell the Difference.

A Canadian university built two practice exams from the same course goals: one written by ChatGPT-4 and checked by Google Gemini, one written by senior medical students. Students rated them about the same for clarity, difficulty and usefulness, but the student-written questions told stronger students from weaker ones a little better.

Short version · the longer version follows, about 6 min

Ask the weeklyAI chat

Ask me about AI in your life: I point you to the stories weeklyAI has published on it, this week and before.

Conversations are saved for as long as weeklyAI exists, to improve the publication. Answers come in the language you write in.

The study at a glance
Who
First-year medical students at one Canadian medical school
How many
258 first-year students
Where
University of British Columbia, Vancouver, Canada
When
December 8 and 9, 2024
Kind of study
experiment
Who did it
University of British Columbia Faculty of Medicine
The limit that matters
A voluntary open-book practice test at one school; the comparison was senior students, not professional examiners.
Share of exam questions that were weak at telling stronger students from weaker ones
Questions written by AI62.5%
Questions written by senior students42.9%

Share of each exam's questions that fell in the poorest category for separating stronger from weaker students, in a voluntary open-book practice test at one school.

AI-written questions versus student-written questions

AI-written questionagainstStudent-written question

About 4.2 minutes to make, against about 19.6 minutes

AI-written questionsagainstStudent-written questions

Students rated both about the same for clarity, difficulty, relevance and usefulness

AI-written questionsagainstStudent-written questions

Student-written ones separated stronger from weaker students a little better

AI-written questionsagainstStudent-written questions

62.5 percent of AI questions were in the weakest band, against 42.9 percent of student ones

AI-written examagainstStudent-written exam

Neither changed how prepared students felt for the real exam

When you next hear that a school, a licensing board or an employer is using AI to write or mark tests, look for who reviewed the questions, and whether the exam was checked against one written by people.
weeklyAI's reading

At the University of British Columbia's medical school, two teams set out to build 112 practice questions for first-year students. One team used a chatbot. The other used senior medical students with no AI allowed.

The AI team fed ChatGPT-4 the same course goals and question guidelines the students worked from. A second chatbot, Google Gemini, then checked each finished question for correct facts and a correct answer key. If a question failed, the team changed the prompt and started again. Writers could not touch the chatbot's output. The students wrote their own questions and had a peer review them. Both exam versions were then given to 258 first-year students, on December 8 and 9, 2024. Each student sat one of the two exams, at random, without being told which was which.

The AI questions took about 4.2 minutes each to make. The student-written ones took 19.6 minutes, nearly five times as long.

Did students notice? They did not. On a 10-point scale, both exams drew similar marks for clarity, difficulty, relevance and how well the questions tested clinical understanding. Students rated the AI exam as a little harder. The gaps between the two exams were small, and none of them was statistically significant. None of the gaps was statistically significant.

One difference did hold up. Student-written questions separated the stronger students from the weaker ones a little better, and the gap was statistically significant. By common benchmarks, 62.5 percent of the AI-written questions fell in the poor-discrimination band, compared with 42.9 percent of the student-written ones. Both exams had plenty of weak questions.

This was a voluntary, open-book practice test at one school, with no effect on any grade. It cannot tell you whether AI-written questions are fair or valid for a real licensing exam. Nor did it test professional examiners: the comparison was senior students, so it does not show that AI matches expert-written tests. The trial was also registered only after it began.

The authors conclude that AI can speed up question writing if a person reviews the results. When you next hear that a school, a licensing board or an employer is using AI to write or mark tests, look for who reviewed the questions, and whether the exam was checked against one written by people.

What this means for you

When a school, a licensing board or an employer says it used AI to write a test, the question to ask is who checked the questions afterward and whether anyone compared the machine's work with a test written by people. This study shows only that one Canadian school's practice questions, written by chatbot and checked by a second chatbot, were accepted about as well as ones written by senior students, and it did not look at real licensing exams.

Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0

Who paid: The study received no external funding, and the authors declare no competing interests; the article names no equipment or software lenders beyond the AI models used (ChatGPT-4 and Google Gemini) and the REDCap platform hosted at the University of British Columbia.

Do not take this as professional medical advice.

The longer versionThe passages copied from the paper, the pictures, and every source with how much of it we read · 1135 words · about 6 minRead it →Close

AI made medical exam questions about five times faster than students did

A Canadian medical school tested whether questions made by ChatGPT measure up to questions made by senior students.

A multiple-choice exam question for medical students looks like a paragraph and a half of work: a patient with a blood pressure of 146/92, a body mass index of 32, a sedentary lifestyle, and five answer options that all sound reasonable. Behind each one sits a learning objective, a correct answer, and a written explanation for why the other four options are wrong. At the University of British Columbia's medical school in Vancouver, two teams of six senior students each wrote 112 of these questions — one team with AI, one without — and 258 first-year students sat a mock exam built from one set or the other, without being told which.

The AI team used two models in sequence. ChatGPT-4 wrote each question from a prompt built around a course objective; Google Gemini then checked the answer key and the facts, and was asked "Are you sure?" to push it harder. If Gemini disagreed, the question was regenerated. No human was allowed to edit the AI's words at any stage — writers could only change their prompts. The student team wrote their own questions and had them reviewed by a peer, with AI use forbidden and confirmed in writing.

The speed difference was the headline. Each student-written question took about 19.6 minutes; each AI-written one about 4.2 — a 5.6-fold gain, with the whole 112-question bank done in 467 minutes instead of 2,195. Asked afterward how the exam felt, students rated the two versions much the same on clarity, difficulty, relevance, quality and educational value. Neither exam changed how prepared they felt for the real thing.

The quieter difference ran the other way. Student-written questions separated strong students from weak ones slightly better, and a majority of the AI-written questions — 62.5 percent — fell into the weakest category on that measure, against 42.9 percent of the student ones. Scores were 73.4 percent on the student exam and 71.0 percent on the AI exam, a gap the study's main analysis did not find significant.

Three limits matter more than the numbers. It was an open-book practice exam at one school, so it cannot speak to a licensing exam, a course grade or a job test. The comparison was senior students, not professors who write exam questions for a living — so it does not show AI matches professional examiners. And the trial was registered only after it began, though its plan was fixed beforehand.

Consider what a single bad question costs. The article describes a study in which the price of one usable exam question is put at over US$200 in inflation-adjusted terms, with other estimates above US$2,000, and licensing bodies spending millions on question banks1. That is why the 4.2-minute figure matters: it is not a curiosity but a budget line, and a school weighing whether to buy a question bank or generate one will read it that way.

The mechanism is worth picturing, because it explains both the speed and the weakness. A person hands the model a course objective and a question stem; the model produces a patient, five options, a key and an explanation for each option; a second model, from a different company, plays examiner and is asked to doubt itself; a human decides only whether to re-prompt2. What no step in that chain does is test the question on real students and throw out the ones that fail. The student team's review did something closer to that, by having a peer attempt each item blind before judging it.

This is not the first attempt to compare machine-written and human-written medical questions. The article describes a study in which few studies have made the direct comparison with faculty-written questions, and none before this one used a randomized design3. The article describes a study in which earlier work tended to judge questions by expert opinion rather than by how students actually performed.

The topic-by-topic results are the part a school would want to read twice. On the heart murmur, upper gastrointestinal and pregnancy material, students did better on the student-written exam; on nutrient absorption, they did better on the AI-written one4. An average hides that. A department adopting this would need to check its own subject areas one at a time, not sign off on the whole bank.

The authors' own conclusion is a division of labour, not a replacement. They support AI generation inside a "human-in-the-loop" — the machine drafts, a person verifies facts, calibrates difficulty, fixes weak wrong-answer options and confirms the question matches the curriculum5. They also want the framework tried at other institutions, other training levels and other exam formats, and with newer, more stable models6.

Here is weeklyAI's reading. When a machine makes the test, the school that uses it stays answerable for a bad question or a wrong grade — the software does not sign anything. That is the ordinary rule, and it gets harder to apply when the tool was bought, adapted or reused across institutions, because then nobody can easily say who checked what. Expect a school or licensing body that adopts AI-written items to be asked, in writing, who reviewed them and who answers if one is wrong. This reading would be wrong if a regulator formally placed that duty on the model's maker instead of the school, or wrote a rule saying no person need sign off.

weeklyAI also reads this as a story about how much human work hides behind a machine's output. Most raw questions here were weak on the measure that matters — nearly two-thirds of the AI items landed in the poorest category — and the protocol that did best was the one where a person attempted each item and sent it back. Expect the real editing to stay with people, and expect the first draft to be cheap. Ask a school how many AI questions were thrown out, and whether anyone compared them against existing question banks, which matters if the test is meant to be original.

What can you do with this where you live? If a school, a licensing board or an employer tells you a test or a grade was made with AI, ask two things in writing: who reviewed the questions, and what route you have to a person who can change a result. Keep your own copy of the score and the reasons given.

Go back to that question about the woman with high blood pressure. It is well built. It is also one of the ones that survived a process of generation, cross-checking and regeneration, and the study cannot tell you how many drafts it took to get there.

Where each piece of context comes from, and how much of it we read

  1. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - the article this story is about — the whole article — the passage: "Developing a single high-quality multiple-choice question (MCQ) is rigorous, often requiring multiple hours of drafting and editing [ 3 – 5 ]. Consequently, cost estimates range from over US$200 (inflation-adjusted) per usable MCQ [ 3 ] to more than US$2,000 per MCQ [ 4 ], with licensure exam question banks requiring multi-million dollar investments [ 5 ]."
  2. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - the article this story is about — the whole article — the passage: "AI-generated items were produced via a dual-model workflow (ChatGPT for generation; Google Gemini for validation); student-generated items were authored by senior medical students."
  3. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - the article this story is about — the whole article — the passage: "However, few studies [ 8 – 12 ] have directly compared the use of LLM-generated and faculty-generated examination questions in medical education and none have used a randomized controlled trial (RCT) study design."
  4. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - the article this story is about — the whole article — the passage: "Student performance was significantly better in the Student-gen exam in the heart murmur, upper gastrointestinal tract, and pregnancy curricular themes, while student performance was significantly better in the AI-gen exam in the nutrient absorption theme."
  5. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - the article this story is about — the whole article — the passage: "Although small differences persist, these findings support the integration of LLM-assisted item generation within a human-in-the-loop framework, combining AI efficiency with expert oversight to preserve psychometric quality."
  6. Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0 - the article this story is about — the whole article — the passage: "Future studies should apply this framework across institutions, training levels, disciplines, and assessment formats, while evaluating newer and more stable AI models as they become available."

Al-Najafi, D., Krause, K. D., Wang, Y. et al. (2026). Psychometric performance and student perceptions of AI- versus student-generated multiple-choice questions: a single-center randomized controlled trial. BMC Medical Education. https://doi.org/10.1186/s12909-026-09671-0

Who paid: The study received no external funding, and the authors declare no competing interests; the article names no equipment or software lenders beyond the AI models used (ChatGPT-4 and Google Gemini) and the REDCap platform hosted at the University of British Columbia.

Do not take this as professional medical advice.

The findings of other studies mentioned here come from this document; those studies were not opened one by one.

The free weekly drip by email

One email a week: the free story of every section, with its picture, and the headlines of the rest.

No payment, nothing to cancel. Unsubscribe from any issue.

El goteo semanal gratis por correo

Un correo por semana: la nota gratuita de cada sección, con su imagen, y los titulares de las demás.

Sin pago y sin nada que cancelar. Puede darse de baja desde cualquier envío.

From the house

About

Every week the new research on AI is read here, and this page tells you what was found, how far it can be trusted, and what you can do with it. This page is free, and every line on it says what was found.

The studies are read by a machine built and ruled by a professor of communication at Sacramento State and Universidad Nacional Autónoma de México (UNAM). Who makes this →

Diego Bonilla, Ph.D.

Full Professor of Communication Studies, Sacramento State, California, United States
Graduate instructor, Universidad Nacional Autónoma de México (UNAM), Mexico

hypergraphia.com

I am a professor of communication at Sacramento State, where I created the course Information Management and Privacy and have taught it many times, and at UNAM, in Mexico City, I teach the graduate course Electronic and Generative Literature with AI. In both I teach AI. I have taught since 1989, at first mathematics and physics to children who found them hard, and since 1997 one subject that has changed its name many times: media literacy, computer literacy, digital literacy, information literacy, ICT literacy, and now AI literacy.

This page exists because most people cannot go back to a university, and these systems are changing their work, their children's schooling, their parents' care and their vote all the same. What a university teaches about them should reach the people who need it, every week, in plain language, without enrolling anywhere. That is what I do here, and it is what I do on Thursday mornings in a seminar for the Renaissance Society at Sacramento State, built on this publication, where anyone can send a question and hear it answered.

In 2003 my doctorate at Syracuse measured what a machine can infer about a person from the traces they leave, and what I found became the privacy course I still teach. In 2007 and 2009 I sat on the committee at ETS in Princeton that set the national standard for digital literacy, and from 2014 to 2016 I was the California State University's ICT Literacy Ambassador. Six names, one job: helping people not be at the mercy of the newest machine.

I have also worked on the other side of the counter. In 2000 I began making films that a machine assembles differently for every viewer; the first of them took first prize at the Moscow International Film Festival in 2004, two decades before the industry shipped a generative model. A generative video poem I made with a poet at UNAM is in the field's definitive anthology, and in 2025 my AI short film The Human Futures Market took Best AI Short Film at the Red Rocks AI Film Festival. My recent published research is on AI and elections, and on how these models represent the cultures they were not trained on. I do not read the papers myself, and this page will never tell you I do. I built the machine that reads them and wrote the rules it reads by, and I analyze what it gives me the way I have taught my students to analyze: asking how it knows, and who benefits from what it says.

The vendors began teaching AI literacy two years ago, because before that there was nothing to sell. I am not against them. I am simply not one of them, and no one who sells these systems pays for this page. Your subscription does, and that is what keeps it honest.

The record, dated
  • 1989 Teaching begins, Mexico City
  • 1995 Hypergraphia, an early Spanish-language web and hypermedia publisher
  • 1996 Mercadotecnia e imagen en Internet, with Jesús del Toro, a book on marketing on the Internet, among the first in Spanish, Editorial Iberoamérica
  • 1999 Autorretrato, first generative poem, published by UNAM's Instituto de Investigaciones Filológicas
  • 2003 Ph.D., Syracuse University, Doctoral Prize, for the dissertation The Medium Is the Measure of Itself
  • 2004 A Space of Time: First Prize, Internet/Multimedia, XXVI Moscow International Film Festival
  • 2007, 2009 ETS iSkills committee, Princeton
  • 2008 Making Sense of Tracking Data, a book on how the traces a person leaves online allow inferences about them, including how they think, VDM Verlag
  • 2011 Keynote on generative work, UNAM
  • 2014–2016 ICT Literacy Ambassador, California State University · California Open Educational Resources Council
  • 2019 Big Data, with Rodolfo Mata: Electronic Literature Collection, Volume 4
  • 2020 Tablada Hipertextual, with Rodolfo Mata, UNAM: the computer reads a poet's words and builds a different reading from them, the ground the language models now work on
  • 2021 Uku Pacha, a generative VR film: world premiere, Vancouver International Film Festival
  • 2023 Keynote on the representation of Latin American cultures in AI, Ibero-American Meeting of Audiovisual Narratives, Trujillo, Peru · Graduate teaching at UNAM begins
  • 2023–2026 Directed the faculty learning community Advancing AI Literacy, Sacramento State · Principles for Cognitive and Pedagogical Sovereignty
  • 2024 A weekly AI publication run on real pipelines, every week since
  • 2025 Best AI Short Film, Red Rocks AI Film Festival
  • 2026 Keynote, with Nancy Huante-Tzintzun, Latin American and Caribbean Digital Humanities symposium, UT San Antonio · AI Watch, a weekly seminar for the Renaissance Society at Sacramento State

The research record →

TertulIA: weeklyAI.watch/tertulia/en/Pictures are illustrations generated by weeklyAI.watch, never photographs