A Chatbot Answered Their Questions About Joint Disease. 92.9% of the Answers People Rated Got a Thumbs-Up.

other · Journal of medical systems · la publicación, 5 oct 2026 · gratis

Nueve de cada diez respuestas de unos chatbots médicos alemanes gustaron a quienes las calificaron

Eran diez programas sobre enfermedades reumáticas, hechos en Alemania y basados en guías clínicas alemanas. La cifra viene sólo de las respuestas que la gente eligió calificar, y el estudio no midió si alguien mejoró.

Versión breve · la versión detallada sigue, unos 5 min

Pregunte al chat de weeklyAI

Pregúnteme por este estudio: a quiénes se estudió, qué encontró y qué no dice.

Las conversaciones se guardan mientras exista weeklyAI, para mejorar la publicación. Se responde en el idioma en que usted escribe.

El estudio, de un vistazo
Quiénes: Pacientes reumáticos que usaron diez programas de conversación con inteligencia artificial
Cuántos: 6291 preguntas registradas
Dónde: Alemania
Cuándo: Entre septiembre de 2025 y enero de 2026
Tipo de estudio: analysis of what people did
Quién lo hizo: Universidades y centros de reumatología de Alemania, con organizaciones de pacientes
El límite que importa: Solo el 42.5% de las respuestas fue calificado y nadie midió si la gente mejoró

Lo que dijeron las 602 personas que respondieron el cuestionario opcional

Dijo que el programa era fácil de usar84.6%

Dijo que las respuestas se entendían bien84.1%

Dijo que era un buen complemento para la educación del paciente80.2%

Porcentajes de las 602 personas que completaron el cuestionario opcional, no de todos los que usaron los programas.

La satisfacción del usuario es una cosa. La calidad clínica verificada es otra. Y el resultado de salud —si la persona aprendió, si siguió su tratamiento, si se sintió mejor— es una tercera.
Lectura de weeklyAI

Así podría verse · ilustración generada por weeklyAI.watch, no es una fotografía

Entre septiembre de 2025 y enero de 2026, diez programas de conversación por computadora respondieron 6,291 preguntas de pacientes reumáticos en Alemania. De esas respuestas, 2,671 fueron calificadas por quienes las usaron. Un total de 2,481 recibió una marca positiva: el 92.9%, casi nueve de cada diez.

En el proyecto participaron reumatólogos, organizaciones de pacientes, investigadores y expertos técnicos. Cada uno respondía sobre una enfermedad distinta y sólo podía apoyarse en la guía clínica alemana más reciente de esa enfermedad. Se usaron en 13 centros de reumatología y en seis organizaciones de pacientes.

Las preguntas más frecuentes fueron sobre la enfermedad misma (50.2%), sobre medicamentos y controles (37.3%) y sobre diagnósticos (28.2%). Cada conversación duró en promedio 4.1 intercambios.

Ahora, el límite. Sólo el 42.5% de todas las respuestas recibió una calificación. El cuestionario final era opcional y lo completaron 602 personas. La cifra de 92.9% viene, entonces, de quienes decidieron responder, no de todos los que usaron los programas.

Y hay algo que el estudio no midió: si las personas aprendieron más, si siguieron mejor su tratamiento o si se sintieron mejor. Nadie comparó a quienes usaron el chatbot con quienes no lo usaron.

Otro programa de inteligencia artificial evaluó las respuestas, y ahí el 95.3% salió completamente seguro. Ese programa marcó 11 respuestas como posiblemente inseguras. Al revisarlas, dos reumatólogos calificaron una de ellas como posiblemente insegura. En 263 casos el programa dijo que no podía responder.

Todo esto ocurrió en Alemania, en alemán y con guías alemanas.

Lo que puede buscar: cuando una clínica, una organización de pacientes o un sistema de salud de su país anuncie algo parecido, pregunte cuántas personas lo usaron de verdad y cuántas calificaron las respuestas.

Qué significa para usted

Cuando su país anuncie algo parecido, pregunte cuántas personas lo usaron de verdad y cuántas calificaron las respuestas, porque aquí sólo calificó el 42.5%. Y tenga presente que el estudio no midió si alguien aprendió más o siguió mejor su tratamiento, así que nada garantiza que usted mejorará. Eso no lo decide un programa: consúltelo con un profesional.

Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6

Quién pagó: El estudio fue financiado por una subvención de la Deutsche Rheumastiftung otorgada a J.K., y J.K. y H.M. recibieron además apoyo parcial de la subvención Horizon Europe (n.º 101080711) del proyecto SPIDERR; el artículo no indica que los financiadores tuvieran algún papel en el diseño, el análisis o la redacción, y el único vínculo comercial declarado es que A.R. trabaja en Zentrum für Telemedizin Bad Kissingen, cuya plataforma Buddy alojó los chatbots.

No tome esto como consejo médico profesional.

Versión detalladaLos pasajes copiados del artículo, las ilustraciones y cada fuente con cuánto leímos de ella · 940 palabras · unos 5 minLeerla →Cerrar

Inteligencia artificial para enfermos reumáticos: qué promete y qué no

Un estudio alemán puso diez chatbots a responder preguntas reales de pacientes. La mayoría de las respuestas gustó. Pero nadie comprobó todavía si la gente aprendió más o se sintió mejor.

Diez programas de conversación con inteligencia artificial respondieron 6,291 preguntas de pacientes reumáticos en Alemania entre septiembre de 2025 y enero de 2026¹. Eran chatbots construidos sobre las guías clínicas alemanas de cada enfermedad. Junto a las respuestas se mostraban identificadores de las fuentes.

De esas 6,291 respuestas, los usuarios calificaron 2,671. De las calificadas, 2,481 —el 92.9%— recibieron una valoración positiva². Entre los 602 usuarios que respondieron un cuestionario opcional, el 84.6% dijo que el chatbot era fácil de usar, el 84.1% que las respuestas se entendían bien y el 80.2% que era un complemento útil para la educación del paciente³.

Una evaluación automática, hecha por otro sistema de inteligencia artificial, consideró que el 95.3% de las respuestas eran completamente seguras y el 79.1% completamente correctas⁴. Pero solo el 45.0% se ajustaba por completo a la guía, y la coincidencia entre esa evaluación automática y la de los médicos fue débil⁵.

En 263 interacciones —el 4.2%— el chatbot dijo que no podía responder⁶. Al revisarlas, los médicos encontraron que 45 de esas 263 preguntas sí tenían respuesta en la guía correspondiente. En otros casos el chatbot respondió con su conocimiento general, fuera de la guía. Y no había ningún mecanismo de vigilancia clínica en tiempo real⁷. Los autores piden más estudios para medir si esto enseña de verdad y para validar la calidad y la seguridad de forma independiente⁸.

Las enfermedades reumáticas son crónicas y exigen manejo de por vida. Quien las tiene debe navegar continuamente preguntas sobre síntomas, tratamientos, efectos secundarios, adaptaciones del estilo de vida y monitoreo de la enfermedad⁹. Muchas de esas preguntas quedan sin responder en la consulta, porque el tiempo de atención es limitado¹⁰. Y no es solo al principio: incluso pacientes con años de enfermedad pueden saber poco sobre cómo se monitorea su condición o qué opciones de tratamiento existen¹¹. Esa es la necesidad que estos programas intentan cubrir.

La tecnología se llama generación aumentada por recuperación. En palabras simples: en vez de dejar que el modelo conteste con lo que "recuerda" de su entrenamiento, el sistema busca primero en documentos definidos y arma la respuesta con esos pasajes¹². En este caso, la base de conocimiento de cada chatbot se limitó a la guía clínica alemana más reciente y vigente para cada enfermedad reumática¹³. Además, se le indicó al modelo que respondiera en lenguaje sencillo, sin terminología médica compleja ni abreviaturas sin explicar¹⁴. Esa es la promesa: respuestas ancladas en una fuente verificable, no en lo que el modelo haya absorbido de internet.

Ya se había intentado algo parecido. LupusGPT fue un chatbot multilingüe construido por pacientes para el lupus eritematoso sistémico, con la misma técnica de recuperación de fuentes¹⁵. Pero se limitaba a una sola enfermedad reumática y no reportó datos sistemáticos de aceptación por parte de los pacientes ni una evaluación estructurada de calidad y seguridad de las respuestas¹⁶. Este estudio alemán intenta cubrir justamente esos huecos: diez enfermedades, uso real y evaluación de calidad.

Vale la pena comparar dos cosas que sí se pueden comparar. Por un lado, este estudio alemán: observación, sin grupo de comparación, en consultorios y organizaciones de pacientes. Es decir: cuando se compara contra otra cosa, el chatbot puede mostrar una diferencia. Este estudio alemán no hizo esa comparación para los resultados de los pacientes. Solo midió satisfacción y evaluó respuestas.

Otra comparación posible es entre lo que un chatbot puede hacer y lo que hace una persona. Eso no invalida lo que encontró el estudio alemán. Lo enmarca: la satisfacción del usuario no es lo mismo que la calidad clínica medida por expertos.

También hay que mirar qué pasa cuando se compara un chatbot con materiales de salud bien hechos. Es la misma familia de preguntas: ¿esto funciona mejor que lo que ya existe, o solo funciona?

Aquí es donde conviene detenerse. La satisfacción del usuario es una cosa. La calidad clínica verificada es otra. Y el resultado de salud —si la persona aprendió, si siguió su tratamiento, si se sintió mejor— es una tercera. Este estudio midió bien la primera, midió parcialmente la segunda y no midió la tercera. Eso no es un defecto del estudio: es lo que se propuso hacer. Pero conviene tenerlo claro antes de sacar conclusiones.

Lo que sí sugiere este trabajo es una dirección: cuando un sistema de salud ofrece información gratuita, disponible a toda hora y anclada en una fuente oficial, la gente la usa. El mayor uso se da durante el día, con actividad importante que se extiende hacia la noche, y los fines de semana baja. Eso es un dato sobre la necesidad, no solo sobre la tecnología.

Lo que el lector puede hacer con esto es concreto. Si usted o alguien en su familia vive con una enfermedad reumática y no consigue cita pronto con un especialista, un chatbot de este tipo —cuando exista en su idioma y con fuentes verificables— podría servirle para preparar preguntas antes de la consulta. Anote lo que le responda. Llévelo al médico. Pregúntele de dónde sacó cada cosa. Y si le dice que no sabe, eso también es información útil.

La pregunta que vale la pena llevarse no es si la inteligencia artificial puede responder preguntas de salud. Ya lo hace. La pregunta es quién verifica esas respuestas, con qué frecuencia se actualizan las fuentes y qué pasa cuando el sistema se equivoca. Eso todavía está por resolverse.

De dónde sale cada dato de contexto, y cuánto leímos de cada documento

Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Between September 2025 and January 2026, 6291 questions were recorded."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Of 2671 responses rated by users, 2481 (92.9%) received a positive rating."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Among 602 questionnaire respondents, 84.6% reported that the chatbot was easy to use, 84.1% that answers were easy to understand, and 80.2% that it was a useful addition to patient education."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "In the LLM-based evaluation, 95.3% of answers were rated as completely safe and 79.1% as completely correct."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Guideline adherence was assessed separately, with 45.0% rated as fully adherent; agreement with physician assessment was weak."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "The chatbots were unable to answer in 263 interactions (4.2%)."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "No real-time clinical safety monitoring or escalation pathway was implemented."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Further studies are needed to evaluate educational effectiveness and independently validate response quality and clinical safety."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Rheumatic diseases are chronic conditions that require lifelong management. Patients must continuously navigate questions related to symptoms, treatment options, side effects, lifestyle adaptations, and disease monitoring."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Yet many of these questions remain insufficiently addressed in routine care, which is often constrained by limited consultation time"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Even patients with longstanding disease may have limited knowledge about disease monitoring and treatment options"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "Retrieval-augmented generation (RAG) can improve reliability by anchoring chatbot responses in predefined source documents rather than relying solely on the model's internal knowledge"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "For each chatbot, the knowledge base was limited to the latest available valid German clinical guideline for the respective rheumatic disease"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "The system prompt instructed the model to answer patient questions about rheumatic diseases in plain language, avoid complex medical terminology and unexplained abbreviations"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "The recently reported LupusGPT initiative provides an important patient-led example of a retrieval-augmented, multilingual chatbot for systemic lupus erythematosus"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - el artículo del que trata esta nota — el artículo completo — el pasaje: "However, LupusGPT was limited to a single rheumatic disease and did not report systematic patient acceptance data or structured response quality and safety assessment."
Kapoor D, Achalli S, Reddy N, Mulla M, Morsy MSM, Mulla M, et al. (2026). Artificial intelligence and its impact on oral health care education. A randomized study. Journal of Education and Health Promotion. 10.4103/jehp.jehp_1971_25 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "A cluster-randomized controlled trial with 1000 participants from 20 villages (10 interventions, 10 controls) was conducted."
Kapoor D, Achalli S, Reddy N, Mulla M, Morsy MSM, Mulla M, et al. (2026). Artificial intelligence and its impact on oral health care education. A randomized study. Journal of Education and Health Promotion. 10.4103/jehp.jehp_1971_25 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "The intervention group showed a marked rise in mean knowledge scores (9.8 to 15.2) compared to controls (10.1 to 11.3; P P"
Wang Y, Ji M, Bai X, Zhuang Y. (2026). Evaluating the Accuracy, Empathy, and Readability of Generative AI Versus Registered Nurses in Discharge Planning: A Vignette‐Based Study. Nursing Open. 10.1002/nop2.70777 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "AI outperformed nurses in information comprehensiveness, but experts identified safety risks in AI-generated texts, whereas no such issues were found in nurse-produced texts."
Wang Y, Ji M, Bai X, Zhuang Y. (2026). Evaluating the Accuracy, Empathy, and Readability of Generative AI Versus Registered Nurses in Discharge Planning: A Vignette‐Based Study. Nursing Open. 10.1002/nop2.70777 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "Nurses significantly outperformed AI in both empathy and readability, and objective NLP analysis confirmed that AI-generated texts exhibited higher syntactic complexity and terminology density."
Wang Y, Ji M, Bai X, Zhuang Y. (2026). Evaluating the Accuracy, Empathy, and Readability of Generative AI Versus Registered Nurses in Discharge Planning: A Vignette‐Based Study. Nursing Open. 10.1002/nop2.70777 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "AI may be better positioned as an assistive tool for nurses rather than an independent communication tool, and its deployment should address potential digital divide issues among patient populations."
Sehgal NKR, Rai S, Tonneau M, Agarwal AK, Cappella J, Kornides ML, et al. (2026). Large Language Model Chatbot Conversations vs Public Health Materials and Parental HPV Vaccination Intentions. JAMA Network Open. 10.1001/jamanetworkopen.2026.16822 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "a 3-minute interaction with an LLM chatbot (OpenAI's GPT-4o) prompted to encourage HPV vaccination using a default response style or a shorter conversational style"
Sehgal NKR, Rai S, Tonneau M, Agarwal AK, Cappella J, Kornides ML, et al. (2026). Large Language Model Chatbot Conversations vs Public Health Materials and Parental HPV Vaccination Intentions. JAMA Network Open. 10.1001/jamanetworkopen.2026.16822 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "This randomized clinical trial was conducted online among individuals in the US, Canada, and the UK from March 3 to May 25, 2025, with follow-up at 15 and 45 days."
Sehgal NKR, Rai S, Tonneau M, Agarwal AK, Cappella J, Kornides ML, et al. (2026). Large Language Model Chatbot Conversations vs Public Health Materials and Parental HPV Vaccination Intentions. JAMA Network Open. 10.1001/jamanetworkopen.2026.16822 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "Compared with no intervention, public health materials increased immediate vaccination intent (Cohen d = 0.53; 95% CI, 0.36-0.70), as did the default chatbot (d = 0.48; 95% CI, 0.30-0.65) and conversational chatbot (d = 0.33; 95% CI, 0.17-0.49)."
Sehgal NKR, Rai S, Tonneau M, Agarwal AK, Cappella J, Kornides ML, et al. (2026). Large Language Model Chatbot Conversations vs Public Health Materials and Parental HPV Vaccination Intentions. JAMA Network Open. 10.1001/jamanetworkopen.2026.16822 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "At 45 days, neither chatbot increased intent relative to controls, whereas public health materials maintained modest effects. No intervention increased self-reported vaccination uptake."
Sehgal NKR, Rai S, Tonneau M, Agarwal AK, Cappella J, Kornides ML, et al. (2026). Large Language Model Chatbot Conversations vs Public Health Materials and Parental HPV Vaccination Intentions. JAMA Network Open. 10.1001/jamanetworkopen.2026.16822 — solo el resumen - no se pudo obtener el texto completo — el pasaje: "The findings of this randomized clinical trial suggest that well-designed public health messaging may match or exceed the impact of short chatbot conversations for HPV vaccine promotion."

No tome esto como consejo médico profesional.

other · Journal of medical systems · the paper, 5 Oct 2026 · free

A Chatbot Answered Their Questions About Joint Disease. 92.9% of the Answers People Rated Got a Thumbs-Up.

Germans with rheumatic diseases rated 2,481 of 2,671 answers as good. But only 42.5% of all answers were rated at all, and the study watched satisfaction, not health.

Short version · the longer version follows, about 7 min

Ask the weeklyAI chat

Ask me about this study: who was studied, what it found, and what it does not say.

Conversations are saved for as long as weeklyAI exists, to improve the publication. Answers come in the language you write in.

The study at a glance
Who: People with rheumatic diseases in Germany
How many: 6291 questions
Where: Germany
When: Between September 2025 and January 2026
Kind of study: analysis of what people did
Who did it: Justus Liebig University Giessen and Philipps-Universität Marburg
The limit that matters: Only 42.5% of answers were rated, and patient outcomes were not analysed.

What users said about the chatbot

said it was easy to use84.6%

said answers were easy to understand84.1%

said it was a useful addition to patient education80.2%

These are the shares of 602 questionnaire respondents who agreed with each statement; the questionnaire was optional, so the people who answered may not represent all users.

If a clinic or patient group near you announces a tool like this, look for one thing: whether it says how many people used it and rated it, and how those answers were checked.
weeklyAI's reading

How it could look · illustration generated by weeklyAI.watch, not a photograph

In Germany, between September 2025 and January 2026, ten chatbots built to answer questions about rheumatic diseases took in 6,291 questions.

Patients asked most often about their disease itself, half of all questions. A little over a third were about medication and monitoring. More than a quarter were about tests and diagnosis. The average conversation ran to about four questions and answers.

Of the 2,671 answers that users chose to rate, 2,481 came back positive. That is 92.9%.

Rheumatologists, patient organisations and technical experts built the chatbots together. Each was tied to the German medical guideline for one disease, so the answers were meant to come from those documents, not from the model's general knowledge. Patients reached the chatbots through 13 rheumatology centres and six patient organisations.

Then the wider picture. Fewer than half of all answers were rated at all. The questionnaire was optional too, and 602 people filled it in. So the 92.9% comes from the people who chose to respond, and they were likely the happier ones.

Among those 602, 84.6% said the chatbot was easy to use. 84.1% said the answers were easy to understand. 80.2% called it a useful addition to patient education. Fewer, 67.3%, found the answers trustworthy.

Those are opinions about the chatbot. They are not health results. No one checked whether users learned more about their disease, kept to their treatment or felt better. The study did not measure that.

The safety picture came mostly from a computer program that graded the answers. Rheumatologists helped guide and check that program. It scored 95.3% of answers as completely safe and 79.1% as completely correct. Only 45.0% were judged to follow the guideline fully.

Two rheumatologists independently reviewed 100 question-and-answer pairs, a small sample of the thousands. The computer and the doctors agreed closely on safety, but less on other measures. In the end, doctors confirmed just one answer as possibly unsafe.

The chatbots could not answer 4.2% of questions. The tool was bound to German guidelines. Nothing here was tested in the Americas.

If a clinic or patient group near you announces a tool like this, look for one thing: whether it says how many people used it and rated it, and how those answers were checked.

What this means for you

If a clinic or patient group where you live ever announces a tool like this, ask what it is built on and whether anyone has checked the answers, since this one was tested only in Germany, with German guidelines, and nobody measured whether patients actually learned more or felt better.

Who paid: The study was supported by a grant from the Deutsche Rheumastiftung awarded to J.K., with J.K. and H.M. also supported in part by the Horizon Europe grant (No. 101080711) for the SPIDERR project; the article does not state that funders had any role in the design, analysis or reporting, and the only declared commercial tie is that A.R. is employed by Zentrum für Telemedizin Bad Kissingen, whose Buddy platform hosted the chatbots.

Do not take this as professional medical advice.

The longer versionThe passages copied from the paper, the pictures, and every source with how much of it we read · 1364 words · about 7 minRead it →Close

Ten guideline-grounded chatbots were deployed across a nationwide German rheumatology network, and users rated them warmly.

Germany's rheumatology patients got access to chatbots built on official guidelines. The ratings were warm. Patient outcomes were not analysed.

People with rheumatic diseases — conditions such as rheumatoid arthritis and lupus that last a lifetime and must be managed for years — carry a running list of questions about symptoms, medicines, side effects, daily life and how their disease is tracked¹. Those questions often go unanswered in routine care, where consultation time runs short, and even people who have lived with a disease for decades may not know how their condition is monitored or what treatments exist²³.

Into that gap went ten chatbots, one for each rheumatic disease, built on a commercial platform and using OpenAI's GPT-4o model, with answers anchored to German clinical guidelines rather than the model's own memory. Between September 2025 and January 2026 they were offered through 13 rheumatology centres and six patient organisations, and they recorded 6,291 question-and-answer interactions⁴.

Of the 2,671 answers that users chose to rate, 2,481 — 92.9 percent — got a positive rating⁵. Among the 602 people who filled in the optional questionnaire, 84.6 percent said the chatbot was easy to use, 84.1 percent that its answers were easy to understand, and 80.2 percent that it was a useful addition to patient education⁶.

A separate automated assessment, in which a language model judged all 6,291 answers, rated 95.3 percent as completely safe and 79.1 percent as completely correct⁷. That is not the same as a doctor clearing every answer: two rheumatologists independently rated 100 randomly selected question-and-answer pairs, and the automated ratings were produced only after that review.

The limits are as important as the numbers. Only 42.5 percent of answers were rated at all, and the questionnaire was voluntary, so the warm figures come from people who chose to respond. The chatbots could not answer 263 questions — 4.2 percent of the total⁸. Guideline adherence was rated separately: 45.0 percent fully adherent, with weak agreement between the automated judge and the physician assessment⁹. No real-time clinical safety monitoring or escalation pathway was implemented¹⁰. Patient outcomes were not analysed at all — nobody checked whether users learned more, stuck to treatment, or got better.

The mechanism is worth understanding, because it is what separates this tool from the general-purpose chatbots people already use. The article describes a study in which retrieval-augmented generation anchors a chatbot's answers in predefined source documents instead of relying on what the model absorbed during training¹¹. Here the source documents were the latest valid German clinical guideline for each disease, and nothing else¹². When a patient asked a question, the system searched those guidelines, pulled out relevant passages, and instructed the model to answer in plain language, avoiding complex medical terminology and unexplained abbreviations¹³. The model was not fine-tuned; its behaviour came entirely from the retrieved passages and the instructions.

That design buys transparency and pays for it in coverage. Because the knowledge base held only guidelines, the chatbot sometimes gave answers that were correct when the guideline was written but are no longer current — for example, stating that no JAK inhibitors are approved for ankylosing spondylitis, which reflects the guideline text rather than today's approvals. The study's authors describe this as a fundamental tension: answers that reach beyond the curated documents undermine source transparency, while strictly limiting output to a single guideline can leave real patient questions unanswered and erode trust.

This is not the first attempt of its kind. The article describes a study in which a patient-led initiative called LupusGPT showed that a retrieval-augmented, multilingual chatbot for a single rheumatic disease could be used internationally¹⁴ — The article describes a study in which it covered only lupus and did not report systematic patient acceptance data or a structured assessment of response quality and safety¹⁵. The German study's contribution is breadth: ten diseases, a nationwide network, and an attempt to measure quality rather than just count users.

Other work in this family of research gives a sobering sense of how hard durable effects are to show. In a cluster-randomised trial of 1,000 participants across 20 villages, an intervention group received four weeks of chatbot-guided oral health education by WhatsApp and interactive voice response, while controls received standard health talks and leaflets¹⁶¹⁷; the intervention group's mean knowledge scores rose markedly compared with controls¹⁸, and the authors concluded that AI-based chatbots can enhance awareness, hygiene practices and clinical outcomes in rural settings¹⁹ — though we could read only the summary of that study. In a separate randomised trial among parents in the US, Canada and the UK, a chatbot prompted to encourage HPV vaccination produced a measurable immediate rise in vaccination intent — but at 45 days neither chatbot had increased intent relative to controls, and no intervention increased actual vaccination uptake²⁰²¹²²; we could read only the summary of that study as well²³.

Another evaluation compared AI-written and nurse-written patient information and found that AI was more comprehensive but that experts identified safety risks in the AI texts and none in the nurse-produced ones, while nurses clearly outperformed the AI on empathy and readability²⁴²⁵; older patients and those with less education were less accepting of the AI texts²⁶, and the authors argued that AI is better positioned as an assistive tool for nurses than an independent communicator, with nurse verification required for all AI-generated discharge content²⁷²⁸ — again, we could read only the summary.

Here is how we read it. A chatbot that answers in your own words, at eleven at night, about your own disease, is doing something a leaflet cannot: it is attending to you. That feeling of being attended to is real, and it is also the thing most likely to be mistaken for medical quality. If you try a health chatbot, notice whether you feel reassured by its tone. Then ask yourself whether the same answer on paper would still satisfy you. If you can, check the load-bearing facts — doses, approvals, when to seek help — with a clinician or a trusted guideline. That is not distrust; it is using the tool where it is strong and covering where it is not.

We also read a second pattern worth naming. When a system is offered as official — built on national guidelines, handed out by your own rheumatology centre — it arrives with authority attached. People tend to accept what an authoritative source says unless the setting deliberately invites them to push back. So watch what happens in your own home: if the chatbot says it cannot answer, or gives a vague reply, that is a gap in the tool, not a verdict on your question. Write the question down. Bring it to your next appointment, or to a patient organisation that can help you sort it out. And if the chatbot's answer conflicts with what your own doctor told you, say so out loud — the conflict is information, and your clinician is the one who can see your case.

What would have to happen next is clear from the study itself: further work is needed to evaluate educational effectiveness and to independently validate response quality and clinical safety²⁹. That means measuring whether patients actually learn, whether they follow treatment, whether anything about their health changes — none of which this study did. Until that exists, a warm rating is a warm rating, and it is not a health outcome.

For a reader in Latin America, the United States or Canada, this exact tool is not something you can open today: it was German-language, built on German guidelines, and deployed only in Germany. What travels is the recipe — a chatbot tied to a named, citable source, with its refusals and its gaps visible — and the questions. Ask any health chatbot you encounter what it is reading from. Ask whether it will show you the source. Ask who checked its answers, and how many of them. Those three questions separate a tool someone measured from a tool someone merely launched.

What would you ask a chatbot that claims to be built on your own country's guidelines?

Where each piece of context comes from, and how much of it we read

Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "Rheumatic diseases are chronic conditions that require lifelong management. Patients must continuously navigate questions related to symptoms, treatment options, side effects, lifestyle adaptations, and disease monitoring."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "Yet many of these questions remain insufficiently addressed in routine care, which is often constrained by limited consultation time"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "Even patients with longstanding disease may have limited knowledge about disease monitoring and treatment options"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "Between September 2025 and January 2026, 6291 questions were recorded."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "Of 2671 responses rated by users, 2481 (92.9%) received a positive rating."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "Among 602 questionnaire respondents, 84.6% reported that the chatbot was easy to use, 84.1% that answers were easy to understand, and 80.2% that it was a useful addition to patient education."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "In the LLM-based evaluation, 95.3% of answers were rated as completely safe and 79.1% as completely correct."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "The chatbots were unable to answer in 263 interactions (4.2%)."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "Guideline adherence was assessed separately, with 45.0% rated as fully adherent; agreement with physician assessment was weak."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "No real-time clinical safety monitoring or escalation pathway was implemented."
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "Retrieval-augmented generation (RAG) can improve reliability by anchoring chatbot responses in predefined source documents rather than relying solely on the model's internal knowledge"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "For each chatbot, the knowledge base was limited to the latest available valid German clinical guideline for the respective rheumatic disease"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "The system prompt instructed the model to answer patient questions about rheumatic diseases in plain language, avoid complex medical terminology and unexplained abbreviations"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "The recently reported LupusGPT initiative provides an important patient-led example of a retrieval-augmented, multilingual chatbot for systemic lupus erythematosus"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "However, LupusGPT was limited to a single rheumatic disease and did not report systematic patient acceptance data or structured response quality and safety assessment."
Kapoor D, Achalli S, Reddy N, Mulla M, Morsy MSM, Mulla M, et al. (2026). Artificial intelligence and its impact on oral health care education. A randomized study. Journal of Education and Health Promotion. 10.4103/jehp.jehp_1971_25 — only the abstract - the full text could not be fetched — the passage: "A cluster-randomized controlled trial with 1000 participants from 20 villages (10 interventions, 10 controls) was conducted."
Kapoor D, Achalli S, Reddy N, Mulla M, Morsy MSM, Mulla M, et al. (2026). Artificial intelligence and its impact on oral health care education. A randomized study. Journal of Education and Health Promotion. 10.4103/jehp.jehp_1971_25 — only the abstract - the full text could not be fetched — the passage: "The intervention group received 4 weeks of chatbot-guided oral health education via WhatsApp/IVR, while controls received standard health talks and leaflets."
Kapoor D, Achalli S, Reddy N, Mulla M, Morsy MSM, Mulla M, et al. (2026). Artificial intelligence and its impact on oral health care education. A randomized study. Journal of Education and Health Promotion. 10.4103/jehp.jehp_1971_25 — only the abstract - the full text could not be fetched — the passage: "The intervention group showed a marked rise in mean knowledge scores (9.8 to 15.2) compared to controls (10.1 to 11.3; P P"
Kapoor D, Achalli S, Reddy N, Mulla M, Morsy MSM, Mulla M, et al. (2026). Artificial intelligence and its impact on oral health care education. A randomized study. Journal of Education and Health Promotion. 10.4103/jehp.jehp_1971_25 — only the abstract - the full text could not be fetched — the passage: "AI-based chatbots significantly enhance oral health awareness, hygiene practices, and clinical outcomes in rural settings, highlighting their potential for scalable public health interventions and integration into primary health programs."
Sehgal NKR, Rai S, Tonneau M, Agarwal AK, Cappella J, Kornides ML, et al. (2026). Large Language Model Chatbot Conversations vs Public Health Materials and Parental HPV Vaccination Intentions. JAMA Network Open. 10.1001/jamanetworkopen.2026.16822 — only the abstract - the full text could not be fetched — the passage: "a 3-minute interaction with an LLM chatbot (OpenAI's GPT-4o) prompted to encourage HPV vaccination using a default response style or a shorter conversational style"
Sehgal NKR, Rai S, Tonneau M, Agarwal AK, Cappella J, Kornides ML, et al. (2026). Large Language Model Chatbot Conversations vs Public Health Materials and Parental HPV Vaccination Intentions. JAMA Network Open. 10.1001/jamanetworkopen.2026.16822 — only the abstract - the full text could not be fetched — the passage: "Compared with no intervention, public health materials increased immediate vaccination intent (Cohen d = 0.53; 95% CI, 0.36-0.70), as did the default chatbot (d = 0.48; 95% CI, 0.30-0.65) and conversational chatbot (d = 0.33; 95% CI, 0.17-0.49)."
Sehgal NKR, Rai S, Tonneau M, Agarwal AK, Cappella J, Kornides ML, et al. (2026). Large Language Model Chatbot Conversations vs Public Health Materials and Parental HPV Vaccination Intentions. JAMA Network Open. 10.1001/jamanetworkopen.2026.16822 — only the abstract - the full text could not be fetched — the passage: "At 45 days, neither chatbot increased intent relative to controls, whereas public health materials maintained modest effects. No intervention increased self-reported vaccination uptake."
Sehgal NKR, Rai S, Tonneau M, Agarwal AK, Cappella J, Kornides ML, et al. (2026). Large Language Model Chatbot Conversations vs Public Health Materials and Parental HPV Vaccination Intentions. JAMA Network Open. 10.1001/jamanetworkopen.2026.16822 — only the abstract - the full text could not be fetched — the passage: "This randomized clinical trial was conducted online among individuals in the US, Canada, and the UK from March 3 to May 25, 2025, with follow-up at 15 and 45 days."
Wang Y, Ji M, Bai X, Zhuang Y. (2026). Evaluating the Accuracy, Empathy, and Readability of Generative AI Versus Registered Nurses in Discharge Planning: A Vignette‐Based Study. Nursing Open. 10.1002/nop2.70777 — only the abstract - the full text could not be fetched — the passage: "AI outperformed nurses in information comprehensiveness, but experts identified safety risks in AI-generated texts, whereas no such issues were found in nurse-produced texts."
Wang Y, Ji M, Bai X, Zhuang Y. (2026). Evaluating the Accuracy, Empathy, and Readability of Generative AI Versus Registered Nurses in Discharge Planning: A Vignette‐Based Study. Nursing Open. 10.1002/nop2.70777 — only the abstract - the full text could not be fetched — the passage: "Nurses significantly outperformed AI in both empathy and readability, and objective NLP analysis confirmed that AI-generated texts exhibited higher syntactic complexity and terminology density."
Wang Y, Ji M, Bai X, Zhuang Y. (2026). Evaluating the Accuracy, Empathy, and Readability of Generative AI Versus Registered Nurses in Discharge Planning: A Vignette‐Based Study. Nursing Open. 10.1002/nop2.70777 — only the abstract - the full text could not be fetched — the passage: "The generalized linear mixed model indicated that advancing age and lower educational attainment were associated with reduced acceptance of AI-generated texts."
Wang Y, Ji M, Bai X, Zhuang Y. (2026). Evaluating the Accuracy, Empathy, and Readability of Generative AI Versus Registered Nurses in Discharge Planning: A Vignette‐Based Study. Nursing Open. 10.1002/nop2.70777 — only the abstract - the full text could not be fetched — the passage: "AI may be better positioned as an assistive tool for nurses rather than an independent communication tool, and its deployment should address potential digital divide issues among patient populations."
Wang Y, Ji M, Bai X, Zhuang Y. (2026). Evaluating the Accuracy, Empathy, and Readability of Generative AI Versus Registered Nurses in Discharge Planning: A Vignette‐Based Study. Nursing Open. 10.1002/nop2.70777 — only the abstract - the full text could not be fetched — the passage: "Healthcare organizations should establish protocols requiring nurse verification of all artificial intelligence-generated discharge content, with particular attention to medication dos"
Wilhelmi, T., Bartsch, V., Platt, M. et al. (2026). Development and Nationwide Multicentre Evaluation of Guideline-Grounded Large Language Model Chatbots to Support Patient Self-Management and Education in Rheumatology. Journal of Medical Systems. https://doi.org/10.1007/s10916-026-02470-6 - the article this story is about — the whole article — the passage: "Further studies are needed to evaluate educational effectiveness and independently validate response quality and clinical safety."

Do not take this as professional medical advice.

The findings of other studies mentioned here are known to us through this document, which is the one we read; we did not open each of those studies.