Clinical trial · Interventional
Evaluating the Potential of Large Language Models for Respiratory Disease Consultations
Evaluating the Potential of Large Language Models for Respiratory Disease Consultations: A Randomized Crossover Trial
- Source
- ClinicalTrials.gov
- Retrieved
- Sep 8, 2026
- Layer
- normalized (units and labels harmonized; values unchanged)
- Run
- ING-CLINICALTRIALS-20260908-000001
Summary
Brief summary (as posted)
The clinical trial aimes to evaluate multiple large language models in respiratory disease consultations by comparing their performance to that of human doctors across three major medical consultation scenarios. The main question aims to answer are: * How do large language models perform in comparison to human doctors in diagnosing and consulting on respiratory diseases across various clinical scenarios? In three clinical scenarios including the online query section, the disease diagnosis section and the medical explanation section, research assistants or volunteers will be asked to cross-question all LLMs or real doctors using predefined online questions and their own issues. After each questioning session, a short washout period is implemented to eliminate potential biases.
Conditions
Conditions (10)
Free-text conditions as registered, with the CancerIndex entity they were reconciled to and the match type.
| Condition (as posted) | Mapped entity | Match | Confidence |
|---|---|---|---|
| Acute Bronchitis | — | UNRESOLVED | — |
| Acute Upper Respiratory Infection | — | UNRESOLVED | — |
| Asthma | — | UNRESOLVED | — |
| Bronchiectasis | — | UNRESOLVED | — |
| Hay Fever | — | UNRESOLVED | — |
| Lung Cancer | Malignant Lung Neoplasm | CURATED_EXACT | 0.92 |
| Pneumonia | — | UNRESOLVED | — |
| Pulmonary Embolism | — | UNRESOLVED | — |
| Pulmonary Fibrosis | — | UNRESOLVED | — |
| Tuberculosis |
Interventions
Interventions (11)
| Intervention | Type | Mapped drug | Match |
|---|---|---|---|
| Diagnosis by ChatGPT-3.5 (without search capabilities) | Diagnostic Test | — | UNRESOLVED |
| Diagnosis by ChatGPT-3.5 (with search capabilities) | Diagnostic Test | — | UNRESOLVED |
| Diagnosis by ChatGPT-4.0 (without search capabilities) | Diagnostic Test | — | UNRESOLVED |
| Diagnosis by ChatGPT-4.0 (with search capabilities) | Diagnostic Test | — | UNRESOLVED |
| Diagnosis by Claude 2 (without search capabilities) | Diagnostic Test | — | UNRESOLVED |
| Diagnosis by Claude 2 (with search capabilities) | Diagnostic Test | — | UNRESOLVED |
| Diagnosis by Claude instant (without search capabilities) | Diagnostic Test | — | UNRESOLVED |
Design
Arms and outcomes
Arms (2)
- type
- OTHER
- label
- Cross-comparison group(the disease diagnosis section)
- description
- Cross-comparison group (including human doctor controls and all LLMs)
- interventionNames
- Diagnostic Test: Diagnosis by three human doctors
- Diagnostic Test: Diagnosis by ChatGPT-3.5 (with search capabilities)
- Diagnostic Test: Diagnosis by ChatGPT-3.5 (without search capabilities)
- Diagnostic Test: Diagnosis by ChatGPT-4.0 (with search capabilities)
- Diagnostic Test: Diagnosis by ChatGPT-4.0 (without search capabilities)
- Diagnostic Test: Diagnosis by Claude instant (with search capabilities)
- Diagnostic Test: Diagnosis by Claude instant (without search capabilities)
- Diagnostic Test: Diagnosis by Claude 2 (with search capabilities)
- Diagnostic Test: Diagnosis by Claude 2 (without search capabilities)
- Diagnostic Test: Diagnosis by Gemini Pro (with search capabilities)
- Diagnostic Test: Diagnosis by Gemini Pro (without search capabilities)
- type
- OTHER
- label
- Cross-comparison group(the medical explanation section)
Eligibility
Eligibility (as posted)
- Sex
- All
Show eligibility criteria text
Inclusion Criteria: 1. Self-reported symptoms of common respiratory diseases, such as cough, chest tightness, fever, and wheezing 2. Ability to engage in LLM dialog operations independently or with minimal peer training 3. A health status deemed suitable for study participation by the pulmonology experts Exclusion Criteria: 1\) Excessively poor health status
References
Publications (1)
- BACKGROUNDMercer SW, Maxwell M, Heaney D, Watt GC. The consultation and relational empathy (CARE) measure: development and preliminary validation and reliability of an empathy-based consultation process measure. Fam Pract. 2004 Dec;21(6):699-705. doi: 10.1093/fampra/cmh621. Epub 2004 Nov 4. PMID 15528286