Human Algorithm Interactions for Acute Respiratory Failure Diagnosis

During 6 clinical vignettes, participants will see AI model predictions without a corresponding AI explanation. The AI model will provide a score for each diagnosis (heart failure, pneumonia, COPD) on a scale of 0-100 estimating how likely the patient's presentation was due to each of these diagnoses. In 3 of the clinical vignettes, participants will be shown standard AI model predictions and 3 vignettes they will be shown systematically biased AI model predictions, with the model specifically biased against one of the three diagnoses.

During 6 clinical vignettes, participants will see AI model predictions with explanation. The AI model will provide a score for each diagnosis on a scale of 0-100. In 3 clinical vignettes, participants will be shown standard AI model predictions and 3 vignettes they will be shown systematically biased AI model predictions with the model specifically biased against one of the three diagnoses. If the AI model provides a score above 50 an AI model explanation will be shown as gradient-weighted class activation mapping (Grad-CAM) heatmaps overlaid on the chest X-ray that highlighted which regions of the image most affecting the AI model's prediction.

In 3 clinical vignettes, participants will be shown systematically biased AI model predictions with the model specifically biased against heart failure, always predicting that heart failure is present with high likelihood in survey vignette patients with a body mass index (BMI) at or above 30. Standard predictions will be shown for the other 2 diagnoses (pneumonia, COPD).

In 3 clinical vignettes, participants will be shown systematically biased AI model predictions with the model specifically biased against pneumonia, always predicting that pneumonia is present with high likelihood in survey vignette patients 80 years or older. Standard predictions will be shown for the other 2 diagnoses (heart failure, COPD).

In 3 clinical vignettes, participants will be shown systematically biased AI model predictions with the model specifically biased against COPD, always predicting that COPD is present with high likelihood in survey vignette patients where a pre-processing filter was applied to the patient's X-ray. Standard predictions will be shown for the other 2 diagnoses (heart failure, pneumonia).