With the recent widespread availability of electronic health record data, there are new opportunities to apply data-driven methods to clinical problems. This has led to increasing numbers of publications proposing and validating machine learning (ML) methods for clinical applications like risk prediction and treatment recommendations. However, despite these methods often achieving higher accuracy than traditional rule-based risk scores, few have been deployed and integrated into clinical practice. Moreover, those that have been are often perceived as nuisances and/or adding little clinical value.
This dissertation demonstrates an approach to translating an ML model into a comprehensive clinical support system, taking sepsis, a dysregulated host response to infection that has severe mortality and morbidity, as an example condition. We take an integrated approach that incorporates technical, clinical, and human factors perspectives. First, we developed a model to predict sepsis from retrospective data and improve the quality of predictions by accounting for the presence of confounding comorbidities during model training. Second, we designed and deployed a live sepsis alert in a hospital setting and iteratively identified key design elements to provide clinicians with relevant alerts that fit with the existing clinical workflow. Finally, a human factors approach was used to understand how clinicians incorporate insights from an ML-based system into their clinical practice and what aspects of the system facilitate or hinder building trust in system predictions. Overall, our findings emphasize that model performance is not enough to achieve clinical success and we propose several strategies for designing systems that address the unique challenges of deploying ML systems in a clinical setting.
Speaker Biography
Katie Henry is a PhD candidate in the department of computer science at Johns Hopkins University, where she works on problems at the intersection of machine learning and medicine. Prior to joining JHU, she received her BS/BA in computer science and linguistics from the University of Chicago in 2013. She was awarded a National Science Foundation Graduate Research Fellowship.