By Healthcare AI is becoming more prevalent and powerful in the industry, but it also poses risks and challenges that need to be addressed. A data scientist from Hackensack Meridian Health will present her findings and recommendations on how to monitor the health and impact of AI applications at the HIMSS24 conference in Orlando.
Healthcare AI refers to the use of artificial intelligence, such as machine learning and natural language processing, to improve healthcare outcomes and operations. Healthcare AI can help with tasks such as diagnosis, treatment, documentation, coding, and billing. However, healthcare AI also requires careful monitoring, as it can have adverse effects on patients and operations if not properly designed, deployed, and evaluated.
Bommae Kim, the lead data scientist at Hackensack Meridian Health, said that the focus on AI often neglects the post-deployment stage, where the real-world impact of AI applications is measured and assessed. She said that many AI products lack rigorous monitoring, which can lead to errors, biases, and inefficiencies.
Kim, who has a PhD in quantitative methods and a master’s in behavioral science, will share her insights and experiences on how to monitor the health and impact of AI applications at the HIMSS24 conference in Orlando, Florida. She will also showcase a robust monitoring framework that covers four key areas: product pipeline, model performance, user behaviors, and business impact.
The Benefits of Monitoring Healthcare AI
Kim said that monitoring healthcare AI can bring several benefits to the industry, such as:
- Detecting potential issues and preventing critical errors that can harm patients and operations
- Measuring program effectiveness and evaluating the return on investment of AI products
- Optimizing the use of AI by enhancing user adoption and designing integrated workflows
- Fostering meaningful discussions among product teams, stakeholders, and leadership on the real-world impact of AI products
Kim said that monitoring healthcare AI can also help address some of the common challenges and barriers that hinder the adoption and implementation of AI in healthcare, such as:
- Data quality and availability
- Regulatory and ethical compliance
- User trust and satisfaction
- Organizational culture and change management
The Best Practices for Monitoring Healthcare AI
Kim said that monitoring healthcare AI requires a multidisciplinary and collaborative approach, involving data scientists, clinicians, administrators, and end users. She also shared some of the best practices that she and her team have adopted for monitoring healthcare AI, such as:
- Developing a dashboard template with standardized metrics and a unified data model to streamline dashboard development and maintenance
- Using an automated alert system to notify product teams of concerning patterns and anomalies in the data and the model
- Conducting regular audits and reviews to ensure data quality, model accuracy, and user feedback
- Communicating the results and insights to the relevant stakeholders and decision-makers in a clear and concise manner
Kim’s session, “Monitoring the Health and Real-World Impact of AI Applications,” is scheduled for March 12, from 4:15-4:45 p.m. in Room W307A at HIMSS 24 in Orlando. Learn more and register.