Title image of the PHORA project
Facilitate communication, education, and understand AI’s role in clinical decision making with an interactive visual explainability feature.

Affiliation

PHORA
DIG, CMU HCII

Advisors

Prof. Adam Perer
PhD Katelyn Morrison

Duration

May, 2023
Present

My Role

UX Researcher
Developer

My Contributions

Icon for users
Synthesized 28 initial user interviews
Icon of question mark
Identified key research questions
Icon of Figma
Redesigned existing PHORA interface
Icon of code
Developed the prototype in Svelte
Icon for users
Conducted pilot testings, will conduct further testings in January 2024

Context

PHORA is a clinical decision-support tool (CDSS) for Pulmonary Arterial Hypertension (PAH) patient risk prediction. PAH is a rare and high risk disease characterized by high blood pressure in the blood vessels of the lungs. The CDSS was co-designed initially, but was insufficient in meeting clinicians' needs, and the integration of the Bayesian Network viewed as "AI" by clinicians was found confusing in interactions.

Process Overview

Project process flow diagram

User Research

Need-finding Interviews

To learn about the project context, clinicians' perspectives and needs,  I drew insights from 3 rounds of user interviews, with 28 total interviews, conducted by previous PHORA team.
Round 1
Understanding the current methods and tools used for risk assessment in PAH, the participants' personal preferences, the general workflow, and their opinions on the treatment guideline feature.
Prototype for round 1 interviews
Prototype for round 2 interviews
Round 2
Continued to gather basic information about the methods and tools participants utilized, but concentrated more on their opinions regarding specific features.
Round 3
Participants were asked to assess the usefulness and comment on potential use cases of the specific features discussed in round 3, with special focus on the What-if Exploration.
Prototype for round 3 interviews

Synthesis

Affinity Diagraming

To identify clinicians' needs and their perspectives on the initial prototypes, I created an overall Affinity Diagram for the 3 rounds of interviews.

Overall Affinity Diagram for 3 rounds of interviews

Focused on the Treatment Guidelines feature to understand the CDSS's impact in clinician workflow, and enhance its educational values for less experienced clinicians.

Focused on the "What-if" Scenario (What-if Exploration) feature to identify gaps and potentials in explainability of the AI.

Enlarged screenshot of the Affinity Diagram's Treatment Guidelines sectionEnlarged screenshot of the Affinity Diagram's What-if Exploration section

Key Insights

"I would do this (use what-if exploration) pretty regularly. I'd probably play with it when I teach others about risk stratification. And this is a good teaching tool, even outside of clinical care"
‍
Participant 8, Round 3
"If a patient is going the wrong direction, you want to see where I can intervene, that's going to make the biggest difference.”
‍
Participant 7, Round 3
"Which one of the particular factors that makes somebody high risk? And what about those can we modify? And what effect will those modifications have down the road?” (questions the participant would ask himself)
‍
Participant 11, Round 3

Clinician Workflow Exploration

To identify key stakeholders and their respective needs, we created clinician workflow diagram based on the 3 rounds of interviews and conducted 2 additional semi-structured interviews.

PAH stakeholders workflow diagram

Design Goal

Based on the insights from the Affinity Diagram and the clinician workflow, we decided to focus on:

Interface Redesign & Clinician Feedback

Overall Mid-Fidelity Wireframe

To address the issues identified from previous interviews, and enhance the new PHORA tool's functionalities in communication and education, I redesigned the PHORA interface overall based on the interview insights and clinician feedback from weekly PHORA team meetings.

Overall redesign images

Iterations on What-if Exploration

I mainly focused on iterating the What-if Exploration feature, which is the central feature for explaining the AI, and educating patients or non-specialists. I iterated on its layout and visualizations for easier interactivity and more efficient information display.

What-if iterations diagram

Development

Check out our deployed site!

Image of deployed PHORA tool.

https://phora-user-study.web.app

Developed the prototype with Katelyn Morrison and Shuyi Han. The current prototype is developed with Svelte, hosted with Firebase, and connected to the latest PHORA bayesian patient risk prediction model.

My Contributions

  • Coded the overall interface structure
  • Coded various features including:
    ‍Input Variables
    ‍Historical Trend of Risk
    ‍What-if Exploration (Testable Variables, Compare Scenarios, and structure of Saved Scenarios)
  • Continue to debug and update the current PHORA interface
Say Hi!zexuanlbt@outlook.com+1 412-969-3602LinkedIn