Ventilator Usability Study

@ HFCSI

This is an evaluative research project I supported at HFCSI. The high-level objective of the study was to determine whether the ventilator could be used by the intended population without serious problems, thus to inform the safety of the device in the real world. Some of the information displayed is removed or modified due to confidentiality.

I was brought in to support the research team by moderating sessions, and contributing in: development of the study guide, analysis of data, synthesizing insights, and writing of the final report.

Background and Goals

The client needed evidence from research to determine the safety and usability of their mechanical ventilator system. This was the client’s final opportunity to detect remaining issues before conducting a summative study, which would be submitted to the FDA.

  • Evaluate the effectiveness of user interactions in safely operating the mechanical ventilator without encountering safety-critical use problems.

  • Identify any new or remaining non safety-critical use problems associated with the operation of the mechanical ventilator for future improvement.

  • Determine whether the training provided contains the necessary information for successful operation of the device.

Figure 1. Behavioral results

Method

The achieve these goals, we collected behavioral and attitudinal data from the intended users. We conducted a round of formative usability testing with 30 participants divided in 15 respiratory therapists (RT) and 15 nurses (RN) representative of the intended users. We also conducted a root cause analysis with each participant to understand the reasoning behind the observations captured while they were operating the ventilator. Finally, an exit interview was conducted to record attitudes the user had towards the ventilator.

Figure 2. Design insight

Crucial Insights

UI location and display of the ventilator’s calibration feature, required to be performed for every incoming patient, failed to match user expectation from previous ventilator experience.

The calibration feature in most ventilators is presented to them as an option during new patient set up; however, in this case users are required to access the calibration feature within a submenu in the main screen. Since users are not guided through their typical workflow and instead are guided to set up the ventilator without performing a calibration, some users initially forgot to calibration the device during set up.

Access to the calibration feature is only permitted while ventilation is paused, and the menu option name changes, performing a different function when ventilation is ongoing, locking out the calibration feature. In the event of users forgetting to calibrate the system before initiating ventilation, the system does not allow access the feature unless they remember to pause it.

The combination of these two factors led to users not performing system calibration during initial set up, and then struggling to find the calibration feature if they had accidentally initiated ventilation without it.

For additional findings and learnings, please contact david_ha_08@hotmail.com

Figure 3. Design insight

Research Impact

Strategic Impact

  • Client had the opportunity to implement iterative research to their current design process to detect needed device changes early and avoid difficult modifications late in the development cycle in future products.

Tactical Impact

  • Client has the opportunity to correct remaining non-safety critical issues and improve overall user experience with the device.

Figure 4. Design insight

My Learnings

  • Work with your stakeholders! Specially when dealing with specialized systems with technical features. Preparation is important, but there might be unexpected system behavior which stakeholders can help you understand with timely answers.

  • Do not be afraid to explain yourself. Remember why iterative usability research is important, and be prepared to let stakeholders know about the benefits of conducting it early and often, in a thoughtful manner.

Challenges

  • Software glitches. The ventilator UI experienced certain alarm and system calibration glitches/errors which interrupted the flow of the session and required the moderator to handle and correct them during the session.

  • Ventilator complexity. The complexity of the ventilator functions and alarms required the research team to spend extra time learning, practicing, and familiarizing themselves with the system to be able to handle and carry informed sessions with respiratory therapists (RT) and root-cause appropriately.

  • Stakeholders’ usability testing experience. Stakeholders expectation of usability testing rigor did not initially match industry standards. The research team thoughtfully explained to stakeholders about the methodology of the study and the risk associated with certain user actions and the importance of proactive action for FDA approval.