Key Takeaways

‍

1. Understand FDA expectations: Familiarize yourself with the FDA's thoughts on Human Factors and Usability Engineering to Medical Devices to ensure your device development process aligns with regulatory requirements.
2. Integrate Human Factors into Design Controls: Incorporate human factors considerations into your Quality Management System (QMS) and Design Control process early to streamline development and commercialization.
3. Define users, uses, and environments: Clearly outline the device's intended users, uses, use environments, and required training to inform the design process and validation protocol.
4. Identify critical tasks: Use risk management processes to determine critical tasks that could lead to serious harm if performed incorrectly, as these will be central to your Human Factors Validation.
5. Design a comprehensive validation protocol: Create a protocol that includes appropriate data collection methods (observational, knowledge task, and interview data), simulates real-world use scenarios, and defines clear acceptance criteria for each critical task. To help, we've created a template for you. Read below on how to access it.

Background

As the industry evolves to focus on the patient and their experiences, FDA has also evolved and clarified their expectations around the application of human factors to medical devices. In February 2016, FDA issued a guidance on Applying Human Factors and Usability Engineering to Medical Devices. This guidance is intended to help maximize the likelihood that new medical devices will be safe and effective for the intended users, uses and use environments.

‍

This guidance  is applicable to the user interface of any kind of medical device including software or hardware-based devices. The guidance discusses how human factors should be applied to medical device development, and goes into extensive detail on expectations for what is expected to be done and documented for a Human Factors Validation. By incorporating FDA’s expectations within the Design Controls process, manufacturers can have a smoother process to commercialization.   

‍

Get started on your Human Factors Validation today with some tips and tricks outlined below. Or download our comprehensive human factors valdation protocol template here <<ADD TEMPLATE>>

‍

‍

‍

Prepare for Human Factors Engineering

While the FDA draft guidance provides the key elements to include in the Human Factors Validation, it's a critical first step to understand how human factors will fit within the Design Control process (and total product life cycle!) of the device. Manufacturers should review their Quality System (QMS) to understand where human factors related activities and deliverables should be incorporated into the Design Controls process for their device. 

‍

In particular, manufacturers should ensure human factors are considered within the Risk Management process by intentionally identifying any potential use-related hazards early in the risk assessment and product development process. A determination of when formative evaluations for the device can occur is also critical as they require resource investments early in design controls to ensure learnings can be incorporated into the product and associated development activities. 

‍

If you have never executed any human factors related activities, now is the time to figure out if any updates to the QMS are required!

‍

DESCRIPTION OF USERS, USES, USE ENVIRONMENTS, & TRAINING: Outline the device’s intended users, uses, use environments, and training

The first step is any human factors process is to understand various aspects about:

• Who will use the device (intended users), 
• What the device will be used for (intended uses),
• Where the devices will be used (intended use environments), and 
• How the device is intended to be used (labeling and training that will be paired with the device). 

While this information needs to be documented in extensive detail as part of the Human Factors Validation protocol, it is best to outline this information early in the design process so any limitations can be considered in the product’s design. Manufacturers should pull together cross-functional representatives from Quality, Regulatory, Clinical, R&D, and Marketing to help develop the descriptions of each. 

‍

When considering users, uses, and use environments, it’s important to outline various characteristics of each. When there is more than one user, use, or use environment, it is also important to identify any characteristics that distinguish one from another to ensure the device can work across all of them. For example, a device may have 2 intended users, a lay person and medical professional. One key characteristic that differentiates those user groups is education and training… a medical professional will have had years of formal education and on-the-job training including potentially using similar devices which may help reduce use related issues. In contrast, a lay person could not be guaranteed to have any of that training and experience to rely on. That may signal that they need to be trained more extensively or need more support in the utilization of a device and these types of nuances should be considered in the development process. 

‍

CRITICAL TASKS: Define the product related critical tasks 

Once there is a high level product design or concept, the iterative risk management process can begin. This process is foundational not only to ensure the product is safe and effective but also this process is foundational to identifying the critical tasks that must be evaluated as part of the Human Factors Validation. 

‍

Generally, as a hazard analysis is conducted, use-related hazards should be identified as part of this. Inputs to that could be task analysis, expert reviews, prior use, or information from formative studies. Use related hazards can be identified as part of an overall hazard analysis or a manufacturer’s risk management process may designate this as a separate document. 

‍

Once use-related hazards have been identified and assigned a severity, the identification of critical tasks can begin. FDA defines a critical task as “A user task which, if performed incorrectly or not performed at all, would or could cause serious harm to the patient or user, where harm is defined to include compromised medical care.” Essentially this means manufacturer’s should identify what meets this definition within their own risk management process. For a manufacturer that uses a severity scale of S=1 to S=5, this could mean that any tasks associated with harms that are S=4 or S=5 are considered critical tasks. For another manufacturer who uses a 1-10 severity scale, this could mean all tasks associated with S=8, S=9, or S=10 harms would be considered critical tasks. But this is highly dependent on how exactly you define each severity level.

‍

It’s important to get alignment on the definition of a critical task early to help inform formative studies and ultimately the Human Factors Validation protocol design. Manufacturers should pull together cross-functional representatives from Clinical, Quality and Regulatory to understand how to define a critical task based on their QMS. 

‍

While each company may approach risk analysis differently, in an FDA Guidance, Content of Human Factors Information in Medical Device Marketing Submissions, issued December 2022, FDA provides an example of a use-related risk analysis that should be included in a marketing submission. Specifically, they lay out a table that contains the following information:

• Use Related Risk Analysis Task Number 
• User Task
• Possible use error(s)
• Potential hazards and clinical harm
• Severity of harm
• Critical Task (Y/N)
• Risk Mitigation Measure(s)
• Validation method for effectiveness of risk mitigation measure

If a manufacturer will not be following the exact format provided in the guidance document, it's important to ensure the above information is captured for a device as this information is recommended to be submitted as part of the marketing submission.   

‍

PROTOCOL DESIGN: Define the types of data to be collected for each critical task

Once the definition of critical tasks and subsequently, the list of critical tasks has been identified, it's time to design how each critical task will be assessed. Generally, there are 3 types of data commonly collected in Human Factors Protocols: Observational, Knowledge Task, and Interview data. 

‍

• Observational Data: Data that are collected by observing and recording events and  behaviors, as they occur. 

Observational data are useful to determine successful completion of the execution of critical tasks. Observational tasks are things such as selecting the correct button for an activity, connecting the right tubes, or responding to an alarm within a specific amount of time. 

‍

This type of data should be collected discreetly and without interfering with participants' use of the device. For example, data could be collected via video recording of user’s interactions with the device, via a one way mirror, or by having a silent observer in the room. In many cases, this method is the most objective to determine if a critical task was completed correctly. 

• Knowledge Task Data: Data that are collected via direct questioning of a participant to ascertain their understanding of essential information. This type of data is useful to help evaluate important aspects that cannot be assessed through hands-on task performance such as comprehension of warnings and precautions or storage conditions. . 

This data should be collected using open-ended and neutrally worded questions and “yes” / “no” questions should be avoided. 

‍

Some examples of knowledge check questions:

• For which co-morbidities is this system not intended to be used? 
• How should this device be stored before its use? 
• What should you use to clean the device? 

‍

• Interview Data: Qualitative data collected from users by the moderator. Interview data is intended to complement observational and knowledge task data and is not intended to be used in lieu of either method of data collection. 

Interviews enable the test facilitator to collect the users’ perspectives, generally after the completion of tasks. The interview should cover the overall experience and specific tasks completed. As part of this interview, how and why use-errors occurred should be investigated. A discussion of difficulties or close calls that were experienced should also be discussed with the participant to help determine why they may have occurred. 

‍

A full set of interview questions cannot fully be prescribed prior to observing the user however a general set of questions can be generated to help ensure consistency across interviews. That list could include the following types of questions:

• Tell me about the overall experience using the device
• What did you find easy about using the device?
• What did you find difficult about using the device? 

‍

In order to help inform the protocol design, it's easiest to take the list of critical tasks and map out what type of data can be used to evaluate each critical task. This will help inform how items can be grouped together as part of the evaluation. 

‍

PROTOCOL DESIGN: Define how the protocol will be carried out 

With the information on the various users, uses, and use environments alongside the information about all the critical tasks and how they can be evaluated, it's time to start mapping out the testing strategy. 

‍

1. Define the cohorts needed to be tested: FDA expects that if a device has more than one distinct population of users, then the validation testing should include a sufficient number of participants (generally N=15) from each user population. 

For example, if users of a device include both health care providers and lay persons, FDA would expect both are tested as part of the protocol and so in this case, a manufacturer would have 2 cohorts. In cases where an intended user may include a patient and caregiver, if there aren’t distinctions between the populations, it's possible to combine these groups and only test patients (who likely have a longer list of critical tasks than a caregiver). 

‍

2. Define the test environments: The test environment should, to the degree possible, simulate the intended use environment. Determining a test environment should factor in aspects (that were previously identified!) such as lighting level, temperature, noise, distraction, and interactions with other equipment. The test environment may also be different based on the cohort as well. For example, if a device is intended to be used in a home setting, the test environment should replicate aspects of that. So instead of doing the testing in an office setting or at a doctor’s office, consider whether an extended stay hotel that mimics a one-bedroom apartment could be used for the testing instead. If one of the intended user groups however is a physician, then using a simulated home environment would not be appropriate for that cohort. FDA also acknowledges that not all elements can be real as part of simulated use testing and testing may need to be conducted with the device not powered or used on a manikin rather than an actual patient. 

3. Define the use scenarios: By aggregating all the information, manufacturers can then create the use scenarios to be tested for each cohort. A use scenario is a specific sequence of tasks performed by a specific user. In defining a use scenario to be tested, all necessary tasks should be included and the tasks should be organized in a logical order to represent a natural workflow. For example, it wouldn’t be appropriate to test turning something off before turning something on. FDA acknowledges a device associated with a very large number of critical tasks might need to be assessed in more than one human factors validation test session (e.g., with the same participants or different participants who are representative of the same user population). This might look like testing device set-up and application in one session and device removal and disposal in another. 

As part of the design of use scenarios, the extent of device use should also simulate the real world use cases. FDA provides the following example: “for devices like over-the-counter automatic external defibrillators (AEDs), only one use session should be conducted since additional attempts would be irrelevant in an actual use setting. For other devices, typical use might involve repeated performance of critical tasks and so multiple performances of those tasks should be included in the test protocol.” 

‍

Once the use scenarios are defined, it's important to review them to ensure all critical tasks are identified within them and no critical tasks are left out. 

‍

4. Define the acceptance criteria: With the user scenarios defined, clear acceptance criteria needs to be defined for each of the critical tasks. This acceptance criteria should be objective such as “user turns on device” or “user understands the device should not be submerged in water”. The acceptance criteria can also help form the basis for the data sheets to record observational and knowledge task data to help make the analysis easier. 

Document the Human Factors Validation

Hopefully documentation has been happening all along the way but before the Human Factors Validation protocol can be finalized, the following should be formally documented and released within your QMS:

• Intended Users, Uses, and Use Environments
• Device User Interface 
• Device labeling 
• User training protocol (and any associated tools or materials) 
• Use-related risk analysis (URRA)

Then it's time to bring together information from those documents as well the information and testing strategy outlined above into a formal Human Factors Validation Protocol. This should be a standalone, controlled document in alignment with Quality System Regulations.  

‍

Once the Human Factors Validation protocol is documented, depending on device type and precedence, FDA encourages manufacturers to leverage the Q-Submission process for obtaining FDA feedback prior to submitting a premarket submission.

‍

Human Factors Validation Protocol Template

Want to save yourself some time or don’t know where to start? We've created a Human Factors Validation Protocol template (in Word format) that you can use to create your own Human Factors Validation. We include instructions and tips on how to create the plan and substantial details on what content to consider including based on the FDA’s guidance.  You can access the guide and template here - https://cosmhq.myshopify.com/

‍

And if you need additional support beyond a template, we can help you figure out how to develop a Human Factors validation strategy within your current quality system. Email us at info@cosmhq.com

‍

‍

Main Image Source:  Created with assistance from ChatGPT, powered by OpenAI

‍

‍

_{Disclaimer - https://www.cosmhq.com/disclaimer}