Software in Medical Devices, by MD101 Consulting

Software GUI evaluation

In software, humans factors engineering is all about the Graphical User Interface (GUI) or user interface for software interacting with humans through hardware buttons or small digital display.
GUI evaluation is usually made with the help of software prototypes. If there isn't a prototype, it's possible to use any type of mockup, like powerpoint presentations or any other graphical tool. These mockups or prototypes are useful to show to the future end-users of the system what the future system will look like and how it will interact with users.
Advanced mockups can be used to elaborate complex workflows or use-cases, to capture the user requirements.
Thus GUI evaluation is not something done apart from capture of user requirement. Mockups are used for both. Hence users often give their feedback in a mix of GUI requirements and workflow requirements.
That's why many software companies do GUI evaluation unknowingly while capturing user requirements.

Interaction with the waterfall cycle

The section 3.27 of IEC 62366-1 defines the concept of user interface evaluation. User Interface Evaluation is made of formative evaluation and summative evaluation, two steps in the usability engineering process.

Formative Evaluation

Formative evaluation may take place during requirements analysis or before the software development project. It means that the graphical user interface can be designed before software development and that GUI can be an input data of software development.
However, GUI is prone to changes during requirements analysis, when software specifications are written with scrutiny. Thus the formative evaluation can also be made also during software requirements analysis.
This is very comparable to a risk assessment report. A preliminary risk assessment report can be made before software development and enriched during software requirements analysis.

Summative evaluation

Summative evaluation is a part of device validation regarding usability and man machine interface. It may take place when software GUI is enough stable to be shown and/or tested by end-users. Beta testing or system testing or even clinical test phases are usually the right step to do this kind of evaluation.

Interaction with agile cycles

Knowing when requirement analysis and software validation take place in agile methods tweaked for medical devices, it's easy to place formative and summative evaluations on agile cycles.

Formative evaluation

Formative evaluation can be made with the help of the product owner at the begining of every iteration. GUI user stories are sketched in the backlog and refined when the sprint starts.
Verification of the outputs of software development includes GUI tests in the sprint. That's in fact verifying that the outputs of formative evaluation were actually implemented.

Summative evaluation

Summative evaluation can be conducted partially when an intermediate software version is released. In this case, summative evaluation of every GUI subject to changes between two releases is obsolete.
Summative evaluation shall take place once the final version is released. Unless the summative evaluation was made incrementally in intermediate releases. But this sounds very theoretical. GUI which doesn't change from one release to another is rare.
Anyway, it's possible to have incremental summative evaluation in every intermediate software release (V0.x or alpha or beta releases). But the real summative evaluation shall take place after the last iteration.

Conclusion

This new version of IEC 62366 brings major changes to the usability engineering process. Manufacturers will have to dedicate resources to make their processes compliant with this new standard.
Be it waterfall or agile, the software development process will require adaptations to cope with IEC 62366-1.