Software in Medical Devices, by MD101 Consulting

Risk management process with agile methods

The iterations of agile methods can be represented in the diagram below:

1. The reflexion phase where intended use and a large part of specifications are written
2. The iterations, core of software development,
3. The consolidation phase where everything is thoroughly re-tested and double-checked.

To whom reflexion and consolidation sound new, I already described these phases in a previous post called How to develop medical device software with agile methods. (Have a look at this post, diagrams are larger than here!).

Iterations of risk management

Since design and risk management work hand-in-hand, the simpliest thing to do is to repeat risk management process like iterations.

The risk management process follows the rythm of agile methods.

Initialization

The intended use of software and the main user requirements are identified during this phase. The risk assessment report is initialized accordingly. Some risks may be mitigated immediately. Some may remain not mitigated, as final implementation is not yet decided.

Risk management process in iterations

The risk assessment report is updated at each iteration. Some new risks are added, some existing risks already mitigated are updated. Some risks not managed yet, are eventually mitigated.

Risk management process in consolidation

The consolidation phase is at the end of the iterations, where everything is thoroughly re-tested and double-checked. The mitigation actions are verified like all other requirements.
So far, we've not done something very clever. We know that we have to repeat the risk management process in iterations. But that doesn't tell us how to make it efficient.

Risk management process in an iteration

The major problem is to have a risk management process compliant to the proces in ISO 14971, whilst having agility.

Types of iterations

We've seen in a previous post that the content of an iteration differs from one iteration to another. Some iterations are just fixing bugs, some other are implementing new user requirements.
The content of an iteration can be simple, with coding and tests, or complex, with specifications, conception, coding and tests, as shown in the diagram below:

Risk management in each type of iteration

Although we had to distinguish the types of iterations, risk management has to be the same for all types of iterations. Why? Because even fixing the simpliest bug means reassessing existing risks and identifying possible new risks raised by the fix.
A dummy example: changing the color of a text field from black to red only requires a few lines of code. Very simple. What are the consequences on hazardous situations? Maybe huge! The user may focus on this value and miss other relevant information displayed elsewhere ... leading to an error of interpretation of clinical data ...

Delayed risk Mitigation

The other problem of agile methods is that risks may not be mitigated immediately in an iteration. There is a non-null probability that:

1. Risks founds during initialisation are not mitigated before iteration 1, 2 or 3 (or more)
2. Risks found during an iteration are not mitigated before the next iteration or the iteration after.

The main reason of that situation is that the design of the software is constantly changing. The right action to mitigate a risk may not be found immediately.

Risk backlog

We lack an indicator to manage the mitigation of risks with an unknown delay. Like the backlog of user requirements and new features, a backlog of risks has to be managed.
The risk backlog lives in parallel to the user requirements backlog. When new user requirements are added to the backlog, new risks are identified. When user requirements of the backlog are implemented, risks are mitigated. Both backlogs follow the same trends. But some new risks may be identifed whereas there is no new user requirements in the backlog and vice-versa.

I'll come back to this indicator and how to manage it in my next post.

Reviews

It is important to plan design reviews during iterations so that the developers team has a global view of what it's doing. Since risk management is tightly bound to software design, risk management reviews should be synchronized with the design reviews:

• The review at the end of initialization contains a review of design and a review of risks already identified and mitigated.
• Intermediate reviews are the opportunity to assess whether software design and risks mitigations are consistent.
• The review at the end of iterations, before consolidation, contains the final reassessment of all risks and mitigation actions.

The overall assessment of residual risks shall also take place before consolidation.

Is This Still ISO 14971 compliant ?

The process described to identify and assess risks, like summarized in figure 1 of ISO 14971 is definitely iterative.
The fundamental difference between how people implement ISO 14971 and the real life of agile methods is that:

• Intended use and users requirements are supposed to be all known, before running the ISO 14971 risk assessment process.
• Agile methods make the assumption that user requirements may be discovered late in the design process.

IMHO, risks management process adapted to agile software development remains ISO 14971 compliant, as long as:

• The risk management plan describes with clarity how risks are managed in the frame of agile methods
• The risk assessment report is updated at each iteration of the agile process.

Yet, a double check at consolidation with risk assessment review is not just a luxury. It's the way to ensure that no risk passed through the mesh of iterations.

My next post will explain how to deal with risks in the timeframe of an iteration. It will also describe the concept of risk backlog more in details.