Software in Medical Devices, by MD101 Consulting

Risk Management inside an iteration

In a previous post, I explained how an iteration should be organized. To make it short, it can be split into three steps:

• Writing specifications and architecture, after the kick-off meeting,
• Coding and writing tests,
• Running tests and fixing bugs.

Even if people should not "officially" code during 1st and last phase, they usually spend a few hours (or minutes) coding. There is always something to code, you know!
The diagram below summarizes these steps.

Let's now add the risk management activities to an iteration.

Kick-off meeting

During this meeting, the team picks user requirements out of the backlog and picks risks to be mitigated out of the risks backlog. These are entry data to do the specifications. The meeting sets the scope of the iteration and the team has to be happy with it.

Specifications and conception

Before writing specifications, it's necessary to do a new risk assessment of the scope of the iteration and create/update new risks, if any,
Doing this way is compliant with §7.3 of ISO 13485 that states that security information are input data of conception. So, beginning with risk assessment before updating the software specifications is mandatory.
Once risks are identified and their criticities are assessed, specification and conception begin. Specifications and conception include new or updated functionalities as well as mitigations actions of new or updated risks.
At the end of specifications and conception, every risk in the scope of the iteration shall be mitigated (if it was unacceptable) and the traceability between risks, specifications and conception shall be defined and consistent.

Coding, writing tests

The rest of the iteration continues as usual in agile methods. Coding and writing tests represent the major part of the human time spent on the iteration. It runs smoothly if the previous step was well managed.

Verification and validation meeting

At the end of the iteration, the new features and the risks mitigation actions are tested. Nothing new, but remember to spend enough time on tests. Tests are also here to prove that mitigation actions work, through test traceability.
The validation meeting closes the iteration. It's important during this meeting to have a look at the results of tests and the risk backlog. Tests results prove that risks are well mitigated and the risk backlog is updated accordingly.
The diagram below summarizes these activities.

The vertical lines are here to show that risks management happen at the begin of the iteration. I added also an arrow between residual risk and assessment to show that this is an iterative process. The risk management process is also an iterative process by construction.

Lifecycle of risks in the iterations

Let's see know the lifecycle of risks during iterations.

Lifetime of risks in the risks backlog vs lifetime in the risks assessment report

The backlog is here to list the risks, which are not mitigated yet.
A risk enters in the risk backlog when it is identified and leaves it when it is proved that the mitigation action is OK.
The risks assessment report is here to prove that risks were well assessed and, if necessary, mitigated. A risks enters in the risks assessment report and never leaves it!
In a very unlikely situation where a user requirement could be removed and a risk could become obsolete, the risk shall remain in the risk assessment report. An explanation should describe why this risk is obsolete. Keeping it in the report is an information that could be very useful. You never know...

States of risks in the risk backlog

Managing the risks in the risks backlog is better if we add states that show how close (or far) they are from being acceptable. I suggest the following states, you can choose yours, provided that your backlog is well maintained!

• Opened: the risk was identified
• Analysed: the risk is assessed before mitigation
• Mitigated: the mitigation action is known (either through software or else)
• Implemented: the mitigation action is implemented
• Verified: the implementation is tested and test result is OK
• Closed: the risk is mitigated and leaves the backlog

Switching risks to closed state should be done at the end of the iteration only, when everything in the scope of the iteration has been tested OK. The diagram below shows the lifecycle of two risks:

• Risk 2 is identified and closed in iteration 1,
• Risk 1 is identified in iteration 1 and survives until iteration 2 where it is closed.

Location of the risks backlog

Keeping the concept of backlog, the risks can be stored in the same place as user requirements backlog.
Keeping the concept of risks list, the risks can be stored in the risks assessment report under construction. A new column can be added at the end of the list of risks, to track the states of risks.
Both are good choices. It's up to you to choose the one ou feel the more comfortable with.

Criticity of risks found during iterations

Risks found during iterations should not be of very unacceptable level. Hence the initialization phase (aka sprint zero) is supposed to be here to do a preliminary work where unacceptable risks are eliminated or well mitigated. The general shape of software is known at the end of the initialization phase of the project and major risks are already identified, assessed and their mitigation actions are known and planned in the further iterations.
However, you could fall in the middle of iterations on an unacceptable risk which is hardly mitigable. This is a situation with a low probability but it could happen after adding new user requirements. The only solution is to "roll-back" the list of user requirements, remove the faulty ones and think with the end-users about an alternative way.

Manage risks as soon as possible

The less risks in your backlog, the better.
As soon as a risk is identified in an iteration, you should make every effort to manage it as soon as possible. The best way is to find, implement and test OK a mitigation action inside the iteration. If it's not possible, you could find immediately a mitigation action and translate it into specifications and conception, leaving the implementation to the next iteration.
A risk shouldn't stay a long time in opened or analysed states. These states shouldn't survive the end of the iteration were the risk was identified. Another way to say it is that all risks in the backlog at the end of an iteration should be in other states than opened and analysed.
Of course, there is a probability that a risk remains in analyzed state accross 2 or 3 iterations. But this shall remain the exception. Agile doesn't mean procrastination!

Delivering software in the middle of development

Having risks not mitigated can be seen as flouting the rules of ISO 14971, hence it's possible to have a risk which is not mitigated in intermediate versions of software. This remains acceptable as long as your software doesn't go out of the design room when the risk backlog is not empty.

Never deliver a software, to beta testers, or even friends or trusted people for demonstration or research purposes while the risks backlog is not empty.

You can do demonstrations, presentations, big shows or whatever, as long as YOU use the software on your machine.

This recommendation is not peculiar to agile methods, though. One could deliver an un-tested software at the end of the integration phase of waterfall process. But the situation is more likely with agile methods as users are involved in the process and software is usable at the end of each iteration.

Conclusion

The risk management process can definitely be adapted to agile methods. An agile software development process complies with ISO 14971 if:

• The risk management plan describes how it works in the case of iterations,
• The risk assessment report is created and updated as many times as there are iterations in the software development,
• The risk assessment process is done at the beginning of each iteration, before specifications and conception.

The risks backlog is a good tool to monitor risks during software development. It helps to detect if the risk management process diverges from the software development process, when too many risks survive an iteration (excepted sprint zero).

The risk management process adds a lot of work, compared to agile methods in an environment free of standards and regulations. You could think that it hampers the agility of modern software development methods. I don't think so, at least for class A & B software.
I will in a next post focus on class C software to see if agile methods are adapted to this case.