Software in Medical Devices, by MD101 Consulting - Tag - AgileBlog about software medical devices and their regulatory compliance. Main subjects are software validation, IEC 62304, ISO 13485, ISO 14971, CE mark 93/42 directive and 21 CFR part 820.2024-03-27T15:32:28+01:00Cyrille Michaudurn:md5:9c06172e7cd5ed0f5b192883b657eabbDotclearIEC 62366-1 and Usability engineering for softwareurn:md5:c139439305960e140a1a5a99fd6ddf622018-07-06T13:41:00+02:002018-07-14T13:49:59+02:00MitchProcessesAgileIEC 62366risk management<p>Usability is a requirement, which has been present in regulations since a long time. It stems from the assessment of user error as a hazardous situation. It is supported by the publication AAMI HE75 standard, FDA guidances, and the publication of IEC 62366 in 2008 followed by IEC 62366-1:2015.
Although usability engineering is a requirement for the design of medical devices, most of people designing software are not familiar with this process. This article is an application of the process described in IEC 62366-1 to software design.</p> <p>Before applying this without critical thinking, please take note that what is described below may not be enough for cases where use errors can have severe consequences, e.g. devices intended to be sold to end-users directly. In such cases, requesting the services of specialists in human factors engineering is probably the best solution.</p>
<h4>Usability engineering plan</h4>
<p>Write what you do, do what you write. The story begins with a plan, as usual in the quality world.
The usability engineering plan shall describe the process and provisions put in place. For standalone software, this process lives in parallel to the software design process. The usability engineering plan can be a section of the software development plan, or a separated document.
The usability engineering plan describes the following topics:</p>
<ul>
<li>Input data review,</li>
<li>Definition of use specification,</li>
<li>Link with risk management,</li>
<li>User interface specification,</li>
<li>Formative evaluation protocol,</li>
<li>Formative evaluation report,</li>
<li>Design review(s),</li>
<li>Summative evaluation protocol,</li>
<li>Summative evaluation report,</li>
<li>Usability validation.</li>
</ul>
<p>Note: you can use the structure and content below in this article to write your own usability engineering plan (if you can afford not to pay for usability engineering specialists :-)).</p>
<h4>Usability input data</h4>
<p>Usability input data is a subset of design input data. They are gathered before or at the beginning of the design and development project. Depending on the context of the project, they can contain:</p>
<ul>
<li>Statements of work,</li>
<li>User requirements collected by sales personnel, product managers …,</li>
<li>Data from previous projects,</li>
<li>Feedback from users on previous versions of medical devices,</li>
<li>Documentation on similar medical devices,</li>
<li>Specific standards, like IEC 60601-1-8 Home use of electromedical devices,</li>
<li>Regulatory requirements, like IFU or labeling.</li>
</ul>
<p>Usability input data are reviewed along with other design input data. So, you should include these data in your design input data review.</p>
<h4>Preparing the use specification</h4>
<p>The use specification is a high-level statement, which contains information necessary to identify:</p>
<ul>
<li>the user groups which are going to be subject of the usability engineering process,</li>
<li>the use environment in which the device is going to be used,</li>
<li>the medical indications which are needed to be explored further.</li>
</ul>
<p>The use specification shall include the:</p>
<ul>
<li>Intended medical indication;</li>
<li>Intended patient population;</li>
<li>Intended part of the body or type of tissue applied to or interacted with;</li>
<li>Intended user profiles;</li>
<li>Use environment; and</li>
<li>Operating principle.</li>
</ul>
<p>Preparing the use specification can make use of various methods, for example:</p>
<ul>
<li>Contextual enquiries in the user's workplace,</li>
<li>Interview and survey techniques,</li>
<li>Expert reviews,</li>
<li>Advisory panel reviews.</li>
</ul>
<p>Usually, the use specification is prepared with expert reviews. This method is the simplest to implement (once again if you can afford not to use other methods :-))</p>
<p>The use specification is recorded in the usability management file.</p>
<h4>Analysis</h4>
<p>The usability engineering process is performed in parallel to the ISO 14971 risk management process.
Below is a diagram showing the links between the risk management process and the usability engineering process. This diagram is non-exhaustive and for clarification purposes only.
<img src="https://blog.cm-dm.com/public/24-IEC-62366-1/.IEC-62366-1-and-ISO-14971-relationships_m.png" alt="IEC-62366-1-and-ISO-14971-relationships.png" style="display:table; margin:0 auto;" title="IEC-62366-1-and-ISO-14971-relationships.png, May 2018" />
I didn’t represent the software development process on this diagram. This is not the purpose of this article to show the relationships between software development and risk management. Please, have a look at <a href="https://blog.cm-dm.com/post/2012/08/01/How-to-deal-with-ISO-14971-in-a-software-company">this post on ISO 14971</a> if you’re keen at refreshing your memory on software development.</p>
<h5>Identifying characteristics for safety</h5>
<p>This step sounds clearly like risk management. It consists in identifying:</p>
<ul>
<li>The primary operating functions in the device,</li>
<li>The use scenarios,</li>
<li>The possible use errors.</li>
</ul>
<p>In a first approach, you can answer the questions in ISO 14971 annex C to identify characteristics to safety. If the man software interaction is prone to be a source of critical hazardous situations, more advanced methods may be required.</p>
<p>These data (primary operating functions, use scenarios and possible user errors) are recorded in the usability management file.</p>
<p>For software, the primary operating functions and use scenarios can be modelled with use-case diagrams and descriptions. Note that UML requires that a use-case diagram contains a text description of the use-cases.</p>
<h5>Identifying hazardous phenomena and hazardous situations</h5>
<p>This step consists in identifying the hazardous phenomena and hazardous situations (ditto). This is classical risk assessment. They are identified with data coming from:</p>
<ul>
<li>The use specification,</li>
<li>Data from comparable devices or previous generations of the device,</li>
<li>User errors identified in the previous step.</li>
</ul>
<p>These elements are documented in the risk management file. They can be placed in a section specific to human factors.</p>
<p>Examples of user-related hazardous phenomena and situations:</p>
<ul>
<li>A warning is displayed, the user doesn’t see it,</li>
<li>A value is out of bounds, the user doesn’t see it,</li>
<li>The GUI is ill-formed, the user doesn’t understand it.</li>
</ul>
<h5>Identifying and describing hazard-related use scenarios</h5>
<p>This step is once again risk analysis: the hazardous phenomena, the sequence of events, and the hazards, resulting of the human factors are identified.
These elements are documented in the risk management file accordingly.</p>
<h5>Selecting hazards-related scenarios for summative evaluation</h5>
<p>It is not required to submit all hazard-related scenarios to the summative evaluation. It is possible to select a subset of these scenarios based on objective criteria.</p>
<p>Usually, the criteria is <em>Select hazard-related scenarios where the severity is higher than a given threshold</em> e.g.: Severity ≥ Moderate. You shall write your own criteria in the usability plan.</p>
<p>Said the other way round, it’s not worth including scenarios with low risks in the summative evaluation.</p>
<h5>Identifying mitigation actions and documenting the user interface specification</h5>
<p>The risks related to the use scenarios are then evaluated according the risk management plan (severity, frequency, and possibly detectability if you included that parameter in you risk management plan), and mitigation actions are identified, by following the risk management process.</p>
<p>Identification of mitigation actions can be done either before or during the formative evaluations. It is necessary to confirm the validity of the mitigation actions during the formative evaluations.</p>
<p>The mitigation actions are documented in the user interface specification, in order of priority (see 6.2 of ISO 14971):</p>
<ul>
<li>Changes in user-interface design, including warnings like message boxes,</li>
<li>Training to users,</li>
<li>Information in the accompanying documents: IFU and labeling.</li>
</ul>
<p>For software, the user interface specification can be included in the software requirement specification.</p>
<p>Note that warnings in the graphical user-interface can be seen as design change, and not information to the user. But this shall be verified in the summative evaluation. The warning message shall be relevant enough, placed at the right step in the workflow and change the user’s mindset to avoid an hazardous situation.</p>
<h4>Design and Formative Evaluation</h4>
<p>The formative evaluation is performed during the design phase. You can have one or more formative evaluations. The sequence of formative evaluations in the design project depends on the software being designed. There is no canonical sequence of formative evaluations.
At least one formative evaluation is required, though this could be a bit too short. Two formative evaluations sound like a good fit. The points not identified or discussed in the first evaluation can be treated in the second evaluation.</p>
<p>To be relevant with the design and development project, the formative evaluations should be placed before the last design review. Thus, the design and the user interface are “frozen” after the design review. This doesn’t mean that the formative evaluation happens during the design review.</p>
<p>The formative evaluation can be done with or without the contribution of end-users. The methods of evaluation depend on the context: questionnaires, interviews, presentations of mock-ups, observation of use of prototypes.</p>
<p>For software, the commonly adopted solution is the presentation of mock-ups or prototypes, with end-user proxies (like product managers, biomedical engineers) and end-users who can “play” with the mockups. It is also a good option to let the end-user proxies review the mock-ups to “debug” them, before presenting them to real end-users. So that the presentation doesn’t deviate (too much) with wacky requests from end-users.</p>
<h4>Summative evaluation</h4>
<p>The summative evaluation is performed at the end of the design phase. It can be done after the verification, or during the validation of the device or, if relevant or possible, during clinical assays. It aims at bringing evidence that the risks related to human factors are mitigated.
The position of the summative evaluation depends on the context of your project.</p>
<p>The summative evaluation shall be done with a population of end-users statistically significant for the evaluation. E.g. at least 5 users of each profile defined in the use specification (see <a href="https://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/HumanFactors/ucm119190.htm">FDA guidance documents</a> on Human Factors Engineering). The summative evaluation shall be done for every scenario selected according to criteria defined above (e.g. severity ≥ moderate). The methods of evaluation are left to your choice, depending on the context e.g.: use in simulated environment, use in the target environment.</p>
<p>If the data collected during the summative evaluation don’t allow to conclude on the proper effectiveness of the mitigation actions, or if new risks are identified, you shall either redo the usability engineering process iteratively, or bring rationale on the acceptability of the residual risks individually and on the overall residual risk acceptability.
The rationale can be sought in the risk/benefit ratio on the use of your device.</p>
<p>For software, the solution commonly adopted is free tests performed by selected end-users on a beta version or a release candidate version. The summative evaluation can end with the analysis of a questionnaire filled by the selected end-users.</p>
<p>The user interface of the device is deemed validated when the conclusion of the summative evaluation is positive. You’re done, good job!</p>
<h4>Application to agile methods</h4>
<p>The steps described above can be disseminated in the increments of an agile development process.</p>
<h5>Use specification, primary operating functions</h5>
<p>Use specification and primary operating functions are usually defined in the initialization/inception phase of the project. This phase sets the ground of the software functions and architecture. Likewise, the use specification and primary operating functions are defined during this phase.</p>
<p>Depending on how much you know about the software being developed, the initialization can also be the right time to write the use scenarios. If you already know, say, 80% of the user requirements, you can write the use scenarios and make the risk assessment on these scenarios at the beginning of the project. If you don’t know much on your future software, the use scenarios have to be defined/updated during the iterations.</p>
<h5>Iterations and usability engineering</h5>
<p>The next steps of the usability engineering process are performed during iterations, as shown in the following diagram and explained in the next subsections.
<img src="https://blog.cm-dm.com/public/24-IEC-62366-1/.Agile-iteration-and-iec-62366-1_m.png" alt="Agile-iteration-and-iec-62366-1.png" style="display:table; margin:0 auto;" title="Agile-iteration-and-iec-62366-1.png, May 2018" /></p>
<h5>Use scenarios and hazards mitigation</h5>
<p>The “objects” you manipulate during the iterations are epics and user-stories. They can be seen as use scenarios or small chunks of use scenarios, depending on their size. Thus, you can use them to identify hazards related to user-errors, identify mitigation actions, and update the user-interface specification accordingly.</p>
<p>Depending on the items present in the backlog (eg a brand-new use scenario), it is also possible that you have to update the use specification and the list of primary operating functions, during an iteration.</p>
<h5>Formative evaluations</h5>
<p>Agile methods usually define “personas”, which represent the user-profiles, and are used by the software development team to understand the behavior of the users.</p>
<p>You may base your formative evaluation on the use of these personas. With the role of end-user proxy for the team, the product owner is responsible for the formative evaluation. He/she does the formative evaluation of the user-stories. He/she may invite another person external to the team (or to the company) to participate to the formative evaluation.</p>
<p>You can do the formative evaluation during the demonstration of the software at the end of the iteration. Depending on the results of the formative evaluation, new items related to the user-interface may be added to the backlog and implemented in a further iteration.</p>
<h5>Summative evaluation</h5>
<p>The summative evaluation is placed after the verification phase of the agile software development process. It is performed as described in the “Summative Evaluation” section above.</p>
<p>Incremental summative evaluation may be performed with intermediate releases. I don’t recommend that method. The user-interface is subject to changes in a further intermediate release, invalidating the conclusions of an incremental summative evaluation.</p>
<p>A way to see things is to say that summative evaluation isn’t something agile.</p>
<h4>Conclusion</h4>
<p>I hope you have a better understanding on how to implement IEC 62366-1:2015 in you software development process. Remember that I'm in software above all, human factors engineering isn't my background.<br />
Congratulations and hate comments are welcome!</p>
<p>Edit: Templates<br />
You will also find in <a href="https://blog.cm-dm.com/pages/Software-Development-Process-templates">the templates repository page</a>, two templates useful to generate records of your usability engineering process:</p>
<ul>
<li><a href="https://blog.cm-dm.com/public/Templates/Usability-Engineering-File.docx">Usability Engineering File</a>,</li>
<li><a href="https://blog.cm-dm.com/public/Templates/Usability-Summative-Evaluation.docx">Usability Summative Evaluation Plan and Report</a>.</li>
</ul>
<p><br />
I share this template with the conditions of <a href="https://blog.cm-dm.com/post/2011/11/04/License">CC-BY-NC-ND license</a>.</p>
<br />
<br />
<a rel="license" href="http://creativecommons.org/licenses/by-nc-nd/3.0/fr/"><img alt="Creative Commons License" style="border-width:0" src="http://i.creativecommons.org/l/by-nc-nd/3.0/fr/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-nd/3.0/fr/">Creative Commons Attribution-NonCommercial-NoDerivs 3.0 France License</a>.
https://blog.cm-dm.com/post/2018/07/06/IEC-62366-1-and-Usability-engineering-for-software#comment-formhttps://blog.cm-dm.com/feed/atom/comments/214IEC 82304-1 - Consequences on agile software development processesurn:md5:8c306de984f1465a149933082434e96a2016-04-08T14:25:00+02:002016-04-09T10:38:26+02:00MitchStandardsAgiledevelopment processIEC 62304IEC 82304Software Validation<p>Continuing our <a href="https://blog.cm-dm.com/post/2016/01/15/IEC-82304-1-latest-news-about-the-standard-on-Health-Software">series about IEC 82304-1</a>, let's see the consequences of this standard on agile software development processes.</p> <h4>IEC 82304-1 in software development cycle</h4>
<p>The most simple presentation of the position of IEC 82304-1 in the software development lifecycle is to use the traditional waterfall process:
<img src="https://blog.cm-dm.com/public/22-IEC-82304-1/.scope_of_IEC_82304-1_in_lifecycle_m.png" alt="scope_of_IEC_82304-1_in_lifecycle.png" style="display:table; margin:0 auto;" title="scope_of_IEC_82304-1_in_lifecycle.png, Dec 2015" />
But it is also applicable with agile methods, like suggested in the following graph:
<img src="https://blog.cm-dm.com/public/22-IEC-82304-1/.IEC_82304-1_with_agile_methods_m.png" alt="IEC_82304-1_with_agile_methods.png" style="display:table; margin:0 auto;" title="IEC_82304-1_with_agile_methods.png, Jan 2016" />
In the example, we still have a separation between the system level, where use requirements and system requirements don't change when a version is being developed, and the software level, where agile cycles (sprints or something else) are performed.<br /></p>
<h4>Continuous user input</h4>
<p>But we can go further in the representation, by considering that user feedback (the input at the top-left corner of the previous graph), continuously changes. In this case, use requirements and system requirements are treated at sprint level. But not all sprints will contain new and/or modified use requirements and/or system requirements. And a validated version won't be released at the end of each sprint.
<img src="https://blog.cm-dm.com/public/22-IEC-82304-1/.IEC_82304-1_with_agile_methods_and_continuous_user_input_m.png" alt="IEC_82304-1_with_agile_methods_and_continuous_user_input.png" style="display:table; margin:0 auto;" title="IEC_82304-1_with_agile_methods_and_continuous_user_input.png, Jan 2016" />
The minor difference on the graphic compared to the previous one: continuous user input, makes a major difference for the agile development process. Thus the user input is not frozen at system level and evolves continuously.<br />
Practically speaking, the system level requirements are not given in a document, like a statement of work. They are continuously given by the users, when they are provided with demo or stable releases, and are added to the backlog.</p>
<h4>Agile at system and software level</h4>
<p>It's difficult for a medical device manufacturer to be agile both at system and software level. Having user requirements changing continuously can be troublesome.<br />
Usually, user feedback doesn't change the intended use of health software. But sometimes end-users have tons of ideas... especially when they are doctors! Being fully agile means that it could be possible that the device description or even the intended use could change! This is a situation that manufacturers don't want to encounter.<br />
That's why agile methods assign the role of product owner to someone, who acts as a filter between users' vows and what goes into the backlog.<br /></p>
<h5>Static immutable requirements</h5>
<p>Coming back to IEC 82304-1, it means that some of the user and system requirements won't change, that they set the basis of the software development project. These user and system requirements are a <strong>subset</strong> of what is found in sections 4.2 and 4.5 of IEC 82304-1. We can quote some of these types of requirements, for which we are 99% sure they won't change during the project:</p>
<ul>
<li>Intended use,</li>
<li>Addressed users or patients,</li>
<li>Main use case,</li>
<li>General system security,</li>
<li>Regulatory requirements (hum, not so sure :-).</li>
</ul>
<p>Using software development vocabulary, we can speak of static immutable requirements. All other requirements are dynamic and mutable.<br />
They can be added, deleted and changed from one sprint to another. They can be at system level i.e. other requirements found in IEC 82304-1 or at software level i.e. requirements found in section 5.2 of IEC 62304.</p>
<h5>Releasing health software</h5>
<p>Since many things can change from one sprint to another, it's not possible to have a stable release after each sprint. You won't put software in the hands of anyone after each sprint. Depending on its status you can determine to whom it will be released:</p>
<ul>
<li>Is it demo-able?
<ul>
<li>No (pliz, agile aficionados, don't frown, c'est la vie!), next sprint,</li>
<li>Yes, do a demo to the product owner,</li>
</ul></li>
<li>Is it stable?
<ul>
<li>No, next sprint,</li>
<li>Yes, consider putting it in the hands of experienced users,</li>
</ul></li>
<li>Is it ready for validation?
<ul>
<li>No, next sprint,</li>
<li>Yes, do a validation.</li>
</ul></li>
</ul>
<p><em>Ready for validation</em> means that its functional scope <em>probably</em> matches the intended use.</p>
<h5>Validating health software</h5>
<p><em>Probably matches the intended use</em> is not allowed for software on the market. A validation step is required to ensure that software does match the intended use.<br />
A formal step of validation remains mandatory for software qualified as medical device. IEC 82304-1 helps manufacturers define what should be validated in section 6 of the standard. As already mentioned, this is a major breakthrough of this standard.<br />
In the context of agile methods, the standard leaves the validation method to the choice of the manufacturer. IEC 82304-1 requires at section 6.2 to write a validation plan, which contains the validation methods. Hopefully, it doesn't impose a methods. Thus, the validation method can be:</p>
<ul>
<li>Either a formal step of validation, performed after a release candidate is delivered by the software team,</li>
<li>Or a continuous validation included in the sprints.</li>
</ul>
<p>A quick search in the scientific literature about software development, or in developers forums, shows that nobody performs a continuous validation of medical device software. It is possible, but it hasn't been implemented (or disclosed) yet.</p>
<h4>Conclusion</h4>
<p>To conclude this series of articles about IEC 82304-1, the key points are:</p>
<ul>
<li>Its scope is standalone health software,</li>
<li>It calls IEC 62304 for software development, and thus requires ISO 14971,</li>
<li>It defines types of requirements to document at user and system level,</li>
<li>It defines requirements on health software validation,</li>
<li>It will be probably recognized by the FDA and harmonized by the EU when it is published,</li>
<li>It is portable to the framework of agile methods.</li>
</ul>https://blog.cm-dm.com/post/2016/04/08/IEC-82304-1-Consequences-on-agile-software-development-processes#comment-formhttps://blog.cm-dm.com/feed/atom/comments/183How to develop a smartphone App to be FDA-cleared or CE Marked? - part 2 IEC 62304 and agile methodsurn:md5:c7ceb181057ddca614b86361e9b7122d2013-12-03T09:41:00+01:002014-10-30T13:22:14+01:00MitchStandardsAgiledevelopment processIEC 62304mobile medical app<p>In the <a href="https://blog.cm-dm.com/post/2013/11/08/How-to-develop-a-smartphone-App-to-be-FDA-cleared-or-CE-Marked-part-1">last article</a> we've seen the consequences of regulations on Apps, which run on smartphone or - more broadly - on mobile platforms.<br />
Today, let's have a look at the main standard to apply when developing software for medical devices: IEC 62304, and the context in which most people want to apply it: agile methods.</p> <h4>Agile development process</h4>
<p>Agile methods are becoming the state-of-the-art of software development. Most of mobile Apps developed today are developed with agile methods. Agile methods allow to release multiple versions of software at a high pace. A heavy trend in mobile platforms, where it's very common to have multiple updates of Apps available on apps stores.<br />
Unfortunately, this is not the case in the medical devices industry. The habit is to have a slower pace of releases (e.g. once every year), compatible with the high level of safety required for most of medical software.<br />
Agile methods are known to be hard to implement, when regulations demand a high level of formalism. The goals of agile methods are even seen in contradiction with the goals of regulations.<br /></p>
<h4>Safety level of Apps</h4>
<p>There is today a profusion of mobile medical Apps. But most these Apps are intended to be used in less hazardous situations than software use in clinical settings. For example:<br /></p>
<ul>
<li>Every medical device manufacturer is pushed by the marketing to release a very pretty App that will be a software companion to their (brick and mortar) medical device products. For instance, to be able to present a simple view of the medical device state from a remote location.<br /></li>
<li>There are many new Apps released by start-ups, which have an intended use that may be considered medical. Such apps are in a grey area where apps are either general health apps (without clinical outcome), or real medical devices with a clinical outcome. See <a href="https://blog.cm-dm.com/post/2013/11/08/How-to-develop-a-smartphone-App-to-be-FDA-cleared-or-CE-Marked-part-1">the last post</a> about the response of the FDA on such apps.<br /></li>
</ul>
<p>I'm going to excess. There are a lot of mobile apps, which are real medical devices with a real clinical outcome. Especially those designed for home-use, for patients with chronic diseases.<br />
But the trend is here. There are a lot of mobile apps intended to be used in low or non hazardous situations.<br />
For all these apps, agile methods are easier to implement, as the level of scrutiny on the software development process, demanded by regulation authorities, is lower than the one for most of "classical" medical software.<br />
<br />
Thus, this is relevant to apply agile methods to mobile apps development. Firstly because mobile apps are usually present low risks, secondly because that's the state-of-the-art of mobile app development.</p>
<h4>Agile and IEC 62304</h4>
<p>IEC 62304 is actually THE standard for software development and maintenance. It contains all the requirements to apply to software development and maintenance processes.<br />
The requirements in IEC 62304 are presented in a waterfall-like manner. It's easier to apply the standard with a waterfall process, given the way the standard presents the requirements.<br />
However, the standard doesn't impose a development process style. It can be a waterfall process, an iterative process, or full agile process. But it's more difficult to apply the standard when developing with agile methods.<br />
Fortunately, the standard is less stringent with software with low risks. It defines 3 classes (see the standard text body for the full definition of classes):</p>
<ul>
<li>Class A: no risk, a few requirements of the standard are applicable, with medium formalism,</li>
<li>Class B: medium risk, most of requirements are applicable, high formalism,</li>
<li>Class C: high risk, high formalism.</li>
</ul>
<p><em>Remark 1: I set formalism to medium or high, compared to the usual (absence of) formalism of agile methods.</em><br />
<em>Remark 2: no formalism doesn't mean no planning or no management.</em><br />
<br />
So, given the following situation:</p>
<ul>
<li>Most of apps are low risk, probably in class A,</li>
<li>Agile methods are the state-of-the-art,</li>
<li>IEC 62304 is light for class A software,</li>
</ul>
<p>We can conclude:</p>
<ul>
<li>Agile methods are recommended for the development of such mobile apps,</li>
<li>By chance, IEC 62304 allows a minimal formalism for low risk apps, compatible with the pace of agile methods.</li>
</ul>
<p><br />
<em>Remark 3 (for experts :-)): the following reasoning "</em>a mobile app of class A means no risk. So this is not a medical device<em>" is not true. No risk doesn't mean no clinical outcome. Regulations authorities may impose the medical device regulation, to confirm by post-market surveillance that there is actually no risk.</em><br />
<br /></p>
<h4>For higher classes</h4>
<p>As we said, there are also many apps of higher classes, which have a well-established clinical outcome and, other side of the medal, represent a medium or high risk.<br />
While waterfall is the classical process for hazardous software, the trend of the market is so heavy that every manufacturer wants to or plans to apply agile methods. This is possible, but with a higher learning curve than for low risk software.<br />
A possible way of implementing agile methods is:</p>
<ol>
<li>To switch to an agile method with a new project, in the phase of feasibility or prototyping, without the overhead of the IEC 62304,</li>
<li>To adapt the agile method to the constraints of IEC 62304, when industrializing the software product.</li>
</ol>
<p>The constraints of IEC 62304 on agile methods are important and may require to tweak the agile process, as I already explained in <a href="https://blog.cm-dm.com/post/2012/05/12/How-to-develop-medical-device-software-with-agile-methods">this article about agile methods in regulated environment</a>.<br />
<br />
But, from a more general point of view, agile methods allow medical devices manufacturer to follow the pace of evolutions in the world of mobile apps. Thus avoiding falling behind competitors.<br />
A massive trend is here, let's switch to IEC 62304 with agile methods!<br />
<br />
<br />
In the <a href="https://blog.cm-dm.com/post/2013/11/22/How-to-develop-a-smartphone-App-to-be-FDA-cleared-or-CE-Marked-part-3-Wireless">next article</a>, we'll talk about wireless connections.</p>https://blog.cm-dm.com/post/2013/12/03/How-to-develop-a-smartphone-App-to-be-FDA-cleared-or-CE-Marked-part-2-IEC-62304-and-agile-methods#comment-formhttps://blog.cm-dm.com/feed/atom/comments/125AAMI TIR45 on the use of agile methods becomes new FDA recognized standardurn:md5:7666a9606d616b71624e90c0c1dceedb2013-02-08T14:41:00+01:002013-02-08T14:41:00+01:00MitchRegulationsAgileFDAGuidance<p>The AAMI TIR45:2012, <em>Guidance on the use of AGILE practices in the development of medical device software</em> enters in the list of recognized standards by the FDA. See <a href="https://www.federalregister.gov/articles/2013/01/15/2013-00605/food-and-drug-administration-modernization-act-of-1997-modifications-to-the-list-of-recognized#h-9" title="Federal Register - list of recognized standards">here on Federal Register website</a>.</p> <p>I read this technical report of the AAMI (yes, I did! No kidding!). Given the concerns of many manufacturers about agile methods, this technical report is here at first to reassure them.<br />
The main conclusion of the TIR is:</p>
<ul>
<li>YES, agile methods can be used to develop medical device software,</li>
<li>YES, they allow to reach the level of scrutiny demanded by regulatory framework.</li>
</ul>
<p>To achieve this goal, a verb is continuously repeated in the document: <em>to align</em>.<br />
<br /></p>
<h4>To align</h4>
<p>The one thing to remember of this TIR, is that the regulatory perspective and the agile perspective are not aligned at first sight. This is what generates a lot of concern about agile methods in a regulated environment.<br />
However, both can be aligned - or reconciled - at the level of perspectives (or goals), concepts, and practices.<br />
The whole document is in fact a discussion on how aligning the regulatory perspective and the agile perspective. It gives a list of recommendations throughout the documents. Some are conceptual, some are practical. Not all of them are immediately useful, I would say. They need to be reviewed many times before they can be applied, with teams maturing in agile methods</p>
<h4>Agile and IEC 62304 mapping</h4>
<p>There is a gem in this document: the mapping between agile methods and IEC 62304 activities.<br />
Agile can be modeled as a method with levels boxed one another:</p>
<ul>
<li>Project,</li>
<li>Release,</li>
<li>Increment (or sprint in scrum),</li>
<li>User-story</li>
</ul>
<p><img src="https://blog.cm-dm.com/public/13-howto-agile-methods/.Agile-modeling_m.jpg" alt="Software_Medical_Devices_-_Modeling_of_agile_methods" style="display:block; margin:0 auto;" title="Software_Medical_Devices_-_Modeling_of_agile_methods, Feb 2013" /><br />
The TIR explains how to map the IEC 62304 activities described in sections 5.x with the levels of agile methods. A diagram in the TIR sums-up the mapping. This diagram is probably the best way to begin with agile methods in a regulated environment.</p>
<h4>Discipline</h4>
<p>As agile coaches say, agile methods need discipline. This is all the more true in a regulated environment. If a team wants to achieve the level of quality in software and scrutiny in documentation that is expected by regulatory requirements, then it has to be disciplined!</p>
<h4>Done is done</h4>
<p>A corollary to discipline is the concept of <em>Done is done</em>. For every artifact produced by the agile team, the concept of <em>Done is done</em> shall embrace the requirements of quality and regulatory frameworks. For instance, the documentation required for detailed design shall be finished and reviewed at the end of an increment.<br />
I took deliberately the example of detailed design documentation, to show that discipline is required. That's so easy to skip detailed design documentation in a software development project!</p>
<h4>Conclusion</h4>
<p>This TIR is not a hands-on approach on agile methods. It's a good discussion on what can be done or what can't be done with agile methods. It's also a good way to reassure people who are skeptical about agile methods!<br />
The best way to begin with agile methods is to hire a professional agile coach. When software development teams are comfortable with agile development, it's time to apply the recommendations of the TIR and insert quality and regulatory constraints in agile methods. Not the other way round.</p>https://blog.cm-dm.com/post/2013/02/08/AAMI-TIR45-on-the-use-of-agile-methods-becomes-new-FDA-recognized-standard#comment-formhttps://blog.cm-dm.com/feed/atom/comments/84Endeavour Agile ALMurn:md5:ba8f8fa57405a9eba5ef6f208581833c2012-11-09T14:06:00+01:002012-11-12T01:22:21+01:00MitchMiscAgiledevelopment processIEC 62304<p>Endeavour Agile ALM is an Open-Source solution for Agile project and resource management.</p> <p>After <a href="https://blog.cm-dm.com/post/2012/08/31/The-concept-of-continuous-certification%3A-get-rid-of-the-Big-Freeze">Open-DO, a set of tools to manage critical software</a>, here is another tool to manage software development project. Endeavour Agile ALM is an Open-Source solution for Agile project and resource management.<br />
Agile doesn't mean zero management. There are a lot of cool tools proposed by agile methods to manage projects. Most of them make an extensive use of postcards / paperboard / blackboard / whiteboard. But tools on a computer can be cool too, if they are well designed!<br />
I don't think that Endeavour Agile ALM can replace a dashboard that gives the current state of the project. But it can automatize a lot of tiny artifacts that are not worth writing on the dashboard. A tool like Endeavour Agile ALM allows to manage more precisely a software development project and, at the end, should enhance the productivity of the software development team.<br />
<br />
From the point of view of software in medical devices, such a tool allows to manage use cases (specifications), tests plan, change requests, defects, and the documentation. Using this tool is a good way to manage agile projects, with regulation constraints like IEC 62304.<br />
<br />
<a href="http://endeavour-mgmt.sourceforge.net/index.html">Endeavour Agile ALM is on SourceForge</a>.</p>https://blog.cm-dm.com/post/2012/10/26/Endeavour-Agile-ALM#comment-formhttps://blog.cm-dm.com/feed/atom/comments/67What is software verification?urn:md5:1e75b060409afc3e60b8cff084082ceb2012-10-26T14:09:00+02:002012-11-13T19:29:42+01:00MitchProcessesAgiledevelopment processIEC 62304Software ValidationSoftware Verification<p>Many people make the confusion between verification and validation. There is no exception for software! I'd even say that the confusion is even worse for standalone software.<br />
<br />
Let's see first the definition of verification and validation. I borrowed these definitions from the FDA website:</p>
<ul>
<li>Verification is confirming that design output meets the design input requirements,</li>
<li>Validation is ensuring that the device conforms to defined user needs and intended uses.</li>
</ul>
<p>OK, this remains theoretical. How to do that with software medical devices?<br />
In this article I focus on verification and will focus on validation in the next article: <a href="https://blog.cm-dm.com/post/2012/08/30/What-is-software-validation">What is software validation</a>.</p> <h3>Verification before validation</h3>
<p>This is very basic but I wanted to repeat it: <strong>the end of verification happens before the end of validation</strong>.<br />
If we place the two activities in the software development cycle, verification happens after coding and software integration activities. Hence there's nothing to verify or validate as long as there's no software or documentation.<br />
However, the position of verification activities depends deeply on the software development cycle.</p>
<h4>Position in the waterfall software development</h4>
<p>In the classical waterfall software development cycle, the verification activities are placed after coding activities. Thus the verification encompasses:</p>
<ul>
<li>Unit tests,</li>
<li>Integration tests,</li>
<li>Alpha 1 tests,</li>
<li>Alpha n tests (if any) ...</li>
<li>Beta 1 tests,</li>
<li>Beta n tests (if any) ...</li>
</ul>
<p>The naming of the tests phases is given here as an example. You may name your test phases as you wish.<br />
Just remember that there's no tests phase in real clinical conditions in this example.<br />
<br />
The diagram below summarizes that. Verification activities are highlighted with a shaded blue background. Note that the shaded blue:</p>
<ul>
<li>covers a part of coding and integration. This is where unit and integration tests take place,</li>
<li>doesn't cover the end of the cycle (where validation happens).<br /></li>
</ul>
<p><br />
<img src="https://blog.cm-dm.com/public/17-verif-valid/.verif-valid-01_m.jpg" alt="Software in medical devices - Software verification Encompasses unit tests, integration tests, and alpha, beta tests" style="display:block; margin:0 auto;" title="Software in medical devices - Software verification Encompasses unit tests, integration tests, and alpha, beta tests, oct. 2012" /></p>
<h4>Position in agile software development</h4>
<p>Agile development is a continuous development cycle. Every activity of software development happens during each iteration.<br />
If an iteration implements new functions, these functions have to be verified. Even if an iteration is only made of bug fixes, these fixes have to be verified. So, verification happens during each iteration. <br />
The diagram below highlights with a shaded blue background the verification activities in an iteration.<br />
<br />
<img src="https://blog.cm-dm.com/public/17-verif-valid/.verif-valid-02_m.jpg" alt="Software in medical devices - Software verification activities in agile iteration" style="display:block; margin:0 auto;" title="Software in medical devices - Software verification activities in agile iteration, oct. 2012" />
<br />
But iterations are only the core of agile software development in regulated environment. As I already explained <a href="https://blog.cm-dm.com/post/2012/05/12/How-to-develop-medical-device-software-with-agile-methods">in the series of articles How to develop medical device software with agile methods</a>, iterations shall be followed by a phase of consolidation.<br />
The consolidation aims at verifying that what was developed earlier is a consistent, risk free and bug free software.<br /> So the consolidation phase also contains verification activities.<br />
Whereas iterations contain incremental verification, consolidation contains a second pass of verification of software.<br />
<br />
The diagram below summarizes that. In shaded blue, the phases where verification happens.<br />
<br />
<img src="https://blog.cm-dm.com/public/17-verif-valid/.verif-valid-03_m.jpg" alt="software in medical devices - Continuous software verification during Iterations and in Consolidation" style="display:block; margin:0 auto;" title="software in medical devices - Continuous software verification during Iterations and in Consolidation, oct. 2012" /></p>
<h3>IQ, OQ and PQ</h3>
<p>Software is not present only in the medical devices but also in the production plants of medical or pharmaceutic products.<br />
When software is used in an equipment to control the production or manufacturing of products, it has to be verified and validated.<br />
The canonical model of verification of such equipment (software or not) is made of three phases:</p>
<ol>
<li>Installation Qualification,</li>
<li>Operational Qualification,</li>
<li>Performance Qualification.</li>
</ol>
<p>Note that the word <em>qualification</em> is used instead of <em>verification</em>. I won't argue on the differences between those two words.<br />
This model is a very robust one and we could get some ideas of it to verify medical device software.</p>
<h4>Installation qualification</h4>
<p>The Installation Qualification verifies that the equipment was installed according to installation drawings and specifications.<br />
For software, this means that your verification tests shall include the install procedures. One or more tests shall exists and verify the installation of software.</p>
<h4>Operational Qualification</h4>
<p>The Operational Qualification includes tests cases of start-up, operational use, maintenance, safety functions, alarms of an equipment.<br />
For software this means that your verification tests shall include tests of all these functions and states. Your verification tests shall also include the inspection of software documentation delivered to the end-user.<br />
Software verification is not limited to the main functions. All functions including the less used shall be tested one by one. Same for software documentation that shall be inspected to verify it contains information required to use your software in safe conditions.</p>
<h4>Performance Qualification</h4>
<p>The Performance Qualification aims at verifying that the user requirements and safety requirements are fulfilled.<br />
For software this means that your verification tests shall contain scenarios to test user requirements and safety requirements. These scenarios shall be close to real use, compared to the tests of functions one by one made in the previous phase.</p>
<h4>Software verification categories</h4>
<p>To sum-up what I said above, the software tests cases shall contain verification of:</p>
<ul>
<li>Software installation,</li>
<li>Software documentation delivered to end user,</li>
<li>Functions of software one by one,</li>
<li>Scenarios of use of software.</li>
</ul>
<p>On top of that, we can add the verification made by:</p>
<ul>
<li>Unit or automated tests and,</li>
<li>Integration tests.</li>
</ul>
<p>Unit and integration tests should be planned before other tests mentioned above. That's logical, when you integrate software, you can't test the installation procedure or the functions from A to Z.</p>
<h3>Software embedded or standalone</h3>
<p>Verification activities are different if software is embedded or standalone.<br /></p>
<h4>Standalone software</h4>
<p>For standalone software, all tests described above follow their "own" planning. There are limited constraints to the planning of tests. Most of time, only test data and/or system in interface should be missing or - worse - end users who do the test scenarios.<br />
Test data, systems in interface and end-users are definitely the critical resources of tests.</p>
<h4>Embedded software</h4>
<p>For embedded software, the constraints on tests are usually more important. The planning of software tests depends on the availability of hardware.<br />
The verification is usually split in two phases:</p>
<ul>
<li>Tests in simulated hardware environment,</li>
<li>Tests with real hardware.</li>
</ul>
<p>Both phases may contain the tests sub-phases described above: installation, documentation, functions one by one, and so on. This can make the planning of verification tests quite long and complex.<br />
About critical resources, embedded software have those of standalone software and add the hardware, which the software runs on.<br /></p>
<h3>Who verifies it and where?</h3>
<h4>Who?</h4>
<p>People who test software aren't the same in every phase of verification.<br /></p>
<h5>Different people at different stages</h5>
<p>During the early stages of verification, tests are done by engineers:</p>
<ul>
<li>Unit tests by software developers,</li>
<li>Integration tests by integrators (or software testers),</li>
<li>Deep testing of functions one by one by software testers (or integrators)</li>
</ul>
<p>During the late stages of verification, tests are by people with knowledge of operational use of software:</p>
<ul>
<li>Test of scenarios by product champions (or software testers with operational knowledge or selected end-users).</li>
</ul>
<p><img src="https://blog.cm-dm.com/public/17-verif-valid/.verif-valid-04_m.jpg" alt="Software in medical devices - Software verification is made by software engineers, integrators, testers, end-users" style="display:block; margin:0 auto;" title="Software in medical devices - Software verification is made by software engineers, integrators, testers, end-users, oct. 2012" /></p>
<h5>Operational not clinical</h5>
<p>If you re-read the paragraph before the diagram just above, you'll see that I wrote "operational use", neither "clinical use" nor "medical use".<br />
This distinction is very important. Even if it is of high value to have someone with clinical knowledge who tests software during verification phases, <strong>there is absolutely neither clinical nor medical use of software during verification</strong>.<br />
<br />
It is possible to have a doctor who tests your software with scenarios, or even free tests during verification.<br />
With a standalone software, you may have the feeling that the conditions are the same (basically a doctor using software on a PC), whether in verification or in validation.<br />
This is where I think resides the confusion between verification and validation, especially for standalone software.<br />
But I'm going to far. I should leave matter for my next article!</p>
<h4>Where?</h4>
<p>Software passes through various locations and platforms before being installed in the machine/PC of the end-user.<br />
The platforms on where software is installed depends on the test phase and whether software is standalone or not.<br />
For standalone software:</p>
<ul>
<li>Unit tests on software development platform,</li>
<li>Software Integration tests on software development platform or software integration platform,</li>
<li>Deep testing of functions one by one on software integration platform or on test platform with real interfaces,</li>
<li>Test of scenarios on test platform with real interfaces.</li>
</ul>
<p>For embedded software:</p>
<ul>
<li>Unit tests on software development platform,</li>
<li>Software Integration tests on software development platform or software integration platform,</li>
<li>Hardware + Software Integration tests on hardware integration platform,</li>
<li>Deep testing of functions one by one on hardware/software integration platform or on test platform with real equipment,</li>
<li>Test of scenarios on test platform with real equipment.</li>
</ul>
<p>Again, even if tests are realized on the real equipment/PC with an end-user with medical knowledge, they remain verification tests.<br />
All of this is not a secret, there's a lot about testing in literature. I just wanted to stress out what verification is.<br />
<br />
<br />
As we've seen here, the verification tests are exclusively technical. I never talked about intended use or clinical assessment and how they intersect/match with verification tests. This is the purpose of <a href="https://blog.cm-dm.com/post/2012/08/30/What-is-software-validation">my next article about software validation</a>.</p>https://blog.cm-dm.com/post/2012/08/30/What-is-software-verification#comment-formhttps://blog.cm-dm.com/feed/atom/comments/62How to use templates with agile methods?urn:md5:3d6cd9c527677f4fb8fa3d12f1e81df42012-08-03T22:01:00+02:002012-08-07T23:26:48+02:00MitchTemplatesAgiledevelopment processTemplate<p>All templates I have written are listed in <a href="https://blog.cm-dm.com/pages/Software-Development-Process-templates">my templates repository page</a>. I already explained in that page and some other articles how to use them with waterfall development process.<br />
But their use is not obvious with agile methods. Here are a few explanations.</p> <p>The concepts used in this post: reflexion, iteration and consolidation are explained <a href="https://blog.cm-dm.com/post/2012/05/16/agile">in another post</a>.<br /></p>
<h4>Very simple project: All in one template</h4>
<p>The <a href="https://blog.cm-dm.com/post/2012/07/12/Template%3A-All-in-one">all in one template</a> is made for very simple projects (a team of 2 or 3 people, 6 months of development including tests, class A software). It is made of:</p>
<ul>
<li>First two chapters about organization,</li>
<li>Next four chapters are specification, architecture, tests and tests results.<br /></li>
</ul>
<p>With agile methods, the document is updated at each iteration:</p>
<ul>
<li>Beginning of project: fill chapter 1 & 2</li>
<li>First iteraton (reflexion): fill chapter 3 & 4</li>
<li>Next iterations: update chapters 3 to 6</li>
<li>Consolidation: Redo chapter 6.</li>
</ul>
<p>To have things simple, increment the revision number of the document at each iteration:</p>
<ul>
<li>Revision 1 at the beginning of iteration 1</li>
<li>Revision 2 at the beginning of iteration 2</li>
<li>and so on until Revision n at the beginning of iteration n</li>
<li>Revision n+1 at the end of iteration n, before consolidation</li>
<li>Revision n+2 at the end of consolidation</li>
</ul>
<h4>Project with agile method</h4>
<p>Let's see now a project with more standard metrics.
Even if it's possible to change everything in your software from one iteration to another, not all documents have to be updated.</p>
<h5>Beginning</h5>
<p>The beginning of the project is the right time to write documents about the organization and the structure of your project:</p>
<ul>
<li>Project Management Plan</li>
<li>Software Development Plan</li>
<li>Software Configuration Management Plan</li>
<li>Risk Management Plan</li>
</ul>
<p>When you have these documents you can begin your real technical work! They shouldn't change. Perhaps you may update the project management plan with a new planning if the initial planning was too short (real life) or too long (dream).</p>
<h5>Reflexion</h5>
<p>Reflexion is the first iteration where concepts and ideas are cast in a first sketch of software. You will probably know most of the functionalities of your software at the end of the reflexion phase. And you will know more or less how to develop it. Thus you initialize the following documents:</p>
<ul>
<li>Software Requirements Specifications: the functionalities,</li>
<li>Usability Specifications Document: the man machine interface,</li>
<li>Risk Analysis Report: Risks and mitigations of known functionalities,</li>
<li>System Architecture Document: General architecture of your software,</li>
<li>Software Test Plan: Organization of tests and resources needed to test it.</li>
</ul>
<p>The creation of the software test plan may be delayed to a further iteration, when you are close to the consolidation.<br />
All documents above shouldn't be 100% complete at the end of reflexion phase. They only should be filled with what you know of your future software, so far.</p>
<h5>Iterations</h5>
<p>Your software becomes more and more mature at each iteration. The documentation shall reflect this maturity. Thus you update documents intialized during reflexion:</p>
<ul>
<li>Software Requirements Specifications: update the list of functionalities,</li>
<li>Usability Specifications Document: update the man machine interface,</li>
<li>Risk Analysis Report: update risks and mitigations of known functionalities,</li>
<li>System Architecture Document: Update and stabilize the general architecture of your software,</li>
<li>Software Test Plan: Update organization of tests and resources needed to test it.</li>
</ul>
<p>And as the software becomes more and more mature, you create new documents that are relevant of this maturity:</p>
<ul>
<li>Software Detailed Design: when design is stabilized,</li>
<li>Software Test Description: when software functionalities don't change,</li>
<li>Software Test Report: when functionalities are testable,</li>
<li>User Guide: as soon as you can (it is the poor relation in the documentation).</li>
</ul>
<p>Not all of these documents should be updated at each iteration. It depends on the content of the iteration:</p>
<ul>
<li>If an iteration contains new functions without change in architecture (because your architecture is smart and uses design patterns!) , you update the SRS but not the SAD,</li>
<li>If an iteration is all about refactoring (to make it smart!), you update the SAD but not the SRS.</li>
</ul>
<p>It's up to you to decide what document should be updated in each iteration.</p>
<h5>Consolidation</h5>
<p>The consolidation is a phase with tests and bugs fixes, not new functions. The following documents shall be ready to do tests.<br />
Af the beginning of consolidation:</p>
<ul>
<li>User Guide: how to use your software,</li>
<li>Software Test Description: how to test it,</li>
<li>Version Description Document: a revision before tests.</li>
</ul>
<p>At the end of consolidation:</p>
<ul>
<li>Software Test Report: the results of tests</li>
<li>Version Description Document: a revision after tests.</li>
</ul>
<p>There should be more fixed bugs in the revision of VDD after consolidation than the VDD before consolidation.</p>
<h4>Conclusion</h4>
<p>No need to create new templates to document your software development with agile methods. The only thing to know is when creating and updating documents! Q.E.D.</p>https://blog.cm-dm.com/post/2012/06/24/How-to-use-templates-with-agile-methods#comment-formhttps://blog.cm-dm.com/feed/atom/comments/12Class C software and agile methodsurn:md5:f152b472a3a6b4af0bb3e8b3a1459bd62012-07-06T13:17:00+02:002012-07-07T10:08:35+02:00MitchProcessesAgilecritical softwaredevelopment processIEC 62304ISO 14971risk management<p>Are agile methods compatible with the constraints of development set by IEC
62304 standard of class C software?<br />
After a <a href="https://blog.cm-dm.com/post/2012/06/08/How-to-combine-risk-management-process-with-agile-software-development">
series of three posts about agile methods and risks analysis</a>. I focus in
this post on IEC 62304 class C critical software.</p> <h4>Agile paradigm and critical software</h4>
<p>If I had to sum-up in one sentence the constraints of critical software, it
would be:<br />
<strong>The more critical, the less user requirements.</strong><br /></p>
<h5>Critical software don't like complexity</h5>
<p>Complexity is a direct source of bugs, so critical software don't like
complexity.<br />
Multiple user requirements are a source of complexity, the more user
requirements, the more workflows, components, branches, conditions and the
like. So the best way to have a critical software run without bugs is to limit
the numbers of user requirements.<br />
Limiting the number of user requirements doesn't mean that we limit the number
of software requirements deduced from the user requirements. In other words,
few user requirements and a lot of software requirements cutting into small
pieces the goals of user requirements.</p>
<h5>Users doing critical tasks don't like complexity</h5>
<p>If you have to do a critical task, say something very critical, like a
Cardiopulmonary Resucitation, you don't want a complex scenario. Complexity
adds stress and you don't want it. You prefer if it is very simple. If the
scenario is not simple, you are trained to act as defined according to that
scenario. This is the case of doctors who spent 10 years of their lives
learning by heart subtilities of medical protocols.<br />
So, when defining the requirements of critical software, users will be prone
to:</p>
<ul>
<li>Define the software functions for A to Z, thinking to all possibilities
according to bibliography and their experience,</li>
<li>Define a simple software.</li>
</ul>
<p>If users aren't prone to define something simple, you should guide them to
do so.<br />
This is the opposite situation of agile methods, where software definition
evolves a lot during design. This is also the ideal situation for the waterfall
model.</p>
<h5>Would user add new user requirements to critical software?</h5>
<p>You could object that it is difficult to think about everything in a
software before implementing it. If you do a presentation of a software in the
middle of the development, no doubt that beta test users will suggest or impose
new user requirements.<br />
But there is a good chance that the new requirements won't need to reconsider
the architecture or refactor the whole software. We are more in a situation of
a waterfall model with multiple runs, like the ones I showed in <a href="https://blog.cm-dm.com/post/2012/05/12/How-to-develop-medical-device-software-with-agile-methods">the
first two diagrams of this post</a>.<br />
So, agile methods aren't really adapted to critical sofware because the agile
paradigm is not respected.<br />
In other words: <strong>No changes in user requirements = no agile
method.</strong></p>
<h4>Separating very critical parts</h4>
<p>One way to mitigate risks is to separate the most critical parts in the
architecture of your software (as asserted in the IEC/TR 80002-1 guideline).
For example, the critical part runs on a microcontroller and the rest runs on
an embedded PC on linux. The first is class C, the last is class B.<br /></p>
<h5>Organizing the development of critical parts</h5>
<p>The separation of critical parts requires a good experience of software
architecture. I can't give advice about architecture, it's too dependent on
technology. But it doesn't prevent me to give my advices about the kind of
organization that manages that kind of software development!<br />
Steps are the following:</p>
<ul>
<li>identify class C critical part</li>
<li>do it in a separate sub-project, with less agile method or absolutely not
agile methods like waterfall,</li>
<li>do the less critical (or not critical) part with agile methods.</li>
</ul>
<p>Doing the less critical part with agile methods doesn't mean that it is open
bar! You still have to respect the contraints of medical software development
with agile paradigm!</p>
<h5>A way to develop the critical software</h5>
<p>The pitfall of this organization is the interface between the critical and
non critical part. There is a good change that the interface can introduce bugs
in the critical part if the interface of the less/non critical part evolves a
lot. To do so, you have to fix the interfaces early:</p>
<ul>
<li>Separate interface development from the rest of the project,</li>
<li>Implement interfaces of both parts,</li>
<li>And implement simulators of components that use the interfaces to verify
they work.</li>
</ul>
<p>The condition to do so is to have a validated design of the critical part
and its interfaces. This is not a 100% efficient solution but it can avoid a
lot of problems with the interfaces. The drawback is to have few or zero
possibilities of evolution of the interfaces when the non critical part
evolves.</p>
<h5>Software development team</h5>
<p>If you have only one unified team (this is a prerequisite of agile methods)
and the same technologies for critical and non-critical, begin with the
development of the critical part with the waterfall and continue with the
development of the non critical part with agile method.<br />
But I think this is rarely the case. There is a good chance that you have two
different teams, one for the critical part and one for the rest, especially if
technologies (framework, language, OS) are not the same in both parts.<br />
This means that both teams have to communicate and work together. Team
communication is also a source of bugs and this is the role of project managers
to avoid quiproquos and misunderstandings (something to handle in the project
management plan).</p>
<h4>Conclusion</h4>
<p>The underlying reflexion in this post is that there is no miracle solution
for the development of critical software.<br />
Very critical software are made with waterfall or equivalent method. Agile
software development and critical software development are somewhat antonyms. I
tried to give my own solutions but their principle is to separate the critical
part from the rest... and to develop critical part with a regular method.<br />
<br />
Applying agile methods to constrained and critical software is like <a href="http://youtu.be/Pk7yqlTMvp8">herding cats</a>!</p>https://blog.cm-dm.com/post/2012/07/02/Class-C-software-and-agile-methods#comment-formhttps://blog.cm-dm.com/feed/atom/comments/14How to combine risk management process with agile software development? - Part 3urn:md5:2bfba7e281695c5271c835b04fd6592a2012-06-23T16:40:00+02:002012-12-13T18:51:56+01:00MitchProcessesAgiledevelopment processIEC 62304ISO 14971risk management<p>We've seen in <a href="https://blog.cm-dm.com/post/2012/06/15/How-to-combine-risk-management-process-with-agile-software-development-Part-2">my last post</a> that it's possible to have agile development methods combined with a risk management process. To be compliant with ISO 14971 standard, a risk management plan that describes this process along iterations, has to be written. And a risk assessment report has to be created in iteration 0 and updated in every iteration, by following the risk management process like the one found in figure 1 or figure B.1 of ISO 14971 standard.</p> <h4>Risk Management inside an iteration</h4>
<p>In a <a href="https://blog.cm-dm.com/post/2012/06/01/How-to-develop-medical-device-software-with-agile-methods-Part-3">previous post</a>, I explained how an iteration should be organized. To make it short, it can be split into three steps:</p>
<ul>
<li>Writing specifications and architecture, after the kick-off meeting,</li>
<li>Coding and writing tests,</li>
<li>Running tests and fixing bugs.</li>
</ul>
<p>Even if people should not "officially" code during 1st and last phase, they usually spend a few hours (or minutes) coding. There is always something to code, you know!<br />
The diagram below summarizes these steps.<br />
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/.Diapositive09trunc_m.jpg" alt="Software in Medical Devices - activities during agile iterations" title="Software in Medical Devices - activities during agile iterations, juil. 2012" />
<br />
Let's now add the risk management activities to an iteration.</p>
<h5>Kick-off meeting</h5>
<p>During this meeting, the team picks user requirements out of the backlog and picks risks to be mitigated out of the risks backlog. These are entry data to do the specifications. The meeting sets the scope of the iteration and the team has to be happy with it.</p>
<h5>Specifications and conception</h5>
<p>Before writing specifications, it's necessary to do a new risk assessment of the scope of the iteration and create/update new risks, if any,<br />
Doing this way is compliant with §7.3 of ISO 13485 that states that security information are input data of conception. So, beginning with risk assessment before updating the software specifications is mandatory.<br />
Once risks are identified and their criticities are assessed, specification and conception begin. Specifications and conception include new or updated functionalities as well as mitigations actions of new or updated risks.<br />
At the end of specifications and conception, every risk in the scope of the iteration shall be mitigated (if it was unacceptable) and the traceability between risks, specifications and conception shall be defined and consistent.</p>
<h5>Coding, writing tests</h5>
<p>The rest of the iteration continues as usual in agile methods. Coding and writing tests represent the major part of the human time spent on the iteration. It runs smoothly if the previous step was well managed.</p>
<h5>Verification and validation meeting</h5>
<p>At the end of the iteration, the new features and the risks mitigation actions are tested. Nothing new, but remember to spend enough time on tests. Tests are also here to prove that mitigation actions work, through test traceability.<br />
The validation meeting closes the iteration. It's important during this meeting to have a look at the results of tests and the risk backlog. Tests results prove that risks are well mitigated and the risk backlog is updated accordingly.<br />
The diagram below summarizes these activities.<br />
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/.Diapositive10trunc_m.jpg" alt="Software in Medical Devices - risk management activities during agile iterations" title="Software in Medical Devices - risk management activities during agile iterations, juil. 2012" />
<br />
The vertical lines are here to show that risks management happen at the begin of the iteration. I added also an arrow between residual risk and assessment to show that this is an iterative process. The risk management process is also an iterative process by construction.</p>
<h4>Lifecycle of risks in the iterations</h4>
<p>Let's see know the lifecycle of risks during iterations.</p>
<h5>Lifetime of risks in the risks backlog vs lifetime in the risks assessment report</h5>
<p>The backlog is here to list the risks, which are not mitigated yet.<br />
A risk enters in the risk backlog when it is identified and leaves it when it is proved that the mitigation action is OK.<br />
The risks assessment report is here to prove that risks were well assessed and, if necessary, mitigated. A risks enters in the risks assessment report and never leaves it!<br />
In a very unlikely situation where a user requirement could be removed and a risk could become obsolete, the risk shall remain in the risk assessment report. An explanation should describe why this risk is obsolete. Keeping it in the report is an information that could be very useful. You never know...</p>
<h5>States of risks in the risk backlog</h5>
<p>Managing the risks in the risks backlog is better if we add states that show how close (or far) they are from being acceptable. I suggest the following states, you can choose yours, provided that your backlog is well maintained!</p>
<ul>
<li>Opened: the risk was identified</li>
<li>Analysed: the risk is assessed before mitigation</li>
<li>Mitigated: the mitigation action is known (either through software or else)</li>
<li>Implemented: the mitigation action is implemented</li>
<li>Verified: the implementation is tested and test result is OK</li>
<li>Closed: the risk is mitigated and leaves the backlog</li>
</ul>
<p>Switching risks to closed state should be done at the end of the iteration only, when everything in the scope of the iteration has been tested OK.
The diagram below shows the lifecycle of two risks:</p>
<ul>
<li>Risk 2 is identified and closed in iteration 1,</li>
<li>Risk 1 is identified in iteration 1 and survives until iteration 2 where it is closed.</li>
</ul>
<p><img src="https://blog.cm-dm.com/public/14-risks-and-agile/Diapositive11trunc.jpg" alt="Software in Medical Devices - lifecycle of risks during iterations of agile software development" title="Software in Medical Devices - lifecycle of risks during iterations of agile software development, Jun 2012" /></p>
<h5>Location of the risks backlog</h5>
<p>Keeping the concept of backlog, the risks can be stored in the same place as user requirements backlog.<br />
Keeping the concept of risks list, the risks can be stored in the risks assessment report under construction. A new column can be added at the end of the list of risks, to track the states of risks.<br />
Both are good choices. It's up to you to choose the one ou feel the more comfortable with.</p>
<h4>Criticity of risks found during iterations</h4>
<p>Risks found during iterations should not be of very unacceptable level. Hence the initialization phase (aka sprint zero) is supposed to be here to do a preliminary work where unacceptable risks are eliminated or well mitigated. The general shape of software is known at the end of the initialization phase of the project and major risks are already identified, assessed and their mitigation actions are known and planned in the further iterations.<br />
However, you could fall in the middle of iterations on an unacceptable risk which is hardly mitigable. This is a situation with a low probability but it could happen after adding new user requirements. The only solution is to "roll-back" the list of user requirements, remove the faulty ones and think with the end-users about an alternative way.</p>
<h4>Manage risks as soon as possible</h4>
<p>The less risks in your backlog, the better.<br />
As soon as a risk is identified in an iteration, you should make every effort to manage it as soon as possible. The best way is to find, implement and test OK a mitigation action inside the iteration. If it's not possible, you could find immediately a mitigation action and translate it into specifications and conception, leaving the implementation to the next iteration.<br />
A risk shouldn't stay a long time in opened or analysed states. These states shouldn't survive the end of the iteration were the risk was identified. Another way to say it is that all risks in the backlog at the end of an iteration should be in other states than opened and analysed.<br />
Of course, there is a probability that a risk remains in analyzed state accross 2 or 3 iterations. But this shall remain the exception. Agile doesn't mean <a href="http://en.wikipedia.org/wiki/Procrastination">procrastination</a>!</p>
<h4>Delivering software in the middle of development</h4>
<p>Having risks not mitigated can be seen as flouting the rules of ISO 14971, hence it's possible to have a risk which is not mitigated in intermediate versions of software. This remains acceptable as long as your software doesn't go out of the design room when the risk backlog is not empty.<br />
<br />
<strong>Never deliver a software, to beta testers, or even friends or trusted people for demonstration or research purposes while the risks backlog is not empty.</strong><br />
<br />
You can do demonstrations, presentations, big shows or whatever, as long as YOU use the software on your machine.<br />
<br />
This recommendation is not peculiar to agile methods, though. One could deliver an un-tested software at the end of the integration phase of waterfall process. But the situation is more likely with agile methods as users are involved in the process and software is usable at the end of each iteration.</p>
<h4>Conclusion</h4>
<p>The risk management process can definitely be adapted to agile methods. An agile software development process complies with ISO 14971 if:</p>
<ul>
<li>The risk management plan describes how it works in the case of iterations,</li>
<li>The risk assessment report is created and updated as many times as there are iterations in the software development,</li>
<li>The risk assessment process is done at the beginning of each iteration, before specifications and conception.</li>
</ul>
<p>The risks backlog is a good tool to monitor risks during software development. It helps to detect if the risk management process diverges from the software development process, when too many risks survive an iteration (excepted sprint zero).
<br />
<br />
The risk management process adds a lot of work, compared to agile methods in an environment free of standards and regulations. You could think that it hampers the agility of modern software development methods. I don't think so, at least for class A & B software.<br />
I will <a href="https://blog.cm-dm.com/post/2012/07/02/Class-C-software-and-agile-methods">in a next post</a> focus on class C software to see if agile methods are adapted to this case.</p>https://blog.cm-dm.com/post/2012/06/17/How-to-combine-risk-management-process-with-agile-software-development-Part-3#comment-formhttps://blog.cm-dm.com/feed/atom/comments/15How to combine risk management process with agile software development? - Part 2urn:md5:db6063e3d24e1e1f3a0ea086e64950272012-06-17T18:29:00+02:002012-09-13T21:26:23+02:00MitchProcessesAgiledevelopment processIEC 62304ISO 14971risk management<p>This post is the continuation of the <a href="https://blog.cm-dm.com/post/2012/06/08/How-to-combine-risk-management-process-with-agile-software-development">post of last week</a>.<br /> We've seen in that post that fixing bugs during software maintenance is like a small chunk of design, excepted that software specifications do not change. Therefore risk management process when fixing bugs is very close to risk management process during design, without the initial assessment of risks at the beginning of the software development cycle.</p> <h4>Risk management process with agile methods</h4>
<p>The iterations of agile methods can be represented in the diagram below:</p>
<ol>
<li>The reflexion phase where intended use and a large part of specifications are written</li>
<li>The iterations, core of software development,</li>
<li>The consolidation phase where everything is thoroughly re-tested and double-checked.</li>
</ol>
<p>To whom reflexion and consolidation sound new, I already described these phases in <a href="https://blog.cm-dm.com/post/2012/05/16/agile">a previous post called How to develop medical device software with agile methods</a>. (Have a look at this post, diagrams are larger than here!).<br />
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/.Diapositive04trunc_m.jpg" alt="Software in Medical Devices - iterations of agile methods" title="Software in Medical Devices - iterations of agile methods, juil. 2012" /></p>
<h5>Iterations of risk management</h5>
<p>Since design and risk management work hand-in-hand, the simpliest thing to do is to repeat risk management process like iterations.<br />
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/.Diapositive05trunc_m.jpg" alt="Software in Medical Devices - risk management activities during agile iterations" title="Software in Medical Devices - risk management activities during agile iterations, juil. 2012" />
<br />
The risk management process follows the rythm of agile methods.</p>
<h5>Initialization</h5>
<p>The intended use of software and the main user requirements are identified during this phase. The risk assessment report is initialized accordingly. Some risks may be mitigated immediately. Some may remain not mitigated, as final implementation is not yet decided.</p>
<h5>Risk management process in iterations</h5>
<p>The risk assessment report is updated at each iteration. Some new risks are added, some existing risks already mitigated are updated. Some risks not managed yet, are eventually mitigated.</p>
<h5>Risk management process in consolidation</h5>
<p>The consolidation phase is at the end of the iterations, where everything is thoroughly re-tested and double-checked. The mitigation actions are verified like all other requirements.<br />
So far, we've not done something very clever. We know that we have to repeat the risk management process in iterations. But that doesn't tell us how to make it efficient.</p>
<h4>Risk management process in an iteration</h4>
<p>The major problem is to have a risk management process compliant to the proces in ISO 14971, whilst having agility.</p>
<h5>Types of iterations</h5>
<p>We've seen <a href="https://blog.cm-dm.com/post/2012/05/16/agile">in a previous post</a> that the content of an iteration differs from one iteration to another. Some iterations are just fixing bugs, some other are implementing new user requirements.<br />
The content of an iteration can be simple, with coding and tests, or complex, with specifications, conception, coding and tests, as shown in the diagram below:<br />
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/.Diapositive06trunc_m.jpg" alt="Software in Medical Devices - different types of agile iterations" title="Software in Medical Devices - different types of agile iterations, juil. 2012" /></p>
<h5>Risk management in each type of iteration</h5>
<p>Although we had to distinguish the types of iterations, <strong>risk management has to be the same for all types of iterations</strong>. Why? Because even fixing the simpliest bug means reassessing existing risks and identifying possible new risks raised by the fix.<br />
A dummy example: changing the color of a text field from black to red only requires a few lines of code. Very simple. What are the consequences on hazardous situations? Maybe huge! The user may focus on this value and miss other relevant information displayed elsewhere ... leading to an error of interpretation of clinical data ...</p>
<h4>Delayed risk Mitigation</h4>
<p>The other problem of agile methods is that risks may not be mitigated immediately in an iteration. There is a non-null probability that:</p>
<ol>
<li>Risks founds during initialisation are not mitigated before iteration 1, 2 or 3 (or more)</li>
<li>Risks found during an iteration are not mitigated before the next iteration or the iteration after.</li>
</ol>
<p>The main reason of that situation is that the design of the software is constantly changing. The right action to mitigate a risk may not be found immediately.</p>
<h4>Risk backlog</h4>
<p>We lack an indicator to manage the mitigation of risks with an unknown delay. Like the backlog of user requirements and new features, a backlog of risks has to be managed.<br />
The <strong>risk backlog</strong> lives in parallel to the user requirements backlog. When new user requirements are added to the backlog, new risks are identified. When user requirements of the backlog are implemented, risks are mitigated. Both backlogs follow the same trends. But some new risks may be identifed whereas there is no new user requirements in the backlog and vice-versa.<br />
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/Diapositive07trunc.jpg" alt="Software in Medical Devices - evolution of risks backlog during agile iterations" title="Software in Medical Devices - evolution of risks backlog during agile iterations, juil. 2012" /><br />
I'll come back to this indicator and how to manage it in my next post.</p>
<h4>Reviews</h4>
<p>It is important to plan design reviews during iterations so that the developers team has a global view of what it's doing. Since risk management is tightly bound to software design, risk management reviews should be synchronized with the design reviews:</p>
<ul>
<li>The review at the end of initialization contains a review of design and a review of risks already identified and mitigated.</li>
<li>Intermediate reviews are the opportunity to assess whether software design and risks mitigations are consistent.</li>
<li>The review at the end of iterations, before consolidation, contains the final reassessment of all risks and mitigation actions.</li>
</ul>
<p>The overall assessment of residual risks shall also take place before consolidation.<br />
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/Diapositive08trunc.jpg" alt="Software in Medical Devices - risk management activities and reviews during agile iterations" title="Software in Medical Devices - risk management activities and reviews during agile iterations, juil. 2012" /></p>
<h4>Is This Still ISO 14971 compliant ?</h4>
<p>The process described to identify and assess risks, like summarized in figure 1 of ISO 14971 is definitely iterative.<br />
The fundamental difference between how people implement ISO 14971 and the real life of agile methods is that:</p>
<ul>
<li>Intended use and users requirements are supposed to be all known, before running the ISO 14971 risk assessment process.</li>
<li>Agile methods make the assumption that user requirements may be discovered late in the design process.</li>
</ul>
<p>IMHO, risks management process adapted to agile software development remains ISO 14971 compliant, as long as:</p>
<ul>
<li>The risk management plan describes with clarity how risks are managed in the frame of agile methods</li>
<li>The risk assessment report is updated at each iteration of the agile process.</li>
</ul>
<p>Yet, a double check at consolidation with risk assessment review is not just a luxury. It's the way to ensure that no risk passed through the mesh of iterations.<br />
<br />
My <a href="https://blog.cm-dm.com/post/2012/06/17/How-to-combine-risk-management-process-with-agile-software-development-Part-3">next post</a> will explain how to deal with risks in the timeframe of an iteration. It will also describe the concept of <strong>risk backlog</strong> more in details.</p>https://blog.cm-dm.com/post/2012/06/15/How-to-combine-risk-management-process-with-agile-software-development-Part-2#comment-formhttps://blog.cm-dm.com/feed/atom/comments/19How to combine risk management process with agile software development? - Part 1urn:md5:ce334dba39309148ad28949ec761329d2012-06-09T16:45:00+02:002012-09-13T21:21:52+02:00MitchProcessesAgiledevelopment processIEC 62304ISO 14971risk management<p>This post comes after <a href="https://blog.cm-dm.com/post/2012/05/12/How-to-develop-medical-device-software-with-agile-methods">a series of three posts</a> where I exposed my thoughts about development of software medical devices with agile methods. <br />
These posts were focussed on software development. Risk management deserves its own series of posts. Here is the first of three.</p> <h4>Risk management process during design: a sequential process</h4>
<p>We've seen that the waterfall development process is the most natural way to implement a software development process compliant with IEC 62304. The description of risk management process in ISO 14971 is also sequential. It is natural for engineers who read the standard to implement it the same way.<br />
This is not a criticism of these standards. Standards must be explicit and articulate to be understood by everybody. Describing steps sequentially is actually the best way to remain explicit and articulate. There are plenty of pitfalls which make this process difficult enough like that. You can imagine how it would be if there were 2 different ways (or more) to do things described in the standards!<br />
Here is a short and simplified description of that process:</p>
<ol>
<li>Identifying risks</li>
<li>Assessing risks</li>
<li>Defining mitigation actions</li>
<li>Assessing risks after mitigation</li>
<li>Assessing the overall residual risk</li>
<li>Implementing mitigation actions</li>
<li>Verifying that mitigation actions are implemented</li>
</ol>
<p>Note: To keep it simple, I deliberately ignored the iterative steps to identify, assess risks and their mitigations.</p>
<h5>Parallel with software development process</h5>
<p>This process doesn't lives alone, it is done in conjunction with the software development process.
Here is a short reminder of the software development process:</p>
<ol>
<li>Specifications</li>
<li>General Conception (or architecture conception)</li>
<li>Detailed conception</li>
<li>Coding, unit tests, integration</li>
<li>Tests, verifications</li>
</ol>
<p>The diagram below displays the parallel between both processes. Risk activities are in blue letters.<br />
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/.Diapositive01_m.jpg" alt="Software in Medical Devices - risk management activities and software development in phases of waterfall development process" title="Software in Medical Devices - risk management activities and software development in phases of waterfall development process, Jun 2012" /><br />
These processes are linked by the data they exchange one another, input data of risk management process are input data of software development process and vice-versa.</p>
<h4>Different Intensities of work</h4>
<p>Even if both processes work hand in hand, the intensity of work is far from being equivalent in every step. One process has to "wait" that the other process has completed a task before continuing. Like two threads in a computer process!</p>
<h5>Risk management: Most of work at the beginning</h5>
<p>As it is shown in the diagram, most of work of risk management is done at the beginning of the software development process. The watchdogs in the shrine of ISO 14971 risk management process would even say that identifying, assessing and even defining mitigations shall happen before the beginning of software development. Hence, risk management data are input of software development process.<br /></p>
<h5>Software development: Most of work at the beginning ... for architects</h5>
<p>The intensity of work of software development process related to risks management is at its maximum for software (or system) architects. As a result, those who deal with identification, assessment and mitigation of risks are architects. Hence mitigation actions are most of times translated in hardware requirements, software requirements, or architecture design.<br />
At the end of general conception, 100% of risks shall have been identified, assessed and mitigated. The overall residual risk shall have been assessed as well. And the traceability between mitigation actions and software requirements/architecture shall be 80%-90% done.</p>
<h5>Detailed conception, coding and integration</h5>
<p>During these phases, only traceability of mitigations which can be proved by software units in detailed conception are actually linked to risk management. The rest is pure software engineering activities. Developers develop, integrators integrate and architects (or project managers) manage the whole.</p>
<h5>Verifications</h5>
<p>During these phases, the proofs that mitigations were well implemented are collected. Form the point of view of integrators and testers, the tests cases related to mitigations actions are treated the same way as other tests cases. They verify that what was specified is what the software does.<br />
<br />
The diagram below give a synthetic view of the intensity of work of risk management process.<br />
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/.Diapositive02trunc_m.jpg" alt="Software in Medical Devices - intensity of risk management activities during software development" title="Software in Medical Devices - intensities of risk management activities during software development, Jun 2012" />
<br />
Risk management definitely doesn't have the same pace as software development. But they have to be intertwined to be efficient. Using the waterfall software development process keeps things relatively simple:</p>
<ul>
<li>high level design of software is done in parallel with risk management,</li>
<li>low level design, coding, integration and verification come after and are "only" here to give proofs of mitigation actions.</li>
</ul>
<p><br />
All of this is very classical. I just refreshed our minds about the interdependence between the two processes. From now on, we don't have anything very useful for agile development. Let's now see risk management during software maintenance.</p>
<h4>Risk management process and software maintenance</h4>
<p>Maintaining a software placed on the market is a big challenge. No software is perfect, the software manufacturer has to implement a problem resolution process and a software maintenance plan to fix bugs.</p>
<h5>Complaints and bugs have an impact on risk assessment report</h5>
<p>When bugs are identified internally or received from external sources (mainly customers complaints) the software has to be updated. But, <strong>before</strong> any software change, the impact of the bug and bug fix on existing risks has to be assessed, and the risks generated by this change has to be assessed as well.<br />
Like software development and risk management during design, software maintenance and risk management after design work in parallel.</p>
<h5>Maintenance of risk assessment report</h5>
<p>The risk assessment report of software has to be maintained during the whole life of software placed on the market. The steps are, upon any event (complaint, remark...):</p>
<ul>
<li>Identifying risks changes (new risks, modified risks)</li>
<li>Assessing risks changes</li>
<li>Assessing impacts of risks changes on unchanged risks</li>
<li>Defining mitigation actions of new/modified risks</li>
<li>Assessing new/modified risks after mitigation</li>
<li>Implementing mitigation actions</li>
<li>Verifying that mitigation actions are implemented</li>
<li>Verifying that there is no regression on previously implemented mitigation actions</li>
<li>Assessing the overall residual risks</li>
</ul>
<h5>Maintenance of software</h5>
<p>The maintenance of software medical device has to be done with the same level of scrutiny as during design. The documentation has to be updated and the verifications of modifications and their side-effects have to be realized.<br />
Fixing a bug is like back to design, McFly:</p>
<ol>
<li>Updating General Conception (in very rare cases, but it is possible)</li>
<li>Updating Detailed conception</li>
<li>Modifying Code, adding unit tests, and re-integrating</li>
<li>Testing bug fixes and, if necessary side-effects, or even the whole software.</li>
</ol>
<p>You see that, compared to software design, I removed specifications. If you change specifications you're not maintaining your software, you're creating a new version with, best case, minor functions, or, worst case scenario, a new intended use.</p>
<h5>Intensities of work</h5>
<p>Like in design where risks are assessed before implementing software, the maintenance of the risk assessment report happens before the implementation of bugs fixes. It can be summarized in the diagram below:
<img src="https://blog.cm-dm.com/public/14-risks-and-agile/.Diapositive03trunc_m.jpg" alt="Software in Medical Devices - Intensities of risk management during software maintenance" title="Software in Medical Devices - Intensities of risk management during software maintenance, Jun 2012" /></p>
<h4>Software maintenance is a bit of design process</h4>
<p>As I wrote above, software maintenance is actually like doing a small chunk of design process. Not all steps of design but most of them. That's why the risk assessment report is so subject to changes and has to be updated.<br />
<strong>So we know how to deal with small changes in the design of software during its life.</strong><br />
This is not so far from what happens in iterations of agile methods. Changes of design happen in almost every iterations. Starting from what we know about dealing with risk management during software maintenance, we can adapt it to agile methods.<br />
The main difference between iterations and software maintenance is that:</p>
<ul>
<li>design changes are big</li>
<li>they are almost systematic at every iterations.</li>
</ul>
<pre></pre>
<p>We'll see that in <a href="https://blog.cm-dm.com/post/2012/06/15/How-to-combine-risk-management-process-with-agile-software-development-Part-2">my next post</a>.</p>https://blog.cm-dm.com/post/2012/06/08/How-to-combine-risk-management-process-with-agile-software-development#comment-formhttps://blog.cm-dm.com/feed/atom/comments/20How to develop medical device software with agile methods? - Part 3urn:md5:e45fa2bef4feab1c770aa374a48b7ca92012-06-02T09:45:00+02:002012-12-13T18:53:56+01:00MitchProcessesAgiledevelopment processIEC 62304<p>In my <a href="https://blog.cm-dm.com/post/2012/05/16/agile">previous post</a>, I explained how to adapt agile methods to IEC 62304. I finish this series of 3 posts with some advices about the organization of an iteration and the software development team.</p> <p>The fundamental difference between a "classical" iteration and an iteration of software development for medical devices is the additional activities related to mandatory documentation of software development.</p>
<h4>Organization of an iteration</h4>
<h5>Typical iteration</h5>
<p>A typical iteration can be summarized on a diagram like this one:<br />
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/.Diapositive13-trunc_m.jpg" alt="Software in Medical Devices - activities during an iteration of agile development" title="Software in Medical Devices - activities during an iteration of agile development, Jun 2012" /><br />
Like any agile method, the development shall be centered on:</p>
<ol>
<li>The user's requirements</li>
<li>The coding activities</li>
</ol>
<p>To make it compliant with medical devices software development, additional tasks are mandatory.</p>
<h5>Software documentation tasks</h5>
<p>Specifications and risk analysis shall be updated at the begining of the iteration, after the startup meeting. If there are no changes in user requirements or technical specifications, there's quite a good chance that these updates are not needed.<br />
An iteration can switch directly to coding if there's nothing to update!<br />
However, there's little probability that an iteration doesn't need documentation updates.</p>
<h5>Software Testing tasks</h5>
<p>Software testing is made of two parts:</p>
<ol>
<li>Describing tests</li>
<li>Testing software</li>
</ol>
<p>IMHO, tests is the weak link, like in any other software development method.<br />
Describing tests may be skipped, either unit tests <em>I've already coded it! Coding unit tests is boring.</em> or manual tests <em>There's so much to code! I don't have time to write tests!</em><br />
Testing may be skipped as well. Testing happens at the end of an iteration and it may be skeezed by bug fixes and the like.</p>
<h5>Duration of an iteration</h5>
<p>The duration of an iteration is not fixed. It depends on the size of the requirements, bugs fixes, small enhancements which are coded.<br />
I can give some metrics:</p>
<ul>
<li>bugs fixes only: 3 to 5 days</li>
<li>minor enhancements and not chalenging user requirements: 5 to 10 days</li>
<li>challenging user's requirements, architecture refactoring: 10 to 15 days</li>
</ul>
<p>Having an iteration of more than 10 days should be an exceptional case. Those who are used to agile methods know these metrics, this doesn't sound new.<br />
<strong>Be careful however to reserve enough time to software documentation and testing.</strong> These additional tasks either extend the duration of an iteration, or reduce the number of other tasks completed.<br />
Once again, <strong>skipping documentation and testing is the main pitfall of agile methods</strong>. It happens quite faster than you imagine!</p>
<h4>Organization of the team</h4>
<p>First of all, the project manager should be someone with experience in the field of medical devices. Nothing new here.<br /></p>
<h5>Software team</h5>
<p>The additional tasks mentioned above should be shared by different people. It's not possible to have only one person responsible for these tasks. He/she won't have time to code and will be put aside of the group.<br />
From my own experience:</p>
<ul>
<li>Documentation of specifications, architecture and risks analysis is reserved to the project manager or an experienced sofware architect,</li>
<li>Documentation of detailed design should be allocated to a developer,</li>
<li>Documentation of manual tests should be allocated to a developer and/or an integrator/tester,</li>
<li>Software testing shall be done by a tester who is not the coder of the tested part.</li>
</ul>
<p>A few remarks and justifications about this:</p>
<ul>
<li>Someone who only writes documentation would be put apart quickly, he would be a source of disturbance to developers, continuously asking about the software content.</li>
<li>A tester won't be put aside, even if he/she doesn't code. A tester generates bugs and closes them, a big concern for developers!</li>
<li>A developer who hasn't the willingness to write documentation will never do it. So choose carefully the ones to whom you give documentation tasks!</li>
<li>A developer who is happy with documenting software is more prone to understand challenges outside pure coding tasks. It's one of the good ways to identify future software architects and projects managers.</li>
</ul>
<h5>Quality manager</h5>
<p>I haven't told about the quality manager yet. He/she is an important piece of the puzzle. However it's difficult to have quality managers who are software specialists at the same time. That's why I didn't include him in the software team. If you have one who has both knowledges, you're lucky.<br />
The quality manager shall be involved in the process by being present to reviews. Maybe not all meetings of iterations but from time to time during important iterations with big changes (the project manager has to decide when to invite him). His external view of the project will be of great use.<br />
Of course, the quality manager shall be present to main reviews like those I mentioned in my <a href="https://blog.cm-dm.com/post/2012/05/16/agile">previous post</a>.</p>
<h4>Conclusion</h4>
<p>To sum-up, iterations for medical devices software development are constrained by the mandatory tasks of software documentation and thorough software testing.<br />
These tasks increase the quantity of work to do in one iteration and they should be allocated to people who have the will and the time to do it. Reserving documentation to only one person is not a good solution.<br /></p>
<p>In this series of posts, I talked a little bit about risks management. But dealing with risk management in agile methods deserves its own article. It will be the subject of <a href="https://blog.cm-dm.com/post/2012/06/08/How-to-combine-risk-management-process-with-agile-software-development">my next post</a>.<br />
Bye.</p>https://blog.cm-dm.com/post/2012/06/01/How-to-develop-medical-device-software-with-agile-methods-Part-3#comment-formhttps://blog.cm-dm.com/feed/atom/comments/21How to develop medical device software with agile methods? - Part 2urn:md5:a57f24580742f0ecd1ec3d377b4630e92012-05-16T17:15:00+02:002012-09-13T21:14:01+02:00MitchProcessesAgiledevelopment processIEC 62304<p>In <a href="https://blog.cm-dm.com/post/2012/05/12/How-to-develop-medical-device-software-with-agile-methods">my previous post</a>, I explained how I tweaked the waterfall model to obtain something close to agile methods. But still not agile, actually...</p> <h3>Fourth derivation: no user requirements!</h3>
<p>In previous derivations, I made the hypothesis that we have user requirements at the beginning. What happen if we don't or have something very vague?<br />
This is what people call agile methods. Let's see how to deal with it.</p>
<h4>Agile methods, solution 1</h4>
<h5>Phase of reflexion</h5>
<p>The solution is to have a short phase of reflexion at the beginning of the developement, to draw a sketch of the architecture of the software, its main functions and its intended use. Then, the iterations begin and all documents are updated during iterations.<br />
Borrowing the terms of agile vocabulary, this phase of reflexion is like the sprint zero, which aims at initializing the backlog.<br /></p>
<h5>Not ready for reflexion?</h5>
<p>This phase of reflexion can also be a bit longer. If you only have the idea, you will need one or two months to "translate" it to software. Usually, the decision to begin a software development project stems from a pre-prototype or the result of research phase or even the competitor's products. You could take advantage of this phase of reflexion to develop a sketch of your future software. This will incredibly increase the size of your backlog!</p>
<h5>Iterations</h5>
<p>After reflexions, the iterations begin. The software documentation previously initialized is updated on the fly, according to the content of the current iteration. Not all documentation should be updated at each iteration, hence some iterations may only contain bugs fixes and some others may contain brand new user requirements.<br />
To make a parallel with the good old waterfall model, we can see iterations as small chunks of the waterfall cycle. You choose the parts of the cycle you should do in the iteration, given the content of the backlog you implement. This is represented in the image below (warning: this is not a representation of an iteration).<br />
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/.Diapositive7_m.jpg" alt="Software in Medical Devices - different types of iterations" title="Software in Medical Devices - different types of iterations, juil. 2012" />
<br />
If an iteration contains only bugs fixes, then almost no documentation should be updated. Perhaps, only detailed conception or a bit of architecture, and if necessary a small test case to verify a bug fix. If an iteration contains software requirements, then the SRS and all other documents shall be updated. And take enough time to update it!<br />
Taking our previous diagram, documents updates can be represented accordingly.<br />
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/.Diapositive9_m.jpg" alt="Software in Medical Devices - different types of iterations with documents updates" title="Software in Medical Devices - different types of iterations with documents updates, juil. 2012" /></p>
<h5>Consolidation</h5>
<p>Once again, test are the weak link of the process. You won't have the time to test everything at each iteration. You have to accept this rule to be agile. (why re-testing everything if it changes so much ???) <br />
At the end of the iterations, a consolidation phase is necessary to ensure that every technical requirement, every functional requirement, and the intended use are well implemented by the software. This consolidation is nothing else than a tests phase where the whole software is tested from A to Z. A consolidation should not be very long, since everything was already defined and tested before. This phase aims only at ensuring that there was no hole in the mesh of your iterations (holes = remaining bugs, most of times).<br />
The tests phases shall be the classical ones: unit tests, functional tests and use cases tests. Depending on the content of your software, you may have deep unit tests verification (eg: complex algorithms) and/or deep use cases tests (eg: complex GUI) in a STD.<br /></p>
<h5>Backlog</h5>
<p>The backlog is a central concept of agility. To sum-up what I said before, the diagram below shows how the size of backlog of software shoud progress, compared to the phases of reflexion, iterations and consolidation.<br />
One important thing: any user requirement in the backlog after the beginning of the consolidation is an enhancement. It is postponed to the next software version.
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/Diapositive8-trunc.jpg" alt="Software in Medical Devices - Evolution of backlog during agile development" title="Software in Medical Devices - Evolution of backlog during agile development, juil. 2012" /></p>
<h5>Where are the reviews?</h5>
<p>The user requirements review should happen when the user requirements are well defined. The software specifications review should happen when the specifications are well defined. The software architecture review should happen when the architecture is well defined...<br />
The best way is to do the reviews ... at the end, before the consolidation phase!<br />
This is not surprising, with agile methods you know only at the end how your software looks like.<br />
However, I recommend to have an intermediate review during the cycle of iterations. You can estimate when the backlog is at its maximum and plan a review with your customer to verify that all main user requirements are here. Main user requirements found after this review may be implemented (if you have time and money!) or postponed to a further version.<br />
A review is also the good time to place the pen in the inkwell and think about the content of your software. By the way, this is recommended by the agile methods! So take the chance of a review to formalize your requirements and have your documentation up-to-date.<br />
To complete the previous diagram, intermediate reviews should be planned during the development. It is up to you to determine when to place them, given the evolution of the backlog.
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/Diapositive10-trunc.jpg" alt="Software in Medical Devices - agile iterations with intermediate reviews" title="Software in Medical Devices - agile iterations with intermediate reviews, juil. 2012" /></p>
<h4>Agile methods, solution 2</h4>
<p>Here is another solution to deal with agile methods. Some of you will find it crap, but it works!<br />
Very simple:</p>
<ol>
<li>You do an agile process with poor documentation,</li>
<li>You end it with a formal waterfall process.</li>
</ol>
<p><img src="https://blog.cm-dm.com/public/13-howto-agile-methods/Diapositive11-trunc.jpg" alt="Software in Medical Devices - agile iterations followed by classical waterfall phase" title="Software in Medical Devices - agile iterations followed by classical waterfall phase, juil. 2012" />
I like this one. :-)<br />
The main drawback is to add costs to the software development project. Doing things the right way at the end of the process may be more expensive than doing it all along the project.<br />
This is often the case when a R&D software development project is turned into a product used for research purposes. It is almost or fully functional and the homologation/clearance can be seen as the gap to clinical use. So much has been done during the research phase that is not necessary to redo it with a customized agile process for medical devices. The gap between an IEC 62304 compliant software and a functional software made by research teams is the formalized method and the documentation.<br />
You know how your software looks like. This is the ideal case of waterfall process, when you know everything at the end of each phase. So this is not so stupid to do it like this!</p>
<h4>To sum-up</h4>
<p>Agile methods are the state-of-the-art for software developers today. Adapting agile methods to IEC 62304 is not difficult. Documentation has to be initialized and updated according to the content of iterations. Reviews should be planned before, during and after iterations, according to the content and the evolution of the backlog. A consolidation phase is necessary to verify that everything was implemented the right way.</p>
<p><a href="https://blog.cm-dm.com/post/2012/06/01/How-to-develop-medical-device-software-with-agile-methods-Part-3">In my next post</a>, I will explain how to organize the content of iterations and the team to have the requirements of IEC 62304 met by an agile software development project. This will be the last post of this series. Then I will in a further post address the risk analysis in agile methods.</p>https://blog.cm-dm.com/post/2012/05/16/agile#comment-formhttps://blog.cm-dm.com/feed/atom/comments/23How to develop medical device software with agile methods? - Part 1urn:md5:b0865e2b36b4fef345595b85f17f764f2012-05-12T20:55:00+02:002012-09-13T21:08:27+02:00MitchProcessesAgiledevelopment processIEC 62304<p>IEC 62304 is the standard to apply for software in medical devices. It is not bound to any software development method or model. Though not explicit, using the waterfall development model is the most straighforward way to apply this standard.<br />
The waterfall model has become old fashioned to the eyes of most of software developers, with the emergence of agile methods. Agile methods are so popular now that everydody wonders how to apply IEC 62304 with agile methods.<br /></p>
<p>An <a href="http://www.aami.org/applications/search/details.cfm?WebID=P1541_D6567">AAMI/CDV-1 TIR(SW1) - Guidance on the use of agile practices in the development of medical device software</a> exists about this subject. But it is still a draft and only members of AAMI have access to it.</p>
<p>So, how to use agile methods and still be compliant with IEC 62304?<br />
Here are my thoughts about these questionings!</p> <p>This is a preliminary article in a series of three about agile methods.</p>
<h3>Tweaking the waterfall model</h3>
<p>A word to explain how I managed to reach a conclusion about dealing with agile methods. I start from the waterfall model, the de-facto standard, and derive it sequentially, until it looks like the agile methods.</p>
<h4>First derivation of waterfall model</h4>
<p>The most simple thing to do is to repeat the waterfall cycles, instead of one plain waterfall cycle in n months, we repeat the cycle p times, each cycle lasting n/p months.<br />
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/.Diapositive1-trunc_m.jpg" alt="Software in Medical Devices - phases of waterfall development process" title="Software in Medical Devices - phases of waterfall development process, May 2012" /><br />
<br />
For example, we replace a cycle of 18 months by 3 cycles of 6 months each.<br />
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/.Diapositive2-trunc_m.jpg" alt="Software in Medical Devices - waterfall development process repeated 3 times" title="Software in Medical Devices - waterfall development process repeated 3 times, May 2012" /></p>
<h5>User requirements</h5>
<p>The percentage of user requirements completed increases during each cycle.
We obtain 3 versions of software:</p>
<ul>
<li>Iteration 1, with 33% of user requirements,</li>
<li>Iteration 2, with 66% of user requirements</li>
<li>V1.0, fully functional final version.</li>
</ul>
<p>The duration of cycles is still compatible with the production of the documents at each cycle. The SRS, SDD, STP, STD, STR (and so on, see <a href="https://blog.cm-dm.com/pages/Software-Development-Process-templates">my page about templates</a>...) are created during iteration 1; updated during iteration 2 and completed in iteration 3.<br /></p>
<h5>Repetition of tests</h5>
<p>One fundamental thing is to avoid regressions, when something used to work in iteration 1, doesn't work in iteration 2 or 3. Testing the whole software at each iteration is the only way to avoid regressions.<br />
So tests are repeated at each iteration.</p>
<h4>Second derivation</h4>
<p>Let's add more cycles, by doubling the number of cycles in 18 months.<br />
We have now 6 cycles of 3 months each.
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/Diapositive3-trunc.jpg" alt="Software in Medical Devices - waterfall repeated n times, like iterations" title="Software in Medical Devices - waterfall repeated n times, like iterations, May 2012" />
The situation tends to become difficult to manage. The burden created by the repetition of iterations becomes to high (excepted if you have plenty of time/money and people dedicated to it):</p>
<ul>
<li>User requirements may not be easily split in many iterations,</li>
<li>Technical requirements, software architecture and software design may not need being updated at each iterations,</li>
<li>Software tests have to be updated, whatever the burden (less tests --> more bugs).</li>
</ul>
<p>A solution is to spend time of the first stage to software specifications, architecture design and (a bit of) detailed design. This first stage is consacred to the reflexion on the functions, the architecture and the design. Developers may (and will) write sketches of software but this remains a low percentage of activity.<br />
The SRS, STP, SAD and SDD should be almost 100% finished at the end of the first stage.<br />
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/Diapositive3-trunc2.jpg" alt="Software in Medical Devices - waterfall development repeated n times, with first and last iterations that are modified" title="Software in Medical Devices - waterfall development repeated n times, with first and last iterations that are modified, May 2012" />
Then the iterations begin, of 3 months each. They stem from the results of the first stage to implement sequentially the software requirements in each iteration and the tests cases. So only the SDD and the STD are updated at each iteration. Others documents should remain unchanged or be slightly changed to match the small changes in software requirements and design.<br />
During the last stage, the whole software is verified, last bugs are fixed and the software is validated.<br />
Again, as I said before, the whole software has to be tested when it is complete. You should spend time of the last stage only to re-test extensively the software. This is the only way to ensure that it is correctly verified.</p>
<h4>Third derivation</h4>
<p>Let's add even more cycles, during 18 months we do 36 iterations of 2 weeks.
We absolutely don't have time to do the whole documentation of the waterfall process during each cycle. Keeping the first stage of reflexion and the last stage of testing, the iterations are shrinked to fit the timeline of the project.<br />
<img src="https://blog.cm-dm.com/public/13-howto-agile-methods/Diapositive3-trunc3.jpg" alt="Software in Medical Devices - reflexion and validation surrounding iterations" title="Software in Medical Devices - reflexion and validation surrounding iterations, May 2012" />
This is like pushing the second iteration derivation to its limits. Nothing new here compared to the second derivation.<br />
Again, the testing phases are mandatory in each iteration. Since there is no time left to retest everything in two weeks, only the software parts impacted by the coding of the current iteration should be tested at the end of the iteration, plus expected regressions.</p>
<h4>Derivations but not agile</h4>
<p>What I showed above is still not agile methods, like scrum or extreme programming.<br />
In previous derivations of the waterfall model, I made the hypothesis that we have user requirements at the beginning. Not having the user requirements is a fundamental parameter of agile methods.<br />
So, what happen if we don't have user requirements or have something very vague about the intended use?<br />
This will be <a href="https://blog.cm-dm.com/post/2012/05/16/agile">the second part of this series</a>.<br />
See you next week.</p>https://blog.cm-dm.com/post/2012/05/12/How-to-develop-medical-device-software-with-agile-methods#comment-formhttps://blog.cm-dm.com/feed/atom/comments/8