Software in Medical Devices, by MD101 Consulting - Processes - CommentsBlog about software medical devices and their regulatory compliance. Main subjects are software validation, IEC 62304, ISO 13485, ISO 14971, CE mark 93/42 directive and 21 CFR part 820.2024-03-27T15:32:28+01:00Cyrille Michaudurn:md5:9c06172e7cd5ed0f5b192883b657eabbDotclearSoftware release vs design transfer - Mitchurn:md5:896ed021401e600aa25233dfa80457042022-12-13T14:24:23+01:002022-12-13T14:24:51+01:00Mitch<p>Hi Matt<br />
IEC 62304, IEC 82304 all are applicable. I totally agree with you.<br />
As you say, delivery is different. And this is exactly where 4.1.6 or, more precisely 7.5.6 is applicable. When the manufacturer puts the device into production on servers under their responsiblities, they shall verify that the installation went well. Thus something like an installation qualification is required.<br />
More, if manufacturer's personnel perform some service with the SaMD (e.g. medical images segmentation / analysis) then they shall be trained to the device, and some performance qualification shall be performed.<br />
Regards.</p>Software release vs design transfer - mattVurn:md5:df29c3230bbdef7d90dc4f7047b49ecd2022-12-05T15:12:44+01:002022-12-05T15:12:44+01:00mattV<p>Hi Mitch<br />
Thankyou for your articles. I do have two questions.<br />
1) 4.1.6 of ISO 13485 :<br />
Did not understand why 4.1.6 is required for SaMD in SaaS context.<br />
4.1.6 is about the validation of tools used in the development (non product software ?)</p>
<p>2) What would be different in the agile software development cycle for SaaS. Is there any difference at all? IEC 62304, IEC 82304 all are applicable i guess. Only the delivery/distribution is different ?</p>Validation of software used in production and QMS - Part 1 introduction - Mitchurn:md5:7cb758000c1c2940fd2b8cedb4c612f72021-09-12T20:00:47+02:002021-09-12T19:16:48+02:00MitchThanks Mos for your comment. <br/>
Does the software user still need to reproduce their own URS, compatibility check, back up and recovery, configuration management, etc. --> Yes. <br/>
If the validation package includes configurations and references to information in the system about security management, what more should the user validate? --> you still validate your use of the eQMS against the URS. So you need some test plan and test report. Security is a specific topic within the validation process. If the vendor have their security certificate, this is probably fine. Auditing the vendor's security process would probably be overkill.Validation of software used in production and QMS - Part 1 introduction - Mosurn:md5:b70a4cdcc3e27ae5c593ceba550973152021-08-10T09:43:18+02:002021-08-10T08:43:18+02:00Mos<p>Thank you for clarifying the question on software validation.I am curious about how one should validate an eQMS that is provided with supplier validation package, including URS (User requirements Specifications).<br />
Does the software user still need to reproduce their own URS, compatibility check, back up and recovery, configuration management, etc.<br />
If the validation package includes configurations and references to information in the system about security management, what more should the user validate?</p>Validation of software used in production and QMS - Part 1 introduction - Mitchurn:md5:f211ca4a92828a5f057c29a3e9446e6b2021-03-02T23:36:40+01:002021-03-02T23:36:40+01:00MitchHi Ben,
This list is an interpretation of statements found in AAMI TIR36 and ISO/TR 80002-2 guidances.
For example, in AAMI TIR36, you have the section 4.2 titled "in scope?", with questions helping the reader identifying software in the scope, like:
"a) Could the failure or latent flaws of the software affect the safety or quality of medical devices?"
I can't reproduce all the questions, AAMI TRI36 is copyrighted.Validation of software used in production and QMS - Part 1 introduction - Benurn:md5:3a2a85bbcedd7453273c02a9be8c61862021-03-02T09:04:07+01:002021-03-02T09:04:07+01:00Ben<p>Hi Mitch,</p>
<p>In the Scope of Validation, you have defined very clearly what software needs to be validated in ISO 13485:2016, clause 4.1.6. May I know:<br />
1) where did you reference from? Because I could not find a clear definition of what software needs to be validated in ISO 13485:2016.<br />
2) which ISO document states clearly what software need or not needed to be validated.</p>Validation of software used in production and QMS - Part 1 introduction - Oliverurn:md5:c605378fe34470da0dba7546c110b6392020-11-13T22:48:33+01:002020-11-13T22:48:33+01:00Oliver<p>Thanks for your reply! Yeah, that is also my experience - applying a risk-based approach and having very light-weight documentation for certain (most?) software. As one auditor put it: "Software validation is where the 13485 became a bit unreasonable". I just wasn't sure whether other auditors and consultants also see it that way, but your experience confirms that. Thanks!</p>Validation of software used in production and QMS - Part 1 introduction - Mitchurn:md5:60a2c263443ae9695dc588bc2de52f9a2020-11-13T17:37:17+01:002020-11-13T17:37:17+01:00MitchHi Oliver,
Thanks for your feedback!
"creating validation documentation which ultimately doesn't make too much sense" --> 100% agree with that. This is the feeling I have on most of QMS projects.
The only solution is to use a risk-based approach on software. Does a SW failure lead to an unacceptable risk on product or on QMS documentation? If no, then that software is low-risk and a very lightweight validation is enough.
Trying to answer to your question about time required is not possible, though. It depends on the number of software present and the risk related to sw failures.Validation of software used in production and QMS - Part 1 introduction - Oliverurn:md5:4fc995fce3cbabfc1c8235a57d117d302020-11-10T20:37:30+01:002020-11-10T20:37:30+01:00Oliver<p>Hey Mitch,</p>
<p>Great article, as many others on your website! :) I think it's super impressive that you publish these step-by-step guides for free and even open-source your templates. I'm trying to do something similar on my website as, ironically, the quality control of regulatory consultants doesn't seem to be very good and that way we can hopefully alleviate the regulatory pain which many companies typically experience :)</p>
<p>I've noticed that the (13485-compliant) validation of software seems to be a very controversial topic which is also poorly understood by most consultants, especially those who don't have software experience.</p>
<p>I looked through your templates and am curious: How much time would you typically spend on validating a specific software package? In my experience, even startups may already have upwards of 20 software packages in use which influence production processes and/or the quality of their product. There's hardly any time left to validate those before the QMS audit; even less so for removing/changing any should validation of some fail.</p>
<p>So it often ends up becoming an exercise in creating validation documentation which ultimately doesn't make too much sense.</p>
<p>So: Interested to hear your experience here on how much time it takes you and how you get this done in a pragmatic way.</p>
<p>Oliver</p>En route to Software Verification: one goal, many methods - part 2 - Mitchurn:md5:fb0b14f8979d67678a47fc779f0ae3032020-03-18T11:26:30+01:002020-03-18T11:27:41+01:00MitchHi Sebastien,
I'd say why not, IEC 62304 doesn't require to have automated tests. You should however be prepared to answer to auditor's questions about the rationale of having only static code analysis and code reviews.En route to Software Verification: one goal, many methods - part 2 - Sébastienurn:md5:b2587c315fe331429bc0e6d434c4c7942020-03-05T17:27:15+01:002020-03-18T11:24:08+01:00SébastienHello Cyrille,
Can static code analysis or code review be considered sufficient methods of verification of software units for a DM class B?IEC 62366-1 and Usability engineering for software - Hobberurn:md5:749d11257d378959cda79a8ff4aa360b2019-12-17T13:07:26+01:002019-12-17T13:07:26+01:00Hobber<p>That was really useful, thx a lot. You explained the human, knowhow side of doing it, that others don't.</p>Validation of compiler and IDE - Why, when and how to? - Part 3 - Mitchurn:md5:124469200be89db79e3d1d12274ada192017-12-08T16:15:43+01:002017-12-08T16:24:44+01:00Mitch<p>Hi Chip,</p>
<p>Thank you for your comment, it is definitively relevant. This article is three years old and based on my experience since then, I have to agree with you. 21.CFR.870.i is applicable to software development tools. But, with a risk-based approach, I will always discard most of software development tools from validation, excepted bug trackers as said in comment #4.</p>
<p>I consider that the example on compilers in AAMI TIR 36 is a joke. And I'm polite. The fact that software development tools are in the scope of 21.CFR.870.i doesn't change my mind on validation of compilers. It's written there: http://blog.cm-dm.com/post/2014/03/28/validation-of-compiler-and-IDE-Why%2C-when-and-how-to-Part-2</p>
<p>I will never spend a second validating a compiler, even if I'm working on a critical life-sustaining medical device. For such device, I will set a mitigation action to manually review the assembly code. For other devices, I will do nothing.</p>Validation of compiler and IDE - Why, when and how to? - Part 3 - Chip Wellerurn:md5:b017fa556eb9e44ffcc7c26216b516252017-12-07T15:15:51+01:002017-12-07T15:15:51+01:00Chip Weller<p>The FDA has in multiple documents and presentations stated that 21 CFR 820.70(i) applies to software tools used during design and development. They have expressed that these tools fall under the "quality system" section. (I think there is a valid argument that the FDA retroactively interpreted this regulation, but they have been consistence on this interpretation at least since the GPSV.) One of the examples from AAMI TIR36:2007 is the validation of a C/C++ Compiler (Annex C, Example 4) where it states the compiler is used to partly automate the quality system process of "Software Implementation".</p>
<p>Based on this, I do not agree with your conclusion that from the FDA regulations "validating tools used in a software development process is not mandatory".</p>
<p>The FDA always requires validation, as does ISO 13485:2016. Based on risk this can be very simple, but there should always be a statement such as "tool XYZ has been determine acceptable for its intended use".</p>Testing is overrated - Mitchurn:md5:9fcca2a2413c41ccbabd5c7dfc2ab1632016-01-29T09:27:44+01:002016-01-29T09:28:18+01:00Mitch<p>Hi Matt,</p><p>I would say that it depends on the lifetime of the issues.</p><ul><li>Issues that are kept open from one sprint to another, of course yes. To remember what needs to be fixed.</li><li>Issues inside a sprint found by a tester, yes. To show that the tester has done his/her work.</li><li>Issues inside a sprint found by a developer and fixed by himself, or by a pair (if you do pairing rounds), probably no.</li></ul><p> Bye.</p>Testing is overrated - Matturn:md5:098f33f8caef724a84d80fea5d64823f2016-01-29T06:59:44+01:002016-01-29T09:22:54+01:00Matt<p>Hi Mitch<br />Had a question about issues which are found during development. We follow the agile way of working where all content to be worked is part of a prioritized backlog. So issues found during development time are solved in the iterations and is part of the done-ness of the story. At the end of development all issues have to be closed before starting final verification of the product. Do we need to identify and track the issues which are part of the regular work during development ? I know that from verification onwards all issues have to be identified and tracked</p>Validation of software used in production and QMS - Part 1 introduction - Mitchurn:md5:5a79c11cdd8e344b189f3e1f4b5d5b662015-07-09T17:08:51+02:002015-07-09T16:12:28+02:00Mitch<p>Hi Kataput,</p>
<p>Thank you for your comment.</p>
<p>A barebone MS Office document or spreadsheet can be seen as electronic paper. It can be admitted that MS Office does his job well enough to create and modify electronic documents and records. I agree with you that the controversy is in <em>"It can be admitted that"</em>.</p>
<p>Challenging the capability of MS Office to write documents and records is like challenging the capability of paper and pen to write documents and records. As long as you use basic functions of MSO, you stay in a safe zone.</p>
<p>The risk is more in the storage of documents: how to make sure that the ink won't fade?, likewise how to make sure that old MSO files will be readable? But this is not MSO that we have to validate here, this is the long-term storage processes, allowing us to be compliant record storage requirements of regulations.</p>
<p>Note that it's not what I wrote for MSO macros. Macros are company-made piece of software and the need of their validation should be assessed with scrutiny.</p>Validation of software used in production and QMS - Part 1 introduction - Kataputurn:md5:e707f246f42102a1d459c3e86116f6722015-06-29T09:25:02+02:002015-07-09T15:55:41+02:00Kataput<p>Dear Mitch,</p>
<p>Thank you for your article. Software validation is still a controversely discussed topic here in Europe...at least for SME.</p>
<p>I got a question to the exclusions you mentioned: why is MS office excluded? I know a lot of companies that use MS O to manage there requirements (Design Input) and other crucial design steps. This is, in my eyes, one product quality related topic, isn't it? I would like to know your thoughts on this.</p>
<p>Thanks again!</p>Validation of compiler and IDE - Why, when and how to? - Part 3 - Mitchurn:md5:629a50a6ae3cd240eedef36f0b8d96f12015-06-01T17:27:36+02:002015-06-01T16:35:30+02:00Mitch<p>Alas, if mgmt wants it, or if an auditor once asked for it, do it as minimal as it can be.</p>
<p>Buy AAMI TIR 36 guidance and do it as they write it in this guidance for compilers. It shouldn't be a massive expense.</p>
<p>But IMHO, this is not a way to validate compilers. It's just a way to make people happy with regulatory requirements.</p>
<p>One last thing, it should be necessary to validate software development tools if they contain records related to regulations. For example, using a tracker tool, like redmine, to manage software requirements and bugs. In this case, yes, it should be validated.</p>Validation of compiler and IDE - Why, when and how to? - Part 3 - zedurn:md5:bc9136afb2c96f009584d56400e031312015-05-21T14:16:40+02:002015-05-21T13:16:40+02:00zed<p>Hi-</p>
<p>So if mgmt has decided that we must validate all software development tools (compilers, make, etc), how might I go about convincing them this is not in the regulation? It is a massive expense, and it seems it is all because an auditor once asked if we had validated our compiler.</p>
<p>Thoughts?</p>