New version of the FDA guidance on off-the-shelf software use in medical devices
By Mitch on Friday, 20 October 2023, 14:12 - Regulations - Permalink
The FDA released in August 2023 a new version of their guidance on off-the-shelf software (OTSS) use in medical devices. It’s worth noting that this guidance didn’t go through a draft version.
Something visible in the content of that guidance.
The recommendations present in this guidance are supposed to be following a least burdensome approach. But when you read the guidance, you wonder how this principle has been applied. The people who wrote this guidance are absolutely knowledgeable in medical device software, but the recommendations aren’t applicable to many software, if you try to apply them by the book.
The guidance begins slowly by aligning the documentation effort with the basic and enhanced documentation levels, found in the guidance on premarket submissions for device software functions. Then, things get complicated when the guidance dives into details.
This is where an extensive interpretation of this FDA guidance is necessary. This is a bit unusual to react like this in the columns of this blog. But this is actually what a manufacturer should do. Otherwise, the guidance is unapplicable.
Software failure in OTSS
Depending on the architecture, the data structures and data streams in your software, OTSS failure may or may lead to an unacceptable risk to the patient. We can borrow the approach of the IEC 62304, which allows placing software items (including SOUPs) in a class lower than the software system class.
Based on this approach, you can have OTSS leading to no or low risk (meaning acceptable risk), even in a software of enhanced documentation level. Thus, we can split OTSS in two categories (binary approach and not ternary approach as in the former guidance), with high risk OTSS and not high risk OTSS. “not high-risk” is clumsy, thus we use “low-risk” in this post.
Interpretation based on OTSS risk profile
The table below summarizes the interpretations that could be made to alleviate the burden of the recommendations, based on this binary approach. If the FDA doesn’t write clearly the least burdensome approach in their guidance, let’s help them in this task!
| Guidance Section | Interpretation high risk OTSS | Interpretation low risk OTSS |
|---|---|---|
| III OTS Software documentation elements | ||
| A. Description of OTS Software | ||
| 1. What is it? | No tricky question here, excepted: “Why is this OTS software appropriate for this medical device?” The answer could be based on the functions delivered by the OTSS, its maturity level, and the supplier evaluation. “What are the expected design limitations of the OTS software?” The answer should conclude that the design limitations don’t bring an unacceptable risk |
|
| 2. What are the Computer System Specifications for the OTS Software? | For high-risk OTSS, the validated configuration shall be clearly documented. Changes in configuration may lead to unacceptable risks. | The “complete list of any patches that have been provided by the OTSS manufacturer” can be hard to answer. Especially with OS. If the OS is a low-risk OTSS, it may be better stating that patches applied by the OS vendor don’t represent an unacceptable risk. Same thing for cloud software, where the configuration may not be fully managed but the manufacturer. |
| 3. How will you assure appropriate actions are taken by the End User? | For high-risk OTSS, these questions are ok. Unspecified or forbidden user actions may lead to unacceptable risks. | Questions about non-specified OTSS may be difficult to address, especially for SaMD. Such non-specified software shall not lead to unacceptable safety risks (Security risk remain anyway if the user can install malware). Then, the only way to address them is by labelling and, perhaps summative evaluation. |
| 4. What does the OTS Software do? | For high-risk OTSS, these questions are ok. Unspecified performance may lead to unacceptable risks. | “specify[ing] exactly which OTS components will be included in the design of the medical device” or knowing “the links with other software including software outside the medical device” can be difficult. Especially with SaMD, or OS. It may be better stating that such lack of configuration management at runtime doesn’t represent an unacceptable risk. |
| 5. How do you know it works? | No tricky recommendation here. For most of OTSS, knowing it works is based on the absence of bugs in test results. |
|
| 6. How will you keep track control of the OTS Software? | For high-risk OTSS, lack of configuration control may lead to unacceptable risk. However, “On startup, ideally, the medical device should check to verify that all software is the correct title, version level, and configuration” may be impossible in many cases where scarcity of hardware resources is the rule. The best way to address that is to sign software with state-of-the art cryptographic algorithms. When hardware is powerful enough. |
Implementation of measures “to prevent the introduction of incorrect versions” is not practically feasible in most cases with embedded software. It may be better stating that such lack of configuration management at runtime doesn’t represent an unacceptable risk. For SaMD, signing software is made possible or mandatory on app stores and shall be fostered. |
| B. Risk Assessment of OTS Software | No tricky recommendation here. Just do risk assessment on OTSS. | |
| C. Software Testing as Part of Verification and Validation | For high-risk OTSS, it may be relevant to have test plans and reports provided by the OTSS vendor. This means that open-source software without such documentation may not be adequate. Once again, the lack of documentation may not represent an unacceptable risk. |
For low-risk OTSS, the absence of documentation on OTSS tests doesn’t represent an unacceptable risk. Tests as those presented in the line below are enough to support this. |
| What is recommended is good testing practices. Most of times, it is not possible to test OTSS directly when it is integrated in the Medical Device Software (MDSW). Thus, qualifying an OTSS is most of times an indirect result of functional tests run on the integrated software. Providing a list of OTSS defects is necessary. They shall not represent an unacceptable risk. |
||
| D. Assurance of Development Methodologies and Continued Maintenance of OTS Software | “Provide assurance to the FDA that the product development methodologies used by the OTS software developer are appropriate and sufficient for the intended use of the OTS software within the specific medical device” This should be reserved to very high-risk OTSS, like a certified multitask real-time OS. See MAFs below. |
For 99.99% of OTSS (choose your number of 9 after the comma), the manufacturer has no clue of the product development methodology. If there is one! The recommendation in the left cell is absolutely unapplicable and shall be discarded for low-risk OTSS. Said in a diplomatic way: For low-risk OTSS, the absence of such documentation on OTSS doesn’t represent an unacceptable risk. |
| IV. OTS Software in Marketing Applications | ||
| A. Master Files for Devices (MAFs) | This should be limited to OTSS representing a high risk in case of failure. MAFs are a good solution for software vendor targeting the MD market. |
This will not be used by low-risk OTSS vendors (or vendors, which don't target the MD market). |
| B. OTS Software Changes requiring a 510(k) | No comment. | |
| C. Investigational Device Exemption and Changes to OTS Software | No comment. | |
| D. Premarket Approval and OTS Software | No comment on PMA procedure itself. The sentence “The extent to which the medical device manufacturer ensures that the OTS software was developed using appropriate life cycle control depends upon the overall risk of the medical device, the role of the OTS software, and the probable risk of death or serious injury” is in line with the principle used throughout this blog post. |
|
| E. Product Labeling | This is in line with question 3: “How will you assure appropriate actions are taken by the End User?” | |
Appendix A
Looking at the appendixes, the discussion on OS, drivers and utilities is interesting. In short, the FDA doesn’t recommend to have general public OS in a MDSW.
E.g.: it is possible (acceptable risk) to use Windows, MacOS, iOS or Android with a SaMD of basic documentation level. Likewise, it is acceptable with a SaMD of enhanced documentation level, when OS failure doesn’t represent an unacceptable risk. But it is hardly possible to have an industrial PC with such an OS inside a life-sustaining medical device.
The discussion is also interesting on the “excess of functionality” of general public OS. Sure! Imagine an OS “detecting” some medical image as a photo, and suggesting to share it on social media!
Appendix B
The examples given in this appendix are interesting. But curiously, OTSS documentation level is always attached to the device documentation level. Since all cases can be found in nature, a case like this could be added:
(3) A retinal diagnostic software device
Description: The device is limited to prescription use and incorporates an AI/ML-enabled algorithm that is intended to evaluate images for diagnostic screening to identify retinal diseases or conditions.
Example OTS Software 1: OTS AI/ML libraires.
OTS Software Documentation: A failure or latent flaw of these libraries can lead to a failure of the device software function(s), such as a diagnostic algorithm failure that provides a false result. These failures could present a hazardous situation with a probable risk of death or serious injury to the patient prior to the implementation of risk control measures. Therefore, since the device has an Enhanced Documentation Level, the AI/ML libraries OTS Software Documentation will follow the Enhanced Documentation Level in this guidance.
Example OTS Software 2: OTS cloud-based platforms and operating systems on local computers.
OTS Software Documentation: A failure or latent flaw of the OTS cloud-based platform and operating systems on local computers can lead to a failure of the device software function(s), where the failure modes are device unusable or not responding. These failures don’t present a hazardous situation with a probable risk of death or serious injury to the patient prior to the implementation of risk control measures. They only represent a risk of delayed diagnosis or treatment with minor consequences to the patient. Therefore, in the absence of unacceptable risk in case of OTSS failure, the OTS cloud-based platforms and operating systems on local computers OTS Software Documentation will follow the Basic Documentation Level in this guidance. Recommendations in sections III and IV of this guidance are given only for information for such Documentation Level.
Conclusion
This guidance is unapplicable.
Why? Manufacturers can't apply recommendations without interpretation.
Why? The recommendations aren’t technically / practically applicable.
Why? The guidance didn’t follow the draft guidance / public review workflow.
Why? Dunno, FDA, please check your SOPs.
Proposed correction: Release a new draft guidance for comments.
