How to externalize software development for a medical device?
By Mitch on Monday, 12 December 2011, 11:36 - Processes - Permalink
Case example
You are a medical device company and you want to replace your old products by “smart” ones. Your competitors are doing it and you customers want them. “Smart” means something like an tiny computer (its amazing how small they are), with an operating system and big software. For electronic stuff, you have your own engineers or you have your subcontractors, which you have been working for 20 years with. They knew how to design embedded software on microchips. But they don’t know how to design software of higher level, with intelligent behaviour and complex algorithms.
Foreword
This article reflects my point of view on how the development of software integrated in a medical device should be externalized to a 3rd party company. It is assumed that this company has a good knowledge of software development but no or poor knowledge of software development standards for medical devices. Especially, he doesn’t have knowledge of IEC 62304, the main standard for software in medical devices and ISO 13485 plus ISO 14971, generic standards for medical devices.
Back to our case
Alas! Your competitors already have so smart appliances … Pushed by the winds of change (very poetic, isn’t it?), your company decides to design its own smart appliances. And it is your job to manage a new subcontractor specialised in high level software, written in so-called C++ (say “C plus plus”) java or .Net (say “dotnet”) technologies. The project is going to last one year. Needless to say that this project will need a big commitment of you and your team.
By the way, if you forecast to launch a software development project, which lasts more than one year, this is too much. Consider splitting it into smaller projects with multiple versions of your software. This rule of thumb is not specific to software in medical devices.
Specifications
The software development process begins with specifications. In this phase, there is a lot to do with the end user or a group of experts like a medical advisory board. There are things you can’t delegate to your subcontractor.
First, risk assessment and conformity of the software to your statement of work. You may invite your subcontractor to risk assessment meetings. His ideas may be of great interest for risk assessment, based on his software knowledge. But the risk assessment activities shall remain your responsibility.
Second, usability and human factors. You shall manage usability specifications by yourself. You may invite your subcontractor to usability specifications meetings and reviews as well. Better, you should present mock-ups of software to end users, if possible, made by your subcontractor. But the usability specifications remain your responsibility.
Third, the overall behaviour of the software. The inputs, outputs, expected results, algorithms used, and so on, translated into specifications are your responsibility. Your subcontractor’s responsibility it to have the internals of software work to meet your requirements.
A crucial phase for you
The level of detail of the specifications depends on your own knowledge of software. If you have absolutely no knowledge of software, your specifications will be high level – aka “functional” specifications – Functional specification contain an intended use, scenarios of use and risk assessment. These specifications will be translated into technical specifications by a software expert of the subcontractor company. You shall have to a thorough review of these technical specifications, maybe with multiple iterations until you think that your subcontractor 1. has understood want you want, 2. is able to do it. Of course, the conformity of these technical specifications to the intended use and the mitigations actions issued from risk assessment is your responsibility.
Do it with the right person
A better solution is to hire someone to your company, who knows your job and is able to translate functional specifications into technical specifications. It may look like an expensive solution at first sight, but someone with the good knowledge can save time and money. I can bet that most of companies specialised in software development won’t have software experts with a significant knowledge about your industry. Only a very few subcontractors claim this high level of expertise. You’re lucky if you work with one of these companies.
Conception
The software development process continues with conception. This phase is a hard work for your subcontractor. It has to translate your technical requirements into software architecture. Conception is split in two sub phases:
- general conception, and
- detailed conception.
The first is like drawing the plan of a building and the last is like designing each room and water / electricity networks. For small projects, they are merged in a single phase.
A curcial phase for your subcontractor
Your subcontractor is certainly going to ask you some more information, while he dives into the description of the future software. Most of time, you will be able to answer to his questions: precisions about the specifications, explanations about your knowledge. But in some cases, it may be more difficult to immediately give the right answer. This happens often with input data like, say, an abacus in an excel sheet, medical imaging data. If you can’t, you are going to drag it along the rest of the project. It is very important to fix this kind of problem during conception or it may impediment the next phases.
Understand what he does
If you don’t have any knowledge about software, it may be very difficult to understand what your subcontractor does. The more the conception become detailed, the less the terms and methods used by your subcontractor become understandable. This may be actually a big concern for software with a high level of risk. IEC 62304 standard requires a very detailed conception for class C software. It is your responsibility to follow the standard and you shall know what your subcontractor does, even if you look like someone maniac to him. Once again, the best solution is to have someone in your company, who knows software development and is able to follow up the subcontractor. For software of lower risk level (class A and B), the standard doesn’t require to deeply document the detailed conception. In this case, this is not necessary to have a strong follow-up of detailed conception. Yet, I didn’t say that you don’t have to follow up what he does!
Coding
The next phase is called coding. In this phase, software engineers do strange things on their computers and, tadaa, at the end you have your software. Huh, not yet! There’s still a lot to do.
The tunnel effect
Coding may last a long time and you should avoid the “tunnel effect” during this phase. The tunnel effect happens when your subcontractor gives no news or doesn’t give significant news during a long time. You should have your subcontractor present to you the results of their work. A presentation every month for short projects (6 months) or every two or three months for longer projects (1 year or more) is a good frequency. It may require an active participation of yours to deliver input data or hardware or hardware simulators, to have your software work like in its future environment.
Prepare the tests phase
In parallel to coding, you have to prepare the tests phase. You have to define a test plan with your subcontractor and let him write this plan and the description of each test, which will be executed during the tests phase.
It is often necessary to have the user manual before doing any tests. Hence the user manual is a good way to understand what is in the tests. You shall write user/admin/maintenance manuals because they always contain information about risk mitigation actions like safety warnings. And because only you knows how to write and talk to your customer. But it is not easy to write a user manual of a software you don’t know perfectly yet. The solution is to let your subcontractor initialise the document with the minimum information, like the software workflows. This draft manual shall exist at the end of the coding phase. Verifying the content of a user manual is also a task to do in tests.
Verification:
Tests, tests and tests
Whereas you were little involved in the coding phase, you are going to be very involved in the test phase. Test phase may be very long and split in many tests sessions. Usually, tests are split in three phases:
- tests in subcontractor’s offices on its test platform,
- tests in your offices on your prototype or your test platform, and
- tests with ends users in semi-real or real situation.
The chain of tests
During the first phase, the engineers who have developed your software do the tests plan under your control. This step is important to become familiar with what is still their software – not for a long time – and soon your software. Most of time, the software is installed in a simulated environment. Hence the subcontractor hasn’t all the final hardware.
During the second phase, you test your software – at last, it is yours – according to the test plan. It is installed in your environment, with prototypes of hardware.
During the third phase, an end user tests your software, on a finalized version of your hardware. If the end user is someone understanding, you may let him follow the tests plan. Otherwise he will do free tests, with his own way of thinking how software works. On top of that, it’s important to choose an experienced end-user, who is able to detect remaining bugs.
Bugs bugs bugs!
Many bugs will be found and fixed in these three phases. Each phase is important, as the types of bugs discovered are not the same in each phase. In the first phase, basic bugs and bugs easy to find will be fixed, in the second phase, bugs risen by hardware interaction and some more complex bugs, in the last phase, bugs which require the knowledge of an end user.
These phases, especially the third, shouldn’t be confused with clinical tests. Clinical tests happen after verification.
Delivery
The delivery phase is very short compared to the verification phase. It basically aims at delivering the good version of the software, with its documentation and data, if any.
Delivery content
Your subcontractor should give you a CD or a DVD with your software in its final version after the tests, and everything, which is necessary to rebuild the software from code. It is your responsibility to store the content of this media in a well-defined place, with its documentation and any documentation you have done by yourself. This step is important. Hence in real life there is never one delivery but many deliveries. It’s an iterative process to find and fix bugs. Even after tests, some bugs may be found in very specific conditions, which were not met before.
Problem resolution
There are many bugs, most of them very minor, found during the use of your devices by end users. When almost bugs have been found, your company decides to launch a new version of your device. Of course, it contains new software functions, with bugs. For these reasons, one can say that software is never finished. It’s always evolving. For this reason, it’s important to track any bug found during the operational use of your software. A process was invented for this activity: problem resolution. Problem resolution means to you tracking any issue arisen by someone, which uses the device. During the development process, this is easy, since your company controls the use of the device prototype. But after delivery, you shall track any issue arisen by end users. This is typically the same process as post market surveillance, adapted to software.
Things to do
For that purpose, you shall maintain a list of all bugs, comments, and enhancements, issued by end users. And you shall have your subcontractor fix them periodically, say, twice every year. If the bug is major, for example, a software crash and device reboot happened in the hands of one end-user, you shall have your subcontractor fix the bug in emergency, to be able to deliver patches to your customers. By the way, your device shall contain a mean to patch software (eg usb slot + patch function from usb stick).
Validation
There is often a confusion between verification and validation. Whereas the verification can be assimilated to the tests phases of software, validation is the review of the whole medical device with software. Especially, validation is reviewing that the device is in total conformance with its intended use. Validation is a kind of final clearance. This is and will remain your job. Your subcontractor has nothing to do with it eventually.
What about the certification\?
The previous sections were pretty general and can be applied to any software development project. However, there are some specific tasks to achieve to have your software obtain the certification. Mainly, document all the tasks done during the project and demand to your subcontractor to document what he does. Documentation is the main mean in the hands of reviewers of public regulation organisations, to validate that everything was done the right way. Here are the documents that you and your subcontractor should write.
| Phases/Document | You | Subcontractor |
| Risk Assessment | Write Document | Analyse document (include risks mitigation actions in technical specifications) |
| Functional specification | Write Document | Analyse document (translate to technical specifications) |
| Technical specification | Review document (technical specifications fit your functional specifications) | Write document |
| Conception | Review document (conception fit technical and functional specifications) | Write document |
| User manual / admin manual / maintenance manual | Write document | Write draft |
| Tests plan | Review document (test plan really tests everything) | Write document |
| Tests report phase 1 | Review document (tests passed) | Write document (do tests) |
| Tests report phase 2 & 3 | Write document (do tests) | Analyse document (fix bugs) |
| Delivery | Review document (delivery contains everything and bug fixes are ok) | Write document |
You should not let your subcontractor write the documents his own way. To have him do it in your own way, you shall have your own templates and make your subcontractor follow them. Your templates shall especially fit the requirements of IEC 62304. Note that when I say this, this is a shortcut. Your quality system and conception procedures shall fit the requirements of IEC 62304. Associated to these procedures, you have documents templates, which in return you have to fill to adhere to the procedures.
You need all this documentation to compile the technical files you submit to reviewers of regulation organisations. This documentation allows reviewers understanding how your software was developed. Reviewers love knowing how software is developed, to reassure them on the fact that it is risks free.
Templates are a hard stuff to write, I regularly post some on my blogs. Have a look at the Templates repository page, you may find one interesting for you. If not now, maybe later on. And templates suggestions are welcome. Please give me feedback by mail.
Conclusion
If you follow the right standards and have the right procedures, subcontracting a software development project is not too hard a task. If I could give three advices (yes, I can't stop myself giving advices!),
- first choose the right subcontractor,
- second communicate a lot with him,
- third make him follow your own tailor-made process.
These advices are not very specific and you may think I state the obvious. In the light of standards for medical devices, this is really important. If you don’t follow these rules, you will hardly get the certification.
