Software in Medical Devices, by MD101 Consulting

IEC PAS 63621 2026 published

Mitch — Fri, 24 Apr 2026 15:58:00 +0200

IEC PAS 63621 2026 - Artificial intelligence enabled medical devices - Data management, has been published in March 2026.
A PAS is a Publicly Available Specification, a kind of early version of a standard. It was developed by IEC at a pace quicker than usual. Too quick to follow the long process of standard development.

This document is definitely a missing link in the chain of activities performed to design Medical Devices incorporating Artificial Intelligence (MDAI). This better explains why a PAS was published with a fast track process. MD industry couldn't be left without such standard.

This document describes a data management process for MDAI. Something we already explained in this post about data management.
The post was based on IEC 5259-x series. Fortunately, we retrieve the same steps in IEC 63621:

Data Requirements,
Data Planning,
Data Acquisition,
Data Development,
Data Provisioning,
Data Decommissioning.

The only difference: Data Preparation in IEC 5259-x is replaced by Data Development in IEC 63621. Data development is a process where data are prepared and verified against data quality criteria. This is already present in IEC 5259-x. The new provision present in IEC 63621 is an iterative process to ensure continuous improvement of data quality criteria.

The other main differences between are:

IEC 5259-x series is a full set of standards, covering various aspects of data management for AI.
IEC 63621 is focused on the core process, present in IEC 5259-2,
IEC 5259-x target any industry, requiring intensive interpretation to match medical device safety and performance goals, and personal health data regulatory requirements,
IEC 63621 target medical devices. Thus, little interpretation is needed to implement your own processed. But not more than any other MD standard,
IEC 63621 requires to make a link with risk management and identify if data can contribute to a hazardous situation. In such case, the process described in the PAS shall be followed.

Looking for ideas on how to implement this PAS? You can use the Data Management Plan template.

Claude Code: your new QARA team

Mitch — Fri, 20 Mar 2026 13:50:00 +0100

People say AI is going to replace their job.
This is not true. People using AI are going to replace people who don't.
My experience using Claude code from Anthropic perfectly demonstrates this.

Rebuilding the IEC 62304 template repository with Claude Code

A blog post by Panos Ipeirotis gave me the push I needed. He described how Claude Code can become your new teammate. Not a chatbot assistant you query, but like a team mate you hand a task to and who carries it through to completion, commit by commit, pull request by pull request.
I happened to have exactly that kind of task waiting.

For several years I have been publishing on this blog a set of software process templates for medical device development. Useful, but with good old MS Office files. Probably, too old. I had to convert them to something closer to the code. Something that software engineers can put in their repo and update while they update their code.

The idea was straightforward: take those templates, reorganise them as .md files into a structured Git repository. But I never had the time to do it. Then Claude Code came on the market. Able to do the repetitive job. It was time, I had no more excuse to procrastinate converting my templates.

Converting the templates: accelerated time

The templates were produced in three waves from 8 March to 19 March. Something that would have taken me months to do without Claude code. Between productive tasks, which bring money. Productive tasks first, templates when I have time.
Thanks to my new team mate, it took 2 weeks only.
Only one team mate? No, an infinity of team members doing the job for you. You can fire multiple branches at the same time, asking Claude different tasks. The only caveat is maintaining consistency between branches, by rebasing and merging. Not a big deal for software engineering teams.

As if I was operating in a relativist world, where time stretches in an uncanny manner for newtonian beings. My Claude team does the job and meanwhile, I can do something else. In my Claude rocket, it takes two weeks. On the earth ground, it takes ages.

I had an experience like this a looong time ago, with Redmine. A few of my clients used to place lots of documents in Redmine. Every change was managed thanks to Redmine workflow. Drawbacks: Redmine wiki was basic, its GUI was basic (though doing the job) and it has become a bit old, compared to more modern tools like commercial e-QMS and e-Documentation I won't quote here (this is enough with Claude!). Redmine was definitely not relativist!

Review and merge

Of course, you have to review what Claude does. But this is exactly the workflow, which sets the foundation of git paradigm. Somebody writes code. Somebody else reviews it. Here, Claude writes .md files, I review them and merge.

More, Github remembers every bit of changes in your files. It helps a lot in recording documentary changes in a change control process. Placing software documents as close as possible to source code also helps managing changes in agile iterations. It eases the verification of DONE status at the end of an iteration.
The closer the software documentation to the IDE, the better.

Methodic work

This was not like "vibe coding" — some fancy tool giving you an application with bells and whistles in one prompt (hello, fake prompt engineering). It was structured work, with explicit tasks, conventions documented in the CLAUDE.md file (for those who don't know, it is a kind of configuration file but written in natural language, giving general instructions to Claude), and a separate pull request for each deliverable.
It was fed in a RAG-like manner with my own internal repository of documents.

Claude Code managed file creation: I copied the content of doc templates as raw text. Claude converted them to md files. And that rascal actually had the nerve to give me some great ideas for improvement!
More, it had the ability to maintain consistency over time. Decisions made on 8 March — the structure, the naming conventions — are respected on the subsequent iterations, without explicit reminders. All of this thanks to that CLAUDE.md file.

Manually updating

The limits show up too. Several manual passes were needed to fix this and that throughout the documents. I could have written my instructions to Claude in a better manner, to limit these manual fixes. But that is inherent to iterative development. Claude Code is no exception to that rule.

The result: a library of 20 templates

Ten days and 34 pull requests later, the repository contains:

4 planning templates: project management plan, software development plan, configuration management plan, and the brand new data management plan for you AI models + a new review report,
1 software requirements specification + a new review report,
1 software architecture document + a new review report,
1 software design specification + a new review report,
1 new pull request review checklist,
1 software unit implementation and a new review report,
3 verification templates (plan, description, report) + a new test plan review report,
1 version delivery description,
1 a new release review report,
1 data management report (design phase).

Precision: they’re not fully AI-generated. The content is similar to the good old word files. Their conversion to .md files is AI-generated.

Theses templates are published on this public GitHub page: templates repository for MDSW. The templates are reusable with CC-BY-SA License.

Conclusion

Yannis's article was right: Claude Code changes the nature and the pace of your work. Building this repository by hand would have taken months. With Claude Code, two weeks were enough.

But the tool does not replace judgement. Claude Code executes, you take decisions. It does not make them for you.
I'm the manager, Claude is like an infinity of teammates.

This is the equivalent of coding with AI for Quality Assurance and Regulatory Affairs. This is AI-augmented QARA.

New Templates: Data Management Plan and Data Management Report

Mitch — Fri, 06 Mar 2026 13:32:00 +0100

After talking about AI in a series of articles, and especially about the data management process, we introduce two new templates: the Data Management Plan and Data Management Report.

Current templates present in the Software Development Templates Repository are adapted to classical software development and IEC 62304. But templates for data-driven design were missing. This loophole is fixed by adding these two new templates!

Here are the links to these templates:

They can be used:

For the data management plan: in parallel to / together with the software development plan,
For the data management report: it can be part of the software release, it is an artifact for both configuration management and design V&V.

I share these templates with the conditions of CC-BY-NC-ND license.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 France License.

New version of FDA guidance on Clinical Decision Support Software

Mitch — Fri, 23 Jan 2026 14:07:00 +0100

A new version of the FDA Guidance on Clinical Decision Support Software (CDSS) was published in January 2026. Here is a review of the new cases presented in the document and a short comparison with the status of CDSS with regard to the MDR.

This post is an update of the previous blog post of 2023, analysing of the previous version of the guidance.

Criterion 3

The FDA added new cases to the guidance. Here is a comparison of these cases with their MDR status. The "however" cases present in the FDA guidance are not taken here, they are obviously in the scope of MDR medical device definition.

Note 1: most of FDA examples below require the output be "reviewed, revised, and finalized by the HCP", or some similar language. There no equivalent consideration of review of the output neither in the MDR definitions, nor in MDCG guides, nor in the manual on borderline qualification.

Note 2: the guidance now adds the case where if software gives only one recommendation, then it is subject to "law enforcement discretion". When only one specific recommendation is given, the software is definitely an aid to diagnosis or treatment and thus matches the MDR medical device definition ; unless it points to an existing guideline (see one case below).

Software Function	MDR qualification
A software function that predicts risk of future cardiovascular events for an HCP to consider based on a patient’s weight, current and historical smoking status, blood pressure, and brain natriuretic peptide (BNP) in vitro diagnostic (IVD) test results.	This is an aid to diagnosis. This is a medical device software in class IIa or higher. It could be qualified as IVD device if the IVD test result is the main information used to output diagnosis.
A software function that creates a recommended treatment plan, including possible medication(s), for patients diagnosed with cognitive impairment for an HCP to consider based on the patient’s diagnosis related to cognitive impairment as well as potential comorbidities, age, sex, and patient preferences, and that should be reviewed, revised, and finalized by an HCP	This is an aid to treatment. This is a medical device software in class IIa or higher. Note that the software "creates a treatment plan". It doesn't displays an existing treatment recommendation from an existing guideline.
A software function that recommends a specific FDA-approved antibiotic agent for an HCP to consider based on the patient’s symptoms, recent hospitalizations, and previous antibiotic exposure.	This is an aid to treatment in class IIa or higher. In the EU, a software recommending an EMA-approved antibiotic agent is a medical device software.
A software function that analyzes a radiologist’s clinical findings of an image to generate a proposed summary of the clinical findings for a patient’s radiology or pathology report, including a specific diagnostic recommendation based on clinical guidelines that should be reviewed, revised, and finalized by an HCP	The proposed summary can be seen as clinical data communication and is not qualified as medical device. The recommendation based on clinical guidelines can be probably seen as "simple search", not MD. The Manual on Borderline and Classification V1.22 of 2019 contains a somewhat similar case at section 9.4. Qualification of software for interpretation of a guideline.
A software function that provides an HCP with a differential diagnosis based on a patient’s symptoms, vital signs, and laboratory values, and that, depending on the clinical context, may present either multiple diagnostic considerations or a single clinically appropriate diagnostic recommendation when alternative diagnoses are highly improbable. The output is intended to support clinical reasoning and to be reviewed, revised, and finalized by the HCP.	This is an aid to diagnosis. This is a medical device software in class IIa or higher.
A software function that classifies patients with chronic low back pain into a single recommended appropriate clinical care pathway (e.g., conservative management or referral to a surgical spine specialist) based on patient history, symptom duration, and documented clinical findings, where the recommendation is intended to be reviewed and acted upon by an HCP	This is an aid to treatment, in class IIa or higher. This is a medical device software.
A software function that estimates 90-day and 1-year postoperative mortality and complication risk following lung transplantation based on patient-specific clinical characteristics and published clinical evidence, where the output is intended to support pre-transplant planning and shared decision-making and is reviewed by an HCP.	This is an aid to prognosis. This is a medical device software. It may be in class I (no diagnosis or treatment) with a carefully crafted intended use .

Criterion 4

The FDA slightly changed the language about labelling of devices matching the fourth criterion. This change isn't relevant to exclude any software from the MDR definition scope. Likewise, the FDA added considerations about usability, automation bias, or timeliness of output data. Such considerations won't save any software from the MDR definition scope either.

Conclusion

The cases added to the 2026 version of the guidance are all medical devices according to the MDR medical device definition, excepted one. We retrieve the same situation as the 2022 version, where FDA cases are outside medical device scope in the US and inside medical scope in the EU.

TUV.AI Lab whitepaper on ISO 14971 - mind the gap ... or not

Mitch — Fri, 09 Jan 2026 14:09:00 +0100

The TUV.AI Lab published in November 2025 a white paper on gaps between ISO 14971 and risk management requirements present in article 9 of the AI Act.

Comparison of AI Act article 9 and ISO 14971

The main interest of this document, and the little gem it contains, is the table showing the gaps between risk management from the AI Act viewpoint and from the ISO 14971 viewpoint. The work is similar to the one performed on the comparison of ISO 13485 and AI Act already discussed here.

The comparison unravels both risk management frameworks are quite similar. Taking the paragraphs of the AI Act article:

2 gaps on (i) interplay with other AI Act requirements and (ii) on real-world testing,
5 partial coverage of AI Act requirements by ISO 14971,
10 full coverage of AI Act requirements by ISO 14971.

The biggest difference is taking into account fundamental rights in the risk management plan. A concept not present neither in ISO 14971, nor in MDR/IVDR.

Action plan

The white paper continues by stressing out the need of updating product risk management plans to fully implement risk management requirements of the AI Act. This task should represent a little amount of work, given how close the two frameworks are.

Thanks TUV.AI Lab, your work eases the interpretation of MDR/IVDR and AI Act.
However...

Indecent Proposal

The proposal for MDR/IVDR changes published on the 16th December 2025 put the cat among the pigeons. It contains this proposed change:

In Annex I to the Artificial Intelligence Act, MDR and IVDR are moved from Section A to Section B.

Why this change? The answer is in AI Act article 2.2:

For AI systems classified as high-risk AI systems in accordance with Article 6(1) related to products covered by the Union harmonisation legislation listed in Section B of Annex I, only Article 6(1), Articles 102 to 109 and Article 112 apply. Article 57 applies only in so far as the requirements for high-risk AI systems under this Regulation have been integrated in that Union harmonisation legislation.

Deciphering: In practice, MDs and IVDs are kicked out of the AI Act. Manufacturers are just required to apply article 57 in case of testing in a regulatory sandbox, and perform a surveillance of AI Act changes to see if they will need to apply something in the future.
Consequence: this is why regulatory sandboxes are introduced in the Proposal. This allows to apply AI Act article 57.

Why no more AI Act?

Two possible interpretations:

The will of simplification advocates to limit the CE marking process to MDR / IVDR. Removing medical devices from AI Act CE marking process is definitely a significant simplification.
The goals of the AI Act (basically, preserving the fundamental rights of EU citizens) aren't so far from those of the MDR / IVDR safety, performance and favorable benefit / risk ratio:
- One may say it isn't possible to present a favorable ratio, while adversely affecting fundamental rights,
- Likewise explainability or interpretability and trustworthiness or transparency of an AI model enhance the performance of a medical device.

Up to now, we have absolutely on assurance that the text will be kept like this. The change to AI Act Annex I may be discarded. Conversely, more concepts could be added to the MDR and IVDR, for example the inclusion of additional requirements related to AI in the General Safety and Performance Requirements.

There's no need to get lost in conjecture.
Keep calm and wait for the final text voted by the European Parliament.

Consequence for manufacturers: your first emergency action is to do nothing and wait for the final text. Slowing down the MDR/IVDR vs AI Act gap assessment and action plan is probably a quite good decision.

Proposal for MDR/IVDR changes and Rule 11 - software classification

Mitch — Fri, 19 Dec 2025 13:47:00 +0100

Lots of fuzz and buzz on the proposal for MDR/IVDR changes published on the 16th December 2025.
First, keep in mind this is a proposal.
Second, keep calm and wait for the version amended and voted by the European Parliament.

This doesn't prevent us to make a analysis of changes. Lots of consultants, hungry of new regulations, already did their homework! Let's add in this blog an additional voice to all the other ones already expressed!

New Rule 11 proposal

Let’s take our favourite subject: Rule 11. And cut it in swallowable chunks (gulp).

Confers a clinical benefit

Software which is intended to generate an output that confers a clinical benefit

Do you often use the word confer? I don't. The Merriam Webster dictionary, contains the following two definitions of the word confer:

to bestow from or as if from a position of superiority
to give (something, such as a property or characteristic) to someone or something

We assume that the first meaning isn’t applicable in our context. Thus, we may rephrase the first sentence of the newly drafted rule 11 as: Software which is intended to generate an output that gives a clinical benefit (to the patient).

The wording: confers a clinical benefit (or gives a clinical benefit) is puzzling. What does it mean: does it include direct benefit only or does it also include indirect benefit? This is a recurring subject for SaMD, discussed by people in charge of clinical evaluation.
Needless to say, this new rule 11 doesn’t simplify the debate.

Remark: a software, which doesn’t confer a clinical benefit, may be an accessory to a medical device, or a software driving or influencing a medical device, with technical performance only. The fate of accessories may be easier to determine with this new rule.

And is used for…

And is used for diagnosis, treatment, prevention, monitoring, prediction, prognosis, compensation or alleviation of a disease or condition

We retrieve here the first two bullets in the list of medical purposes of the medical device definition: diagnosis, treatment, prevention, etc. This covers the vast majority of software qualified as medical device, per the medical device definition.

But what about the other bullets of the medical device definition: investigation, replacement or modification of the anatomy, etc? They don’t fall into this rule? Where do they go to? Rule 1?

Class I by default

is classified as class I, unless the output is intended for a disease or condition

OK, if we have such software (In fact, almost any MD software is used for diagnosis, treatment and so on), it is class I by default, unless…
A big unless.

Unless, my dear Watson

Let’s continue with the sentences after unless. They are based on the wording of the IMDRF SaMD risk categorisation framework. We can try to fill in the SaMD Risk in the table 7.2 of this IMDRF document, also present (and tweaked) in the Annex III of the MDCG 2019-11 rev.1.

in which case it is classified as class III:

in a critical situation with a risk of causing death or an irreversible deterioration of a person's state of health;

Since, we don’t have more information about this critical situation (severity), all cases of significance of information are included. We obtain this:

in which cases it is classified as class IIb;

in a serious situation with a risk of causing a serious deterioration of a person's state of health or a surgical intervention,

Likewise, we don’t have more information about this serious situation (severity), all cases are included. We obtain this:

or to drive clinical management in a critical situation

Here, we have information about the critical situation (severity), with the expression “drive clinical management” (which confers (L.O.L.) a kind of probability). We can spot a single cell:

in which cases it is classified as class IIa

in a non-serious situation,

Likewise, we don’t have more information about this non-serious situation (severity), all cases are included. We obtain this:

And the last one:

or to drive clinical management in a serious situation or to inform clinical management in a critical or serious situation

We can spot three cells in the table:

All together

Merging all the cases in a single table, we obtain:

Comment 1

I don’t know if this clarifies something to you. For me, it doesn’t clarify anything.

Comment 2

As soon as we have software used for diagnosis, treatment and so on, which brings a clinical benefit, direct or indirect, we have software in class IIa or higher.

The cases after unless cover almost every case found in patient condition or clinical management:

For patient condition, is there something below non-serious patient condition?
- Negligible patient condition? (Funny patient condition? Interesting patient condition? I digress),
- More seriously, If the intended use of the software is for screening, could we say there is no patient (no one is supposed to be sick), so there is no patient condition? Thus, we are in class I?
For clinical management, is there something beyond Inform clinical management.
- Record clinical management? Communicate clinical management? That would place our software outside the MD definition.
- Support clinical management? Facilitate clinical management? This sounds stronger than inform. (Whisper clinical management? I digress too much).

Comment 3

As long as we don't have a rule, which follows one by one the cases of the IMDRF table, we are locked in class IIa.
Yet it's so easy to follow the cases in the classification table. Here is the transcript:

Software which is intended to be is used for diagnosis, treatment, prevention, monitoring, prediction, prognosis, compensation or alleviation of a disease or condition:

is classified as class I, if the output is intended to drive clinical management in a non-serious situation or to inform clinical management in a serious or non-serious situation,

is classified as class IIa, if the output is intended to treat or diagnose in a non-serious situation or to drive clinical management in a serious situation or to inform clinical management in a critical situation,

is classified as class IIb, if the output is intended to treat or diagnose in a serious situation or to drive clinical management in a critical situation,

is classified as class III, if the output is intended to treat or diagnose in a critical situation,

in all other cases, it is classified as class I.

It's as simple as pie. But what were they thinking when they wrote this proposal? Why beating around the bush like this?

Conclusion

This new proposal is definitely not a game changer. It is definitely a missed opportunity of simplification and of more reasonable classification for low-risk software.
Rule 11 rules.

I’m rule 11, I eat MD software. What do you expect me to eat?

Artificial Intelligence in Medical Devices - Part 5 Cybersecurity

Mitch — Fri, 21 Nov 2025 13:59:00 +0100

After safety risks in the previous post, we continue this series of posts on AI with Cybersecurity risks.

Cybersecurity is vast subject. It can be separated in security at IT level (the organization) and security at product level, in our case: the medical device. Product-level security can be split into training / validation / test phases before release, and deployment / use phases, after release.

Remark: data and model cybersecurity requirement is present in the 5th intent of article 15 of the AI Act.

What doesn't change and what changes

AI or not AI, some characteristics remain:

Safety impact: while a loss of confidentiality or availability is a concern, it doesn’t necessarily lead to safety risks. Conversely, a loss of integrity will most probably lead to a safety risk.
Cybersecurity risk management process and threat modelling: either for IT security risks or for product security risks, the processes and methods don't change: ISO 27005 (or another framework) for IT and AAMI SW96 for medical device product.

Conversely, AI brings some new items, hence new challenges:

Assets: the presence of vast amounts of data, and the presence of AI models.
Attack scenarios exploits the unique statistical and data-based nature of ML systems. Thus, AI models deserve their own taxonomy:
- For security experts, the Mitre Atlas (Adversarial Threat Landscape for Artificial-Intelligence Systems) is a tool similar to the Mitre Att@ck® framework, to identify possible attack scenarios on AI systems.
- And the NIST AI 100-2e2023 Adversarial Machine Learning document, by the National Institute of Standards and Technology (NIST), defines a possible taxonomy of methods of attacks. New attack scenarios are continuously discovered, new adversarial techniques could be detected and documented in an update of this document.
- More simple than the NIST document, the OWASP Machine Learning Security Top Ten lists the top 10 security issues of machine learning systems.

Especially, the NIST document on Adversarial Machine Learning describes what the attacker might do:

During the training stage: Taking control of assets,
- The training data, their labels, the model parameters, or the code of ML algorithms,
- Resulting in an impact on integrity, a technique called data or model poisoning,
During the production stage: sending crafted data to the AI model,
- Resulting in an impact on data or model confidentiality or privacy, with techniques called membership inference and data reconstruction, and techniques called model extraction or prompt injection for generative AI,
- Or resulting in an impact on model output data integrity, with techniques called model evasion, and techniques called model abuse, or prompt injection (again) for generative AI,
Last, during the production stage: flooding the AI model with multiple queries,
- Resulting in an impact on availability, the classical denial of service (DoS), or with some more specific techniques like energy latency DoS, or prompt injection for generative AI.

IT-level risks

These risks won't necessarily be present in a product risk management file. They should be placed in a IT-system risk management file or even a broader organisation-level risk management file.

These are the risks that a breach in the IT system leads to an impact on AI data or on AI models. Data can be training, AI validation, testing or performance monitoring data.
Since the breach happens in the IT system, it may happen on:

A development platform,
A production platform,
The maintenance / back-office management / administrative IT system.

Since we are at the IT level, the risks are on a the infrastructure, not the products. Thus, the mitigation actions are at IT-level as well. Starting with a cybersecurity policy put in place by the manufacturer.

IT security management system (ISMS)

The only way to fully manage these risks on data and model designs stored in the IT system of the manufacturer (or some contractor) is to apply an ISMS framework: ISO 27001, SOC 2, NIS2 Technical Implementation Guidance, CIS or else.
An embryo of ISMS can be put in place by following the requirements set out in IEC 81001-5-1 clauses:

4.1 Quality Management,
5.1 Development environment security, and
5.8.4 Controls for private keys.

On the product itself, new cybersecurity risks emerge from AI models. We've seen above that lots of attack scenarios are already well-defined for AI models, including gen-AI and LLMs. Thereafter, we leave IT-level risks and consider product-level risk in the different phases of the medical device lifecycle.

Risks in training / AI validation and test phase

Before release, the risks can be identified from the possible adversarial attack methods identified in the NIST document. These attacks are based on the capabilities of the attacker to control assets (model parameters, source code, training data, labels, and test data) seen above.

For example, on the development platform:
Confidentiality: attackers managed to get access to data or models (model design and data, stored on the development platform).

They may disclose or sell data or ransom the data holder (the manufacturer or data provider) not to do so.
No impact on safety. But possible economic consequences if the manufacturer was holding personal health data used for e.g. AI model tests.
- Possible impact on device availability if the recovery actions on the IT system require to disconnect the device in production (But this is not linked to the fact that an AI is present).
- Mitigation: for such scenario, mitigations are more at IT level (securing the assets on development platform), rather than product level (product specifications) or product design process (software design SOP).

Availability: attackers erased data either at will or by side effect of their attack:

The manufacturer may not be able to release a new product or a new version of their product.
An impact on safety might be possible if a security or safety fix is required, but the manufacturer is unable to release it.
- Mitigation: at IT level, data archiving, data replication.
- Mitigation: at product design SOP level. Data sanitization steps and a design review during the design process might be a way to catch such problems.

Integrity: The attacker corrupted models or data either at will (e.g.: supply-chain attack, and more specific for IA: data poisoning or model poisoning) or by side effect of their attack.

If the corruption isn’t detected, this may lead to an erroneous product on the market and a significant risk on patient safety.
- Mitigation: once again, for such scenario, mitigations at IT level, like data archiving, are a first layer of protection.
- Mitigation: A second layer of protection in the product design SOP level can be added: one or more data sanitization steps or design reviews to check that the data or model haven't been corrupted.

Remarks:

The economic impact isn't fully discussed in the above examples. Only the safety impact is fully discussed.
Risks on data can be exacerbated by the amount of data, and the nature of data with potential privacy impact (fully identifiable, pseudonymised or anonymised).
The line may be blurred between IT-level risks and product-level risks, especially for supply-chain attacks. Such risks may be present in the product cyber risk management file or the QMS process risk management file or the ISMS risk management file.

We've seen also in the above discussion that mitigations are more at IT level (securing the development platform) or process level (adding reviews and checks in the design process), rather than mitigations at product level (e.g.: product security specifications or test cases specific to the product interfaces). This assertion is true as long as AI models in medical devices are locked in production. Hence, data poisoning in production is unlikely to happen, if no continuous training is possible.

Risks in production / use phase

After release, the risks can be identified from the possible adversarial attack methods identified in the NIST document. These attacks are based on the capabilities of the attacker to query the system in a crafted and adversarial manner.

For example, in production:
Confidentiality: attackers managed to get access to data or models (privacy leak by techniques like model extraction or model inversion or prompt injection).

They may disclose or sell data or ransom the data holder (the manufacturer or data provider) not to do so.
No impact on safety.
Possible impact on device availability if the recovery actions require to disconnect the device.
- Mitigations: they can be placed at product level, with safeguards in the pipeline around the AI model, or when the medical device is a module of a bigger health software platform, in the non-regulated health software part.

Availability: attackers send DoS attacks, such as energy latency attack.

The device may be unavailable.
Usually, no direct impact on safety. Unless the unavailable device is used in critical care.
Side effect: it may not be possible to monitor the device performance.
An impact on safety might be possible if performance monitoring is unavailable.
- Mitigations: like confidentiality, mitigations can be in the medical device or outside the medical device. We would prefer a mitigation in the device pipeline, if the attack scenario can lead to a safety impact.

Integrity: The attacker corrupts models or data either at will or by side effect of their attack, or, the attacker sends crafted adversarial data able to fool the AI model (techniques like model evasion, model abuse or prompt injection).

Corrupting models or data by techniques like data poisoning or model poisoning isn't likely in the context of locked-in models, used in medical devices. However, some vulnerabilities might exist, to do so, even if the model is supposed to be locked (imagine an AI model capable of continuous learning in production, with only a poor boolean set to false to lock it...).
Crafting adversarial examples is an attack scenario more likely with locked models.
If the corruption isn’t detected, this may lead to an erroneous product on the market and a significant risk on patient safety.
- Mitigations: like in the availability case above, mitigation can be in the medical device or outside the medical device. And we would highly prefer a mitigation in the device pipeline, if the attack scenario can lead to a safety impact. One example of mitigation in the pipeline would be a subnetwork trained to detect adversarial examples.
- It is also possible to have mitigations in the training of the AI model, by adding adversarial examples to the training dataset, in order to make the AI model robust to adversarial attacks. But such techniques are still subject to research efforts and can also decrease the model accuracy.

Methods to identify risks

As usual with risk management, the biggest challenge is to be confident in the correct identification of all significant risks in your risk management file. For AI cybersecurity risks, methods available to identify risks are:

The aforementioned NIST Adversarial Machine Learning document,
The OWASP Machine Learning Security Top Ten.
- It's a good way to start and try to assess the likelihood of those most frequent scenarios in your device,
- It has the immense advantage of giving the possible mitigation actions for each of the 10 scenarios,
Likewise the OWASP Top 10 for LLM applications is oriented to threats specific to LLM.

Mitigation actions

AAMI SW96 Clause 7.1 Security risk control option analysis prefers inherent security by design to mitigation actions added to the medical device or information to the user. That it is rarely the case with AI models, as already discussed in the previous post on safety risks in MDAI.
We've seen above that this concern is repeated for security risk mitigation actions:

Data sanitization, design reviews, adversarial test cases (e.g.: defensive distillation, which unfortunately comes at the price of high computational costs, decrease of model performance) can be merely seen as safety by design (design process), because there is rarely an option to add something into product design specifications,
Some safegards on model inputs or on model outputs may be seen as inherent security by design, but the effectiveness of such measures is called into question by the NIST document,
Mitigation in the IT system can be seen as mitigation actions added to the device or added to the manufacturing process,
A last type of mitigation action is the surveillance of the AI-model performance. though necessary to detect problems in production, it cannot detect problems before they arise. Surveillance is definitely not inherent safety by design, rather something closer to protective measures added to the manufacturing process

As usual in risk management, there is no silver bullet for AI attack scenarios. The effectiveness of mitigation actions preventing threat scenarios is then generally called into question, in the current state-of-the-art (late 2025). Thus, for cybersecurity risks with a safety impact, like model evasion or model abuse, the poor effectiveness of mitigation actions (once again in the current state-of-the-art) makes it questionnable to incorporate AI models in safety critical medical devices.
Fortunately, and this is the case for lots of medical devices, AI model can be used in medical devices with AI-related risks are of low or moderate profile.

Conclusion

Security risk management for AI in medical devices is not so far from what we know for classical software. At least in the method. It is however radically different in the possible attack scenarios. Hence, it requires people with a background in adversarial AI techniques to manage AI security risks the right way.
Up to know, there is no established standard or guidance, specific to medical devices on this subject. This is why we referred to documents, mainly from NIST or OWASP.
No such draft or project of standard is present in any of well-known repositories (IEC, AAMI, FDA...). This may change in the near future as state-of-the-art is being built.

Final FDA CSA guidance released

Mitch — Fri, 24 Oct 2025 15:06:00 +0200

The final version of the FDA guidance on Computer Software Assurance was published in September 2025, three years after its draft version.

Here are the changes, compared to the draft version, already reviewed in this post.

Coud computing, Saas, Iaas, Paas

A section titled Definitions, not present in the previous version, was added. It clarifies the concepts of Coud computing, Saas, Iaas, Paas. These definitions are important to understand the discussions in the further parts of the document.

Especially the FDA clarifies that such software can be part of a quality management system, thus in the scope of software to be assessed for validation. Likewise, and not surprisingly, the FDA includes artificial intelligence systems in the scope.

Conversely, the FDA also clarifies that such software may not be part of a quality system, hence not in the scope of software to be validated. This is a recurring discourse of the FDA in this document. Some software won't require a validation because they're outside of the QMS scope.

We retrieve a new example of Saas in the last section of this guidance, illustrating the importance of taking into account such systems in the CSA process.

21 CFR part 11

The guidance stresses out the need to apply a risk-based approach on software validation versus part 11 requirements. This was not so well clarified in the previous version. We retrieve this recommendation twice:

In V.A.(1) FDA recommends manufacturers focus the assurance effort on the features or functions relevant to the integrity of the records and 21 CFR Part 11 requirements applicable to the records intended to be stored.
And at the end of V.B: This guidance recommends that manufacturers base their approach to computer software assurance on a justified and documented risk assessment and a determination of the potential of the system to affect product quality, patient safety, and record integrity.

Remark: 21 CFR part 11 is extremely difficult to apply by the book. It is in some cases technically impossible to apply it when the software wasn't designed for part 11. The FDA's solution to make use of a risk-based approach is therefore highly recommended!

High-risk and not high-risk

This is not a change: the FDA maintains its definitions for these two categories. It however clarifies what kind of risk-based approach the manufacturer should use:

High process risk: the software failure may lead to a quality problem compromising safety.
- the manufacturer should identify the assurance activities commensurate with the medical device risk.
Not process high risk: the software failure won't lead to a quality problem compromising safety. Some other quality problems are possible.
- the manufacturer should identify the assurance activities commensurate with the process risk.

Vendors evaluation

This is a new sub-section in the final guidance. While recognizing that manufacturers may have limited access to information from the software vendor as part of an assessment, the FDA recommends to assess the vendors capabilities, with methods like: audits, review of accreditations or certifications, review of their SW development practices, etc.

Like any other task present in this guidance, software vendor evaluation effort should make use of a risk-based approach.

Leveraging digital records

The FDA also recommends to leverage any existing digital record output of a software system. E.g.: system logs, audit trails...
In opposition of old-school (yet, still useful) paper and screen-shots, the FDA accepts such digital records as evidence of validation.
Manufacturers may leverage automated traceability, testing, and the electronic capture of work performed to document the results, reducing the need for manual or paper-based documentation.

Conclusion

No revolutionary change in this final guidance. It was mainly updated to clarify the recommendations. But also to add new examples of the digital age. Should we expect a new version of this guidance including example of CSA for artificial intelligence systems?

Artificial Intelligence in Medical Devices - Part 4 Risk Management

Mitch — Fri, 17 Oct 2025 14:03:00 +0200

We continue this series of articles on Artificial Intelligence with the Risk Management Process.
Simply put, risk management for MDAI isn't dramatically different from risk management for classical MDSW. Yet, some interesting points of view can be found in literature and standards. Namely AAMI TIR 34971 and FDA guidances on AI. The future ISO TS 24971-2 on risks with AI should be published in 2026.

What doesn't change

Let's imagine we have a legacy device with a classical algorithm. We want to place on the market a new generation of this device with AI models.
The risk management process doesn't change:

Risk management steps (risk analysis, evaluation, control, and so on) remain the same,
Your risk management file templates remain the same.

The risk modelling doesn't change:

Software risk stems from software failure, be it classical or AI,
Combination of severity and probability of occurrence of harm, as well as your tables for probability levels / severity levels remain the same,
Hazardous situations (software failure, refined in several possibilities like false positive and so on), and harms (specific to your device intended use), remain the same.

That's not very exciting! Let's continue.

What changes: risks on data

With AI models, we have:

New technology = new failure modes,
New failure modes = new sequence of events.

Examples of risks related to data

Classical algorithm: a value out of bounds may lead to a floating-point overflow or underflow, the software crashes,
AI model: a value out of bounds may lead to an irrelevant result, the software doesn't crash and still gives a result completely out of it.
In both cases, a mitigation action could be to filter input values out of bounds. However, for AI with many input parameters (features), defining "out of bounds" may be challenging.

Data-driven design

As we've seen in the last two articles, AI design is a data-driven design. Thus, lots of risks stem from poor data quality or bias in data. Quoting the content of AAMI 34971, here are some possible sources of risks:

Quality of data:
- Incorrect
- Incomplete
- Subjective
- Inconsistent
- Atypical
- Thus, it is necessary to define data specifications and data quality criteria,
Bias on data:
- Selection bias
- Non-normality
- Proxy variables
- Confusion variables
- Group attribution bias
- Experimental / medical practice bias
- Thus, it is necessary to stratify, analyze, rework data to avoid such biases.

This is exactly what we've seen previously in the data management process, with the following steps (amongst others):

Defining data specifications,
Defining data quality criteria,
Preparing data,
Reviewing data.

Process-level risks

These problems on data quality and biases are primarily mitigated by a proper data management process established into your existing QMS. Practically speaking, this is the daily work of data scientists and people with a clinical or scientific background to manage data to avoid these problems.
We could even assert that such risks are more mitigated by processes, rather than by product requirements.

In classical software development, developers can introduce bugs into software code. This is mitigated by a proper software design process.
In AI model design, data scientists and subject matter experts can introduce problems in data used for AI training, validation and test. This is mitigated by a proper data management process.

Data-level risks

Some risks can be attached to data. Thus, they can be seen as data risks, instead of product risks.

Examples of data risks

In design:

Data Risk: Limited representativeness of data
- Mitigation: collect multi-centric data, mono-centric data is possible with a rationale
Data risk: imbalanced proportion of data with regard to classes
- Mitigation: documenting stratification of data and sample data at random in each stratum to divide cross-validation sets (cross-validation stratification).
Data Risk: Different disease prevalence in datasets collected from multiple centers.
- Mitigation: balancing the prevalence by either deleting true negatives or adding synthetic true positives.
Data Risk: inter-expert variability. For a given dataset, some experts may classify a data item in class A, whereas some others may opt to class B.
- Mitigation: define a parameter to quantify agreement between experts (very poor, poor, good, very good) and set a threshold on the agreement level. When the agreement is below the threshold, assign a new class “undefined” to the data item.
Data Risk: bias in the data representativeness of the patient population, in favour of elderly versus paediatric for a given intended use addressing a large population. But with more severe consequences for paediatric than for adults.
- Mitigation: ensure a minimum of paediatric cases in the datasets, especially the AI test dataset.

In production / post-market:

Data Risk: Data drift when the device is feed with real world data.
- Mitigation: ensure performance monitoring is planned for the MDAI placed on the market.
- Mitigation: plan retraining of MDAI to release a new version every year.

Remark: All these risks and mitigation can be recorded in a new section named "Data" in your risk assessment matrix.

What changes: risks on user interfaces and man-machine interactions

Conceptually, the FDA requires to assess whether human + AI is better than human alone. With a risk-based approach, we also have to determine if we can identify new risks in the case of human + AI+
With AI models, we have:

New sequence of events, with man-machine interactions. Users may react differently when they are confronted to an AI,
Devices with AI may also have a degree of autonomy higher than classical devices. This may lead to cases where users can hardly control the device or even monitor the device behavior.

Examples of risks related to UI

Classical algorithm: the software algorithm is a rules engine integrated in a robot. It always triggers the same action / gesture for data comprised into a predefined range / envelope,
AI model: the AI model is a ML model integrated in a robot. It may trigger similar action / gesture for data comprised into a predefined range / envelope. But not exactly the same, leading to rare loss of accuracy,
Users confronted to the robot with AI may place too much trust in the AI and may overlook this problem of accuracy.

What changes: risks on generative AI

Some MDAI with generative models have recently emerged in the medical device world, like:

Models to generate custom-made implants by reconstruction of anatomy from medical images.
Bots built on LLM, making use of specific bibliographic repository (RAG), to generate medical advice from patient-specific data.

With generative AI models, we have:

New technology = new failure modes, once again,
New failure modes = new sequence of events.

Examples of risks related to generative AI:

AI model: the AI model generates customs-made implants by reconstruction from medical images.

Risk: It may generate an anatomically wrong shape of implant.
- Mitigation: Surgeon shall review and manually adjust the shape.
Risk: It may generate spikes on the shape.
- Mitigation: a classical algorithm “surfaces” the shape to remove spikes.
Risk: It may generate shapes without appropriate mechanical resistance.
- Mitigation: a probe is 3D printed to test the resistance.

Remark: if some classical algorithm exists with the same intended use, we may retrieve these risks. There is no real new sequence of events in these examples above. What's new with generative AI algorithms: possibilities never seen with classical algorithms.

AI model: the LLM-based AI model generates medical advice from patient-specific data:

Risk: the model confabulates some wrong advice, 10% of time.
- Mitigation: Challenge the LLM result. A second LLM, from a different supplier, is asked if this advice is right. If there is a discrepancy of results between the two, the first LLM is rerun with a modified prompt ("are you sure, my little fancy tiny sweet LLM that this is ...?").

Remarks:

This is really a new sequence of events. Classical algorithms, like rules engines (however also named AI, but more traditional) would not be subject to such random failure rate.
In this example, what is the effectiveness of the mitigation?
- Could both LLM confabulate at the same time? Let's say 10% x 10% = 1%.
- Is a wrong result once every hundred advices acceptable? This depends on the intended use:
- Probably "yes" for a daily nutrition assistant,
- Probably "no" for a drug prescription assistant (what a bad idea!).

More classical risks and mitigation actions

Like in classical algorithms, we can have some kind of overarching mitigation action, when the AI model fails. This AI failure is simply a software failure at system level:

Risk: AI system detection algorithm result is irrelevant
- Mitigation 1: the AI system outputs a detection result and a confidence index.
- Mitigation 2: a software supervisor controls the AI system output, and discards it when the result is out of a clinically relevant range.
Risk: AI system failure, embedded in a robot
- Mitigation 1: software, when the AI system outputs irrelevant data, a supervisor fallbacks to a less complex but more reliable classical algorithm.
- Mitigation 2: hardware, circuit breaker in case of safety compromised by both AI and classical software.

Precedence of mitigation actions

ISO 14971 requires to seek for mitigation actions by order of precedence, starting with inherent safety by design. Finding such mitigation for AI model can be challenging:

A proper data quality assurance can be seen as inherent safety by design.
We could object that this is more a remote kind of safety by design. Data quality assurance is supposed to ensure a better trained model. But there is no direct relationship between data quality assurance and the safe behavior of the MDAI. At least with current technologies.
Conversely, filtering AI system outputs with a classical (and deterministic) algorithm is a kind of direct inherent safety by design.
There is a noticeable difference of effectiveness between data quality assurance (supposed effectiveness demonstrated only with statistical AI test data) and the traditional algorithm (effectiveness demonstrated by classical scripted testing methods).

A proper data management plan and a proper MDAI development and V&V plans are necessary to ensure de release of a safe and effective device. This is the only obvious actions for inherent safety by design we have at hand. And this is unfortunately the inherent limitation of AI ML technologies built on statistical models.

For Protective measure, we are more on a ready-made path:

Human oversight is seen as a protective measure (E.g. in AAMI 34971).
This is more realistic to place such a mitigation action as a protective measure. Inherent safety by design shouldn't require a human action.
Another protective measure may be an explanation of the AI output. A LLM quoting its sources is a kind of protective measure. The user has to verify the sources by themselves.

For Information to users, we are also on a ready-made path, but with some peculiarities:

A kind of information to user can be describing in the IFU the boundaries within which a safe use of the AI is possible.
As we've seen previously, AI models may use a lot more parameters than classical algorithms. they can have behaviors difficult to anticipate in case of real-world data being outside the envelope parameters. And there may be no way to systematically check if parameters are in a predefined enveloppe. Explaining how the device can be used and within which boundaries may be the only possible mitigation action.

Information to users also contributes to the requirement of AI model transparency. While we may object that transparency is a concern for any algorithm, it is exacerbated by AI models, validated within a given envelope of parameters.
Thus, information to users can be almost seen has a mandatory mitigation action. This is why in its guidance on AI-enabled MD, the FDA requests manufacturers to document a model card.

Explainable AI can also be seen as a mitigation action. Is it inherent safety by design or a protective measure? Since, the explanation is presented to the user, we are more in the case of a protective measure. A bit like a warning pop-up in a classical software.

Conclusion

Risk management for AI in medical devices is not so far from what we know for classical software. At least in the method. Some new sequences of events are possible, coming from the specific characteristics of AI. The future standard ISO TS 24971-2 will contain a questionnaire on these characteristics, helping manufacturers in their identification of AI-related risks.
This future standard will be a significant update of the current AAMI 34971. We've seen above that this topic is still subject to discussions. State-of-the-art of risk management for MDAI is still being built.

Artificial Intelligence in Medical Devices - Part 3 Data Management

Mitch — Fri, 29 Aug 2025 13:47:00 +0200

We continue this series of articles on Artificial Intelligence with the Data Management Process.
This process is required by article 10 of the AI Act, but also from a MDR viewpoint to document the process feeding the AI design process (seen in the previous article part 2) with the right data.

Basically, the right data = the right software (right performance and safety).
The aim of this process is to start from real-world, not sanitized, or unstructured data. At the end, we want to obtain prepared data, ready to use in AI design phases seen in part 2.

You can use this article as a bootstrap to write your own data management process and / or procedure.

Responsibilities

Like any other process we start by describing responsibilities:
For such process used in MDAI, data management but also standards and regulatory compliance need to be managed. An example of allocation of roles may be:

The chief data scientist or data scientist is in charge of this process.
The quality and regulatory manager is in charge of the relationships with the ISO 14971 risk management process and the regulatory conformity of the data management process.
The chief clinical officer is in charge of clinical relevance of data.
The chief security officer is in charge on data security and privacy.

Process overview

The process described below is inspired from the data management process presented in ISO 5259-1 and collaterals.

The data management process is an iterative process allowing to increase the quality of data at each iteration. It aims to deliver a dataset (processed data in the figure below) to the AI model design process (in green).

The arrow from "Data provisioning" to "Prepared data" symbolizes the data delivery by the data management process to the AI design process. The data management process may deliver data to the AI model design process at early stages, before data review, for dry-runs or when agile methods are used (dotted arrow in the diagram).

Please refer to part 2 about AI design process for a better explanation of the right part in green on this diagram.

Here are some details on these process steps.

Data requirements

When you develop software, you start with software requirements. When you manage data, you start with data requirements.

Inputs

Applicable regulations on data,
- AI Act article 10, especially avoiding to have a negative impact on fundamental rights or lead to discrimination prohibited under EU law,
- Other local regulations like GDPR, HIPAA, to name a few,
Device intended use, and device description,
High-level product requirement or functional requirements, or system requirements,
Software requirements.

High-level requirements or software requirements may not be present or fully completed when the data requirements stage starts. These requirements feed the determination of data requirements, and vice-versa. Especially, the feasibility of collecting some data will have an influence on high-level requirements.

Activities

The data requirements stage involves:

Determining what data are required for the MDAI design, according to the device intended use, target disease, and target population,
Checking the availability of the data, especially clinical / personal health data, the presence of existing registries, the need of retrospective study or clinical investigation,
Checking the appropriateness of data, in order to avoid possible biases that are likely to affect the health and safety of persons,
Checking if data may have a negative impact on regulatory requirements,
Determining if anonymization or pseudonymization is required (spoiler: yes, most of time),
Initializing a data quality model with data quality characteristics aiming to represent in a fair and accurate manner the target medical condition and the target population without bias, including sub-groups.

Outputs

Data Requirement Specification, with:
- Features: Clinical data or other data being considered as having an influence, input of AI model inference,
- Target(s): Clinical or other data being considered as representing the clinical outcome or a contribution to the clinical outcome, output of the AI model inference,
- Traceability to product requirements, functional requirements, system requirements, or software requirements, as appropriate,
Report on data availability,
Draft Data Quality Model:
- High-level inclusion and exclusion criteria,
- Number of data items and a justification why this will be sufficient.

For example:
In medical imaging, data requirements may define features as the types of images and sequences needed, and the targets as some reports or some post-processing overlay accompanying the images. The data quality model may be initialized with the number of DICOM studies needed according to some clinical parameters.

In the analysis of medical reports, data requirements may define features as medical data that shall be present in the PDF files extracted from these reports. The data quality model may be initialized with the quality of information present in these reports, matching expected medical data.

Data planning

Once you have your data requirements, you can plan:

How you’ll get data,
How data will be structured.

Inputs

Data Requirement Specification.

Activities

The data planning stage ensures that:

The data to be used meet the requirements of the data requirements stage,
More generally, the clinical goals of the AI model design project will be met.

This stage involves:

Designing the data architecture (e.g: design data so that the architecture can match the clinical parameters required to represent the target disease and target population),
Defining how data will be used:
- Dataset for training or tuning,
- Dataset for AI validation,
- Dataset for AI test,
- Dataset for device validation (if different from AI test),
- Other dataset, if any….
Defining the data acquisition plan:
- Either acquiring anonymized data, or planning anonymization or pseudonymization,
- Obtaining data from off-the-shelf registries or databases (secondary use of clinical data) maintained by third-parties,
- Or performing a retrospective clinical study (secondary use of clinical data),
- Or using clinical data from a register built in PMCF of another manufacturer’s device (secondary use of clinical data),
- Or performing a clinical investigation.
- Documenting the data retention time, location and protection measures.
Defining a data Quality Model, describing the expected quality measures used to ensure:
- The presence of clinical and technical inclusion and exclusion criteria using relevant attributes,
- A satisfying representation (in terms of coverage, accuracy etc.) of clinical data amongst the target disease and target population,
- Compliance to regulatory requirements on data, the absence of bias, including in subgroups, and the absence of impact on safety, performance, and human rights.
A discussion on the influence of the type and location of planned data acquisition has on the data.

Outputs

Data architecture, such as tabular format, or other format,
Data acquisition plan,
Data Quality Model.

For example:
In medical imaging, data architecture may define precisely the expected structure of DICOM files, the tags needed, the content of overlays, the content of DICOM structured reports. The data quality model may define the voxel size, the slice thickness, the presence of mandatory tags…

In the analysis of medical reports, data architecture may be a simple tabular format, containing information extracted from pdf files, the range of data found in columns, enums representing possible values in some columns. The data quality model may be defined from the presence or absence or relevance of information present in medical reports.

The influence of the type and location of planned data acquisition may be based on the characteristics of the local population where data will be acquired.

Data acquisition

Inputs

Data architecture,
Data acquisition plan,
Data Quality Model.

Activity

Data acquisition is performed according to the data acquisition plan (no surprise). Anonymization or pseudonymization is also performed according to the plan.

Collected datasets are tested against a subset of data quality requirements, if some of these requirements can be tested prior to data preparation.

Outputs

Acquired data,
Data acquisition records, according to data acquisition plan,
Data quality requirements preliminary testing report, including the discussion on the influence of the type and location of data acquisition had on the data.

Data preparation

Data preparation aims to prepare (groom) data to obtain datasets that can be used in the AI model design project.

Inputs

Acquired data

Activities

Various preparation techniques are used to prepare data. Not all preparation techniques may be used for a given project.
One or more Data Preparation Reports log operations performed, and if required, rationale for such operations, and results.

Outputs

Prepared data
Data preparation report

Some commonly used preparation techniques are described below.

Aggregation

Aggregation consists in merging one or more datasets in a single dataset and removing duplicates. Assembling research cohorts can be seen as data aggregation.

The concept of duplicate may not be data items strictly containing the same values. In this case, the removal of duplicates may require a validation by a person with clinical background or other relevant background.

Sampling

Sampling consists in randomly selecting data items in very large datasets, to obtain a dataset of size manageable by the AI model design project team.

The sampling method may be present in a statistical analysis plan and/or may require a validation by a person with a statistical background or a clinical background.

For example:

For diseases with a low incidence, an over-sampling of positive cases may be required to have more positive cases in the dataset, and better train an AI model on these cases,
For data used for clinical validation, the sampling method may require a review by persons with a clinical background to confirm the right representation of the target population.

Imputation

Imputations consist in:

Replacing a missing value by a default value in a data item, or
Deleting the data item.

The choice of the default value depends on the nature of the data. In some cases, it may not be possible to set a default value. Then, the data item shall be deleted if the missing value is relevant (see features selection) for the AI model training.

The choice of default values may require a validation by a person with clinical background.

Cleaning

Cleaning consists in several possible operations, like:

Shifting values, e.g.: from [10..19] to [20..29],
Replacing values by others e.g. {“Woman”, “Teen”, “Baby”} by {“Adult”, “Pediatric”, “Pediatric”}

The choice of the cleaning operations depends on the nature of the data. The choice of operations to be performed may require a validation by a person with clinical background.

Outliers’ treatment

This step consists in detecting outliers and treating them (deleting, replacing).

Detecting outliers may require methods such as:

Visualization, e.g.: with boxplot diagram, scatter diagram,
Basic math treatment on data (min, max, threshold, average…),
Splitting data in quartiles, and keeping data within a min and max thresholds,
Advanced statistical analysis on data.

The choice of the operations used to detect outliers depends on the nature of the data. The choice of operations to be performed may require a validation by a person with clinical background and/or statistical background.

Features Creation

This step consists in creating new features that can capture information in data more efficiently than the original features.

The decision to create features depends on the nature of the data. The operations to be performed may require a validation by a person with clinical background.

Impact analysis: the new features may require the update of the data requirements and data acquisition plan.

Example:

For time-series data, a new feature may be the average of one attribute over a given period of time.

Labelling and annotation

This step consists in generating meta-data representing information relevant for the AI model training.

Labelling and annotation can require methods such as:

Manual annotation by qualified personnel,
Automated annotation by on-purpose tool (in-house developed or off-the-shelf).

The choice of the operations used to generate meta-data depends on the nature of data and meta-data. The choice of operations to be performed may require a validation by a person with clinical background and/or statistical background.

Difference between annotation and labelling: Annotation is a general term; it can be done on any data. E.g.:

Automated segmentation of images or automated placement of bounding boxes,
Adding contextual information like patient information or device used to acquire data.

Labelling is usually performed on dataset in the case of supervised training, associated with target variables:

E.g.: Classification of images: yes or no (for binary classification).

Annotation and labelling operations shall be performed by qualified persons. E.g.: persons with clinical background or other relevant background on the device, the intended use, the technology.

If data annotation is automated (most of cases) the data annotation tool may require a validation according to the QMS SW validation procedure.

Example:

Manual generation: Radiologists annotate DICOM files with regions of interest.
Automated generation: An in-house trained LLM is used to review physicians’ reports (unstructured data), to automatically annotate data with a categorized clinical outcome (structured data).

Features selection

Features selection consists in finding which data are the most relevant in the data model.
E.g.: for data represented in tabular format, which columns are relevant or useful in the table, to train the AI model.

Conversely, features selection can be also performed by finding which data have little or not effect on the AI model.

Features selection usually consists in the following operations:

Deleting data, when variance is below a given threshold,
Selecting data with best independence, using a statistical test like chi square test,
When the model allows to get parameters, pretrain a model with the dataset, e.g.: a linear model like a Stochastic Gradient Descent Classifier.

Data dependence may be represented with heatmaps.

Note 1: Since feature selection aims to enhance model performance, results are usually an increase in model performance.
Note 2: If data is deleted or put apart, it may be necessary to remove this data from the data requirements and data acquisition plan.

Augmentation

Data augmentation consists in adding synthetic data to a dataset in order to better train an AI model.
The augmentation method may require a validation by a person with a clinical background.

For example:

An AI model is intended to be used with photos taken by patients with their mobile phone. Data augmentation may be necessary to generate photos with poor quality (low contrast, wrong exposition, various angles) to better train the AI model.

If data augmentation is automated (most of cases) the data augmentation tool may require a validation according to the QMS SW validation procedure. Tip: such tool is usually a low-risk QMS software.

Encoding

Encoding consists in operations to convert data not represented by a number to data represented by a number:

Converting labels to numbers, e.g. {“Adult”, “Pediatric”} to {0, 1}
Converting dates to numbers, e.g. the number of days since 1970-01-01.

Normalization

Normalization consists in setting all data in range 0…1 or -1…1. It allows to ease computations during training.

Data Normalization usually consists in the following operations:

Centering data on their average and dividing data by their maximum value in the dataset, e.g.: blood pressure values from 5…18 to 0.28…1,
Centering data on zero and dividing data by standard deviation, e.g.: blood pressure values from 5…18 to -1…1,
Splitting data in 4 quartiles and selecting quartiles around the median.

Data normalization may be represented with distribution curves before and after normalization.

Train and AI validation Split

Train and AI validation dataset, split the dataset is to parts. For example:

80% for training,
20% for AI validation.

Document as appropriate the splitting operation and ratio, if specific operations were performed or if unexpected events happened.

Data review

This step is a formal review of datasets prior to their provisioning and use in the AI model design project. This step may be planned for each dataset at distinct milestones or design reviews.
For example: the data review of the AI test dataset will usually happen later than the data review of the training / AI validation dataset.

This review shall ensure that:

Data acquisition and data preparation steps have been performed according to planned provisions,
Datasets meet the data quality requirements, especially inclusion and exclusion criteria,
Datasets meet the regulatory requirements,
If limitations exist, limitations on datasets shall not have an impact on safety, performance and regulatory requirements, nor introduce a bias.

Data provisioning

Data provisioning consists in applying the prepared dataset to the AI model being design. The dataset shall be:

Identified,
Versioned,
Put into configuration management,
Accompanied with descriptive statistics summary,
Provided with metadata allowing the verification of dataset integrity. E.g.: SHA checksum

The data quality issues detected when applying the dataset to the AI model are recorded in the data preparation report. They can be used as feedback to update the data requirements, planning, acquisition and processing.

Remark: augmented data may not be put into configuration. Rationale for not putting into configuration augmented data is usually based on the random generation nature of these data.

Data decommissioning

Input

All datasets

Activities

Data decommissioning consists in:

Either destroying data in a safe manner,
Or archiving data for a retention time defined according to regulatory requirements (with planed destruction)
Or transferring data to another department, entity, organization,
Or returning data to the data holder.

Output:

Data decommissioning report

Relationships with risk management processes

We end this process description with its link with the risk management process.
More precisely, wrong data is a hazardous phenomenon from a risk management perspective. The data management process aims to deliver the right data. Thus, it is a kind of risk control measure at QMS process level. Not product level.

Of course, risks shall be monitored throughout the data management process. This is essentially done by ensuring a good communication with the risk management team, with participation of risk management team representative in:

The review of data requirement specification,
The review of and data acquisition plan,
The data review itself.

Risk management is the subject of the next article. We'll see how to manage risks specific to MDAI.

TÜV AI.LAB White Papers - whack a mole game

Mitch — Thu, 14 Aug 2025 13:29:00 +0200

The TÜV AI.Lab published two white papers on the interaction between the AI Act and the MDR:
Better Safe Than Sorry? ISO 13485 and the EU AI Act
and:
AI Act and MDR - A match made in heaven or double the trouble.

Some serious content behind the itching titles.

What is TÜV AI.Lab?

The TÜV AI.Lab was founded in October 2023 as an independent joint venture of the TÜV companies: TÜV Süd, TÜV Rheinland, TÜV Nord, TÜV Hessen and TÜV Thüringen.
Its mission, present its website is (excerpt):
The TÜV AI.Lab takes on the task of implementing social and regulatory requirements, e.g. the EU AI regulation, into test criteria and processes and to accompany the development of standards for the testing of safety-critical AI applications.

Knowing the problems that Notified Bodies are going to face with the AI Act, this venture is welcomed.

Note: to my best knowledge, only TÜV Süd, Nord, and Rheinland are designated for the MDR. Thus, this venture also addresses other products in the scope of the AI Act.

Problems and Solutions

These white papers adopt the German way:

Listing and analyzing problems,
Giving existing solutions,
And, if necessary, giving a plan to find other solutions.

ISO 13485 vs AI Act

Problem

There is no standard to implement a QMS compliant to AI Act requirements.

Existing solution

The white paper ISO 13485 and the EU AI Act contains a traceability matrix between AI Act article17 on AI management system and ISO 13485 clauses. The traceability also gives a coverage ratio of ISO 13845 to fulfil AI Act article 17 requirements.

This is the first little gem of these white papers.

Use it to drawn you gap analysis and your conformity transition plan for your QMS, from ISO 13485 + MDR QMS to ISO 13485 + MDR + AI Act QMS.

We see in this traceability matrix that the biggest gap is on Article 17.1.f. It requires to establish a new data management process into our existing ISO 13485 QMS.

Plan to find other solutions

This white paper mentions the future harmonized standards on QMS, from the CEN/CLC/JTC21 working group, as future source of information. It also mentions ISO 42001 as another solution to close gaps found in the traceability matrix. More than seeking an ISO 42001 certification, MDAI manufacturers should use ISO 42001 as a tool, as a reference, to find information on how to fill the gaps.

For JTC21 standards, we have to wait for the final version of these standards. Since they will match AI Act requirements for any impacted industry, they may be too generic for MDAI manufacturers, compared to MDAI standards being cooked by some other working groups.

AI Act vs MDR

Problem

AI Act contains lots of new requirements, applicable to MDAI manufacturers.

Solution

The white paper AI Act and MDR contains a comparison showing what MDR and AI Act have in common. This is quite a lot. Of course, lot of gaps remain. To do so, the paper contains an assessment of impact of AI Act requirements on High-risk AI systems: Articles 10 to 15.
There is nothing like an annex with General Safety and Performance Requirements (GSPR) in the AI Act. Articles 10 to 15 are a kind of list of GSPR applicable to the product.

This gap analysis is the second little gem of these white papers.

Use it to drawn your gap analysis and your conformity transition plan for your MDAI technical file, from MDR CE mark to MDR + AI Act CE mark.

Remark: the pyramid representation of MDR classes and AI Act risks is a questionable choice. There is no such thing as a correspondance between the two systems.

Plan to find other solutions

Like in other white papers previously reviewed, this one contains more questions than answers: Definition of safety component, human supervision, sandboxed testing, substantial changes, and continuous learning systems.
Finding solutions will be a long journey: a close cooperation and dialogue with all stakeholders.
Unfortunately, this sounds more like wishful thinking than concrete actions. Manufacturers still remember how unbalanced was any kind of dialogue with other stakeholders.

Welcomed work

Of course, TÜV AI.Lab’s work in welcomed. But so much needs to be done.

It looks like a Whack-a-mole game. As soon as a problem is fixed, a new one pops-out. This looks also like a contest between Notified Bodies, Authorities, Commission, AI Bureau and MDCG. Everyone listing their own problems and possibly giving their own solutions.

This isn't going to stop soon. When you see how many known unknowns we have in the AI Act. Plus all the unknown unknowns to discover.
Of lot of work for TÜV AI.Lab.

Artificial Intelligence in Medical Devices - Part 2 Design

Mitch — Fri, 01 Aug 2025 14:41:00 +0200

After introducing this series in the previous post, we continue with MDAI design.

AI design is different from SW design. That's why IEC 62304 in its current version doesn't help for AI design. It requires a lot of imagination to interpret it for AI design. Especially for architectural design, detailed design, and matching unit tests and integration tests.

If we take the diagram from the previous post, we can see that AI design is a subset on MD design:

Obviously, if the MD is GUI-less and only made of an AI system, the AI design is almost all of the MD design. Some software items remain necessary to integrate the AI system in a usable medical device. Generally, a pipeline for interoperability of the AI system with other components internal or external to the MD. But, even with these software items around the AI System, the AI design is the task requiring most of the time in the whole software design.

AI design overview and steps

Specifications

The first step of AI design is common to any kind of software: software requirement specification.
In this step, we can rely on IEC 62304 clause 5.2.2 Software requirements content, like:

AI functions and performance,
AI inputs and outputs,
etc.

Especially, AI performance specification may contain criteria, like sensitivity, specificity criteria.
These software specifications may already be expressed at product level. E.g.: it may be possible to know and define sensitivity early at product level. In this case, the software requirement will be simply a copy of the product requirement, allocated to the AI system.

Architecture

We continue then by designing an architecture of the pipeline containing the AI model, and the AI model itself. Here is a basic example of architecture, where the AI system is a pipeline on the server side, containing one or several AI models:

Borrowing IEC 62304 language, the AI model itself is a software unit, and the pipeline containing one or more AI models is a bigger software item.
Of course, your AI architecture can be different, with or without pipeline, with or without a foundation model, or something else totally different. The idea of the architecture is to be still able to pinpoint the AI system(s) in the software architecture.

During AI design, the architecture of the AI model itself may be made of several candidate architectures, with different model topologies, and pipeline steps. Only one will be selected, usually at the end of AI validation.

After architecture, we continue with the core of AI design. It is made of:

AI training or tuning,
AI validation,
AI test.

For machine learning, these steps replace detailed design, and unit tests found in IEC 62304.

There can, and will, be iterations between these 3 phases, to obtain the best model, according to the specifications. Like architecture, we can have different loss functions, or reward functions, and hyper-parameters tried during training iterations.

These steps can be seen as the detailed design step of a larger design process, with AI model integrated in a larger architecture.

Note: this is not fully true for the AI test phase. AI test phase is also named AI performance test. This can be seen as a step making a major contribution to the validation of the device itself.

Here is a more detailed description of AI training, validation and test:

AI training or tuning

Input:

AI model specifications (it can be a subset of SRS).
A list of candidate models designed by data scientists.
Dataset for training.
A list of hyper-parameters, a loss function or a list of possible loss functions, to be set by data scientists.

Activities:

Train all (or most of all, if a model really outperforms predefined specification criteria) candidate data models with training dataset.
From a process viewpoint, the training method can be any suitable method for the models. It can be documented in the software development plan.

Output:

Trained models.
Chosen loss function and hyper parameters set for optimal performance.

AI validation

Input:

Trained models.
AI validation plan with acceptance criteria, derived from AI model specifications.

Activities:

Validate models with validation dataset.
Model validation techniques will differ, depending on the type of model. The technique used for a given model is documented in the AI validation plan.
Typical validation technique is cross-validation. These techniques will be described in the future IEC 63450 standard.

Output:

One or more best trained model, and its (their) hyper-parameters and loss function(s),
AI validation report.

AI test

Input:

AI model Test plan with acceptance criteria, derived from AI model specifications,
One or more best trained models,
Test dataset, separated from training/validation dataset.

Activities

Validate the performance of the best AI model with test dataset,
The techniques to perform AI tests will be described in the future IEC 63521 standard.

Output

Best model,
AI model Test report.

We will discuss this AI test phase in a future article about MDAI verification and validation.

Integration in MD design

The three AI design steps above can be integrated in a broader MDSW design, according to IEC 62304 + (IEC 82304-1 SaMD or IEC 60601-1 electromedical device), we obtain a kind of big picture like this:

Classical software detailed design, coding and unit testing are replaced by the AI model design seen above.

AI design & development plan

IEC 62304 2nd edition will contain a clause on AI design and development plan. This new version is expected by mid 2026.
Provisions on AI design and development plan can be included in the software development plan. According to what we've seen above, it can include the steps specific to AI: training, validation and test. The description of these steps given above can be used as a bootstrap to write provisions in your own design and development plan.

And what about the datasets

We see here that almost everything stems from data. For each step, we need a dataset:

Training and AI Validation dataset, usually split into 80% for training and 20% for AI validation,
Test dataset.

Having the right datasets is of utmost importance, to reach the performance objectives defined in the SRS. Thus, data deserve their own data management process.
We'll see that in the next article.

Confiance AI Body of Knowledge

For readers wanting to go deeper into AI design subject, the Confiance AI foundation established what they call a Body Of Knowledge. It models the activities for the engineering of a critical ML-based system. Interesting for defining your own processes when you design a safety-critical software system or software + hardware system.

Artificial Intelligence in Medical Devices - Part 1 Introduction

Mitch — Fri, 25 Jul 2025 14:33:00 +0200

We start here a series of articles about artificial intelligence (AI) in medical devices (MD).
This series will consider mostly machine learning (ML), the most widespread type of AI. The one, which gives remarkable results, compared to classical algorithms.

Concepts

Some concepts are needed before continuing.
Artificial Intelligence in Medical Device is having AI Systems inside a medical device. The medical device can be a hardware electromedical device or a SaMD.
We see sometimes the acronym MDAI for Medical Device with Artificial Intelligence (not to be confused with AIMD - Active Implantable MD).
The nested items in the diagram below show that AI System is only a part of the medical device. However it may contribute significantly to the intended use of the medical device. This is more that often true for SaMD. This may be less true for electromedical devices.

Regulations

As usual, we remain focused on US and EU regulations. (even though this may be restrictive for some readers).

USA

There is no specific regulation on AI in the USA, at federal level.
The US FDA relies on the definition of device software function in the section 201(h) of the FD&C Act. This definition is quoted in the FDA guidance on Artificial Intelligence-Enabled Device Software Functions. The safety and performance of MDAI shall be validated, like any other medical device.

EU

Likewise, the EU MDR and IVDR don't have any specific requirement on AI. The safety and performance of MDAI shall be validated, like any other medical device.
We also have the AI Act. On top of the existing MDR / IVDR regulations. Bringing additional requirements to MDAI CE marking.
This is already discussed here and here.

Guidance

USA

The FDA website has a dedicated page on AI in SaMD.
Several documents have already been published. Reaching a momentum with the FDA guidance on Artificial Intelligence-Enabled Device Software Functions, already quoted above.

EU

the EU started their stream of guides, with a first MDCG 2025-6 & AIB 2025-1 document, already discussed here. It is focused on MDR/IVDR & AI Act interactions.
On a pure MDR/IVDR side, and more practical veiwpoint, the Team-NB also published a checklist on AI, already discussed here.

Other countries

We can partially complete this list with documents from other countries:

MHRA web page on Software and artificial intelligence (AI) as a medical device,
Health Canada Pre-market guidance for machine learning-enabled medical devices,
TGA web page on Artificial Intelligence (AI) and medical device software.
And the overarching IMDRF Good Machine Learning Practice for Medical Device Development.

Standards

The list of standards for MDAI is still being constructed.
We've already seen a long list of standards referenced in the Team-NB questionnaire on AI. (See here). Most of those standards aren't specific to medical devices.
Fortunately, some standards, more specific to MDAI, are being cooked by dedicated working groups. None of the standards listed below have been published yet:

IEC 62304, of course, in its 2nd edition. It will contain new clauses on AI development plan,
IEC 63621 on MDAI data management,
ISO 24971-2 on risk management in MDAI, a kind of fork of AAMI 34971,
IEC 63450 on MDAI verification,
IEC 63521 on MDAI performance validation,
IEC 62/540/NP on MDAI PMS (document in very early stage, no attributed standard number yet),
And also IEC 62366-3 on MDAI usability.

We are going to see in the next articles how these future standards will shape the medical device lifecycle processes.

AIB 2025-1 - MDCG 2025-6 - Welcome to the AI Act, MD manufacturer

Mitch — Fri, 11 Jul 2025 14:09:00 +0200

The Artificial Intelligence Bureau (AIB), in charge of some regulatory aspects of the AI Act, and the MDCG published in June 2025 a common guidance. This guidance is a FAQ, it will be updated when new questions arise.

Compared to both the Team-NB position paper, and the BSI white paper, we don't find contradictory information on how to apply the AI Act for medical devices.

Introduction

The introduction of this guide outlines the definitions of economic operators found in the AI Act and makes a comparison with MDR/IVDR economic operators.
MDR/IVR manufacturers are AI Act providers. No surprise

Deployers

The concept of deployer is new. No such equivalent in the MDR/IVDR. A user isn't necessarily a deployer.
This is understandable for a Healthcare Center. If it makes available a Medical Device Artificial Intelligence (MDAI) to healthcare personnel, then it is a deployer. And healthcare personnel are users.
However, the definition of deployer incorporates natural or legal person. To my best knowledge a physician is a natural person.
Following the definition of deployer, if private practitioners buy a MDAI and use it in their day-to-day practice, then, they are at the same time deployers and users.

It implies they shall comply with the obligations set out in article 15. Good luck European and National Authorities to have private practitioners follow AI Act obligations for deployers, and maintain appropriate records!

Unless a guide is produced on the subject, sprinkling down these obligations on manufacturers, pardon, providers. Urging them to put in place the means in their MDAI or in information provided to deployers, or in their QMS, to let such natural persons meet these obligations seamlessly.

Least burdensome approach

Borrowing this expression from the FDA, this guide confirms that the least burdensome approach is fostered by the AI Act. No need of a new QMS or a new tech file, or new a risk-management process or a new PMS process.
Just incorporate in the existing ones the additional provisions applicable to MDAI.
We retrieve this approach in several questions and answers throughout the guide. No surprise, the AI Act already says that.

High-Risk MDAI and safety component

This guide rephrases the content of AI Act article 6, shedding a new light on this article with the use of MDCG 2019-11 guide. It adds a table showing when a MDAI is a high-risk AI according to its MDR/IVDR regulatory class. Even though it may seem obvious for some readers, it's better to have it in black and white (not black and white actually. There are colors in the table!).

It's also better to have in black and white that the risk classification of the AI Act doesn't change the MDR/IVDR regulatory classification scheme. Even though it may also seem obvious for some readers.

However, we are no further ahead in defining "safety component":

Can we have a MDAI in class I MDR or class A IVDR, containing a safety component? Thus, high-risk AI?
Conversely, can we have a MDAI in class II+ MDR or class B+ IVDR, where the AI isn't a safety component? Thus, low-risk AI?

A guide, like the MDCG 2019-11 or the manual on borderline classification would be welcome.

Substantial Modification / significant change

The concept of substantial modification is an autonomous concept under the AIA says the document. And: the Commission will develop guidelines on the practical implementation of the provisions related to substantial modification.
Will they publish a document adopting the charts we have in MDCG 2020-3? Wait and see.

Fortunately, we have a little time before this kind of guide is required by AIMD manufacturers. Contrary to hardware, software can be changed every other day. This guide should be at the top of the pile of AIB guides.

Do it like the FDA PCCP

The guide mentions the possibility to have what the FDA calls a Predetermined Change Control Plan: high-risk MDAIs that continue to learn after being placed on the market or put into service have the possibility of having pre-determined changes plan checked at the time of the conformity assessment
Changes according to this plan won't be considered as substantial modification.

This is really good news! There is absolutely no equivalent requirement in MDR / IVDR. The AI Act brings us on a platter something we should have expected for years, from a skittish MDCG.

Needless to say that it would be easier for manufacturers, if the fundamentals of a PCCP we find in FDA guidance are reused in EU guidelines. A common ground on PCCP guidance is too much to ask for? Maybe IMDRF could do something.

MDAI that continues to learn

The guide contains this:
For high-risk MDAI that continues to learn after being deployed the post-market monitoring system is key to ensuring continued performance and compliance.

Woohoo, MDAI may continue to learn! Calm down. It doesn't mean it can continue to learn in production. This has never been done up to now. The knowledgable FDA didn't manage to put together a policy on MDAI continuously learning in production. How would MDCG and AIB come with a magic wand and authorize this?

Continuously learning is going to be limited to MDAI in pre-production, for a long time. A discrete V&V phase is and will be required for a long time as well, before releasing a version in production. Unless a scientific breakthrough is made, allowing to continuously monitor and validate a MDAI in production.

Data governance

We retrieve in the guide a message similar to the one present in the two BSI and Team-NB white papers on data governance.
This guide sheds a light on the correspondance between requirements on clinical/performance data in MDR/IVDR and requirements on training, validation and testing data in the AI Act. A new data management process, different or adapted from existing clinical data management, is needed to match AI Act article 10 on data governance. It aims to eliminate biases on these data.

We also retrieve here a message similar to data management requirements present in the FDA guidance on AI enabled device software functions.

The AIB guide also puts the emphasis on GDPR compliance. Thus, this data management process shall also ensure compliance to GDPR or other EU regulations on data.

It also mentions that CEN/CENELEC Joint Technical Committee 21 is working on the developing a harmonised standards on data and bias.
Such standards would be welcomed! However, there is a risk that they will be one-size-fits-all for any AI system in the scope of the AI Act. Thus, either the standardisation committee will add specific clauses or notes for data used in MDAI (optimal solution) or such clauses or notes won't be present, and MDAI manufacturers will have to interpret them for their case.
These standards will probably inspired by the ISO 5259-x series, referenced by the TEAM-NB questionnaire on AI in MD. For example:

ISO 5259-x series have clauses on data requirements (data requirement specification, mirroring SW requirement specification),
These new standards will for sure have clauses on data requirements, similar to those in ISO 5259-x. They may add clauses ensuring that data requirements don't break fundamental rights (something common to any AI system, not present in ISO 5259-x),
But these new standards may not have clauses specific to the scientific validity of data used in MDAI. Something that we find in the IMDRF - FDA document on SaMD clinical evaluation.

Transparency

Once again similar message on transparency, compared to the two other documents.
Whom is transparency for:

First and for all, for users, to be able to use the MDAI, knowing its performance and limitations, its advantages and drawbacks. This comes with the classical documentation:
- IFU, labelling,
- Disclosure of residual risks,
- And the model card, as recommended by the FDA,
Also for deployers, with the same aim as users, but capable of determining if the use of the MDAI is appropriate for users. This is detailed in AI Act article 13:
- Detailed instructions for use.
- The technical level of these IFU for deployers may be higher than what is given to end-users. Especially when end-users are lay persons.
We can add for regulatory authorities, to be able to assess MDAI safety, performance, and compliance:
- Answers to Team-NB checklist,
- Recommendations in FDA guidance on AI enabled device software functions.

Transparency goes hand in hand with trustworthiness. Manufacturers must provide information in their documentation (transparency), in particular IFU and accompanying documentation, so that professionals can assume their responsibility when using the MDAI, and lay users can feel confident in using the MDAI.

In-house MDAI

According to article 5 of MDR / IVDR healthcare centers and healthcare professionals can put into service a medical device in their organization, without requiring the intervention of a Notified Body. This is applicable also to MDAI. Thus, according to article 6 of the AI Act, an in-house MDAI is not a high-risk AI system.

This is really good news for healthcare professionals, who can develop MDAI without any intervention of Notified Bodies.

Conclusion

As expected, this kind of guide won't change the world. But it brings some reassuring messages to MDAI manufacturers. AI Act will be for sure an additional burden. But not so much, compared to MDR / IVDR.

(Imagine the mood of other industries, like education, worker's management, law enforcement, ... who have to implement the AI Act from scratch).

2025 update of FDA Premarket Cybersecurity guidance

Mitch — Fri, 04 Jul 2025 14:11:00 +0200

A new version of the FDA guidance named Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions was published in June 2025.
The body text had minor updates: clarifications, updated references to standards and guides, editorial corrections. Note that this 2025 guide mentions AAMI SW96, which was not cited in the previous version. This was because SW96 had not yet been published and recognized by the FDA in 2023.

Section VII

The major change is the addition of a new Section VII.
This section explains how to apply the requirements of Section 524B of the Federal Food, Drug, and Cosmetic Act (FD&C Act) concerning the cybersecurity of connected medical devices.

Cyber devices

These MDs are called “cyber devices” by the FDA. The definition of cyber device includes in its scope a device that is connected, even indirectly, to a network. A MD with a USB port is considered a cyber device, even if it is not intended to be connected to a network. Hence, a misuse of this USB port could be to connect a USB adapter to a network socket.

Documentation required by Section VII

Section VII specifies the documentation required to comply with section 524B of de FD&C Act. The documentation requirements are bound to sub-sections of 524B.

524B(b)(1) Vulnerability Monitoring and Management Plan

This plan must specify deadlines and justifications for the development and deployment of updates and patches, taking two cases:

Regular updates for known but controlled vulnerabilities, and
Urgent out-of-cycle updates for critical vulnerabilities that could lead to uncontrolled risks.

This plan shall also include Coordinated Vulnerability Disclosure procedures.

Practically speaking, you should have and document a policy to update software according to the CVSS of identified vulnerabilities. E.g.: out-of-cycle updates for CVE with CVSS higher than 9 or CVE with safety impact, regular updates for others.

524B(b)(2) Cybersecurity design and maintenance process

This process must demonstrate that the device and related systems are designed, developed and maintained to ensure a reasonable level of cybersecurity. we recognize here the FDA's risk-based approach.
The application of the other sections of this guidance, IEC 81001-5-1 and AAMI SW76 is one way of implementing this design process.

524B(b)(3) A Software Bill of Materials (SBOM)

Section V.A.4.b of the document is referenced here. The SBOM can follow the NTIA minimal SBOM requirements.

Modifications

In addition, any modification requiring a new regulatory submission must also comply with section 524B. If the modification has an impact on cybersecurity, all relevant documentation must be provided, following this guidance and the guidance on cyber post-market surveillance.

For modifications with no impact on the cybersecurity of an existing device for which such documentation did not exist, the FDA still expects some documentation:

The Vulnerability Monitoring and Management Plan must be provided,
The manufacturer must demonstrate that there are no critical vulnerabilities, and
The SBOM must be provided.

For example:
If you cleared a device before 2023, It is quite possible that you didn't submit any documentation on cybersecurity.
If, today, you submit a change, claiming equivalence with your own device, then the FDA expects documentation on the three points above:

The change has an impact on cybersecurity: fully follow the FDA guidance,
The change has no impact on cybersecurity: follow partially the guidance. Especially for #2 in order to demonstrate that there are no critical vulnerabilities.

In this second case, a very minimal cybersecurity document set to add to your existing device submission is:

The Vulnerability Monitoring and Management Plan
Documents demonstrating the absence of critical vulnerabilities, usually:
1. On the risk management side: A cyber Risk Management File,
2. On device design side: Some cyber risk control measures, as applicable
3. On V&V side: A pen test report.
4. On device IFU side: informations relevant to users but also administrators of the device, as applicable
The SBOM.

Don't expect to get a FDA clearance without these documents.

Rogue 11: a MDR story

Mitch — Fri, 27 Jun 2025 13:00:00 +0200

MDCG 2019-11 rev.1 was published in June 2025. Almost 6 years after the first version.
Despite the passage of time, and the advent of modern AI, nothing has really changed. The rigorous approach is more than ever present in this guide.

Qualification

Crafting a well-defined and clear intended purpose is paramount for the regulatory compliance and safe use of MDSW says the guidance. This is what lots of software editors strive to do. Not for regulatory compliance, but to keep their software outside the regulatory hassle of MDR/IVDR.

Simple search

Some restrictions about what is considered a simple search were added. On purpose, this is consistent with a rigorous approach to consider that using NLP isn't simple search. Searching for patterns in medical images isn't simple search either.

New examples

Some new examples of MDSW were added. This is useful, if you look for some MDSW similar to your case. E.g.: Some VR applications, or SW module integrated in a EHR platform and outputting diagnosis results are actually MDSW.

As long as the definition of medical device in MDR article 2 is what it is, the content of this guidance won't be alleviated. On the contrary, it will be loaded with new examples of MDSW.

We are in a situation opposite to the FDA's law enforcement discretion policy, for example with CDSS.

Classification: Rogue rule 11

Rule 11 is still there, still harmful and merciless for all medical device manufacturers having MDSW in their products:

Extensive interpretation of sub-rule 11a) Software intended to provide information which is used to take decisions with diagnostic or therapeutic purposes makes almost any kind of SaMD class IIa,
No change to table 1 Classification Guidance on Rule 11 either. Bloated by class IIa cells and inconsistent with IMDRF table.

But the definition of medical device in MDR isn't limited to diagnosis and treatment. Other modes of actions are present.
If my software is intended to prevent a disease, in which class is it?

Crossing the MD definition and rule 11

In the MD definition, we find the following modes of action:

diagnosis, prevention, monitoring, prediction, prognosis, treatment or alleviation of disease,
diagnosis, monitoring, treatment, alleviation of, or compensation for, an injury or disability,
investigation, replacement or modification of the anatomy or of a physiological or pathological process or state,
providing information by means of in vitro examination of specimens derived from the human body, including organ, blood and tissue donations,
devices for the control or support of conception;
products specifically intended for the cleaning, disinfection or sterilisation of devices (...).

In rule 11, we find the following modes of action:

11a: (3 first paragraphs of Rule 11) intended to provide information which is used to take decisions with diagnostic or therapeutic purposes;
11b: (Paragraph 4 of Rule 11) intended to monitor physiological processes or parameters;
11c: (Paragraph 5 of Rule 11) all other uses.

Some definitions

The definition of medical device uses, several terms, which require a definition.

The definition of diagnosis can be found in MDCG 2022-5:

Diagnosis is the process of investigation of the anatomy, morphology, the condition or the functions of the human body irrespective if these are physiological or pathological, and subsequent interpretation of this information with a view to determining possible abnormalities. In this context, investigation can include visualisation, detection or measurement.

There is no definition of therapy, therapeutic or treatment or prognosis in MDR or MDCG or MEDDEV documents. Nothing either in 21 CFR or FDA guidance.
Here are definitions from Merriam-Webster dictionary:

Prognosis: the prospect of recovery as anticipated from the usual course of disease or peculiarities of the case
Therapeutic: of or relating to the treatment of disease or disorders by remedial agents or methods
Treatment: the action or way of treating a patient or a condition medically or surgically : management and care to prevent, cure, ameliorate, or slow progression of a medical condition.

The table below takes each word found in the definition of medical device in MDR article 2, and crosses it with the wording of sub-rule 11a: Software intended to provide information which is used to take decisions with XXX purposes.
This exercise was made to envision a maximum of modes of action of a MDSW.

Interpretation of sub-rules vs definition of medical device

SW Purpose	Software class
MDR article 2 definition: diagnosis, prevention, monitoring, prediction, prognosis, treatment or alleviation of disease
Software intended to provide information which is used to take decisions with diagnosis of disease purposes	Easy case with word diagnosis in sub-rule 11.a class IIa, class IIb, class III
Software intended to provide information which is used to take decisions with prevention of disease purposes	The word prevention isn’t present in rule 11. But: The guide contains this example: A device intended to prevent the risk of illnesses or pathologies by analysing physiological parameters (…) can be considered as a device providing information which is used to take decisions with diagnosis purpose (potential detection of pathologies) and in this case is in class IIa Wow, what an extensive interpretation of sub-rule 11.a! This is consistent with the definition of treatment quoted above. class IIa, class IIb, class III However, if the disease has not yet reached the subject, then we cannot talk about treatment. Lots of prevention and prophylaxis actions are performed on healthy subjects. E.g.: COVID prevention doesn’t mean subjects have COVID. Thus, the example of this guide is somewhat misleading, and software for prevention purposes can definitely be in class I.
Software intended to provide information which is used to take decisions with monitoring of disease purposes	Easy case with word monitoring in sub-rule 11.b class IIa, class IIb
Software intended to provide information which is used to take decisions with prediction of disease purposes	Prediction of a disease is a way to provide information which is used to take decisions with diagnosis (prediction of evolution of a disease) or therapeutic purposes (prediction of cure), right? Sub-rule 11.a class IIa, class IIb, class III
Software intended to provide information which is used to take decisions with prognosis of disease purposes	If we have a prognosis, it means that diagnosis is already done. But prognosis of a disease may be a kind of information for therapeutic purposes, right? Sub-rule 11.a class IIa, class IIb, class III It may be class I, if you craft your intended use on the edge of the precipice. Careful, though; you may end up with an upgrade.
Software intended to provide information which is used to take decisions with treatment of disease purposes	Easy case with word treatment in sub-rule 11.a class IIa, class IIb, class III
Software intended to provide information which is used to take decisions with alleviation of disease purposes	If we have alleviation of a disease, it means that diagnosis is already done. But alleviation of a disease may be a kind therapeutic purpose, right? Sub-rule 11.a class IIa, class IIb, class III It may be class I, if you craft your intended use in a way demonstrating that alleviation isn’t a treatment. This sounds like something hardly feasible, if you look at the definition of treatment quoted above.
MDR article 2 definition: diagnosis, monitoring, treatment, alleviation of, or compensation for, an injury or disability
Software intended to provide information which is used to take decisions with diagnosis of an injury or disability purposes	Easy case with word diagnosis in sub-rule 11.a class IIa, class IIb, class III
Software intended to provide information which is used to take decisions with monitoring of an injury or disability purposes	Easy case with word monitoring in sub-rule 11.b class IIa, class IIb
Software intended to provide information which is used to take decisions with treatment of an injury or disability purposes	Easy case with word treatment in sub-rule 11.a class IIa, class IIb, class III
Software intended to provide information which is used to take decisions with alleviation of an injury or disability purposes	If we have alleviation of an injury or a disability, it means that diagnosis is already done. But such alleviation may be a kind therapeutic purpose, right? Sub-rule 11.a class IIa, class IIb, class III It may also be class I, if you craft your intended use in a way demonstrating that alleviation isn’t a treatment. This sounds like something hardly feasible, if you look at the definition of treatment quoted above.
Software intended to provide information which is used to take decisions with compensation for an injury or disability purposes	If we have compensation of an injury or a disability, it means that diagnosis is already done. Such compensation might be a kind therapeutic purpose, right? Sub-rule 11.a class IIa, class IIb, class III It may also be class I, if you craft your intended use in a way demonstrating that compensation isn’t a treatment. This sounds like something more feasible than the case of alleviation. Compensation doesn't mean cure.
MDR article 2 definition: investigation, replacement or modification of the anatomy or of a physiological or pathological process or state
Software intended to provide information which is used to take decisions with purposes of investigation of the anatomy or of a physiological or pathological process or state	If we have investigation, it means that diagnosis is probably ongoing. See the definition of diagnosis above. Thus, such software is intended to provide information which is used to take decisions with diagnosis (or therapeutic) purposes. Sub-rule 11.a class IIa, class IIb, class III
Software intended to provide information which is used to take decisions with purposes of replacement of the anatomy or of a physiological or pathological process or state	Replacement sounds like therapeutic purpose. Thus, such software is intended to provide information which is used to take decisions with therapeutic purposes. Sub-rule 11.a class IIa, class IIb, class III It may also be class I, if you craft your intended use in a way demonstrating that replacement isn’t a treatment. This sounds like something hardly feasible, if you look at the definition of treatment quoted above.
Software intended to provide information which is used to take decisions with purposes of modification of the anatomy or of a physiological or pathological process or state	Modification sounds like therapeutic purpose. Thus, such software is intended to provide information which is used to take decisions with therapeutic purposes. Sub-rule 11.a class IIa, class IIb, class III It may also be class I, if you craft your intended use in a way demonstrating that modification isn’t a treatment. This sounds like something hardly feasible, if you look at the definition of treatment quoted above.
MDR article 2 definition: providing information by means of in vitro examination of specimens derived from the human body, including organ, blood and tissue donations
Software intended to provide information which is used to take decisions with purposes of providing information by means of in vitro examination of specimens derived from the human body, including organ, blood and tissue donations	The information by means of in vitro examination may be for any case listed above: diagnosis, prevention, monitoring, prediction, prognosis, treatment, alleviation, compensation. Depending on the intent of this information, search for the case in the lines above in the table.
MDR article 2 definition doesn't contain the word screening. Let's add it!
Software intended to provide information which is used to take decisions with purposes of screening diseases.	Screening of a disease is a way to provide information which is used to take decisions with diagnosis purposes, right? (Merriam-Webster says: to test or examine for the presence of something, such as a disease). Testing the presence of a disease is actually the first step to diagnosis. Sub-rule 11.a class IIa, class IIb, class III
MDR article 2 definition: devices for the control or support of conception.
Software intended to provide information which is used to take decisions with purposes for the control or support of conception.	Easy case, we have an example in the guide class I
MDR article 2 definition: products specifically intended for the cleaning, disinfection or sterilisation of devices.
Software intended to provide information which is used to take decisions with purposes for cleaning, disinfection or sterilisation of devices.	No doubt that this MDSW will fall in rule 14, either directly or by the use of the implementing rule 3.3 class IIa, class IIb

Class IIa, class IIb, and class III dominate the results of this table. This is not surprising. As soon as software answers to the definition of medical device (diagnosis, treatment, prognosis and so on), it falls into the case of sub-rule 11a. Sub-rule 11a is so broad that almost any type of software delivers information for diagnosis or therapeutic purposes. In a direct or indirect manner, like a 3-cushion billiard shot.

This is why lots of medical device manufacturers strive to craft an intended use, trying to keep their software in class I.
Unsuccessfully, when you read this guidance and interpret sub-rule 11.a in a rigorous way.

Some people greet the addition of an example in class I. One example. Versus lots of cases in higher classes. Nothing to get excited about.

Wishful thinking solution

As already mentioned in a previous post, a little word could be a game changer.

In the good old times of the MDD, we had the rule 10a for active devices where direct diagnosis was covered. Placing MDSW in class IIa. All other software providing information in an indirect way was in class I by rule 12.

Borrowing the spirit of MDD rule 10a, the word direct would make lots of software in class I:
In sub-rule 11.a, replace:
Software intended to provide information which is used to take decisions with diagnosis or therapeutic purposes
By:
Software intended to provide information which is used to take decisions with direct diagnosis or therapeutic purposes.

This little word would change the world of MDSW! Lots of software are used early in the care pathway. Conversely, medical imaging software would stay in class IIa. Their output is actually used for direct diagnosis.

Modular approach

This MDCG guide also contain a welcomed and interesting update of section 7 on the modular approach.
It will be useful for editors of heath-software with modules qualified as MDSW. Thanks to the provided examples.

It also contains a discussion on the combination of MDSW and non-MDSW. This combination shall remain safe and effective.
Evidence of safety and effectiveness shall be brought by:

Architectural representation of MDSW and non-MDSW, clearly delineating the MDSW boundaries,
Risk assessment, taking into account software failures coming from non-MDSW, and consequences on MDSW
Risk assessment, taking into account usability on non-MDSW displaying MDSW output.

Effective segregation

The guide doesn't mention how effective the architectural segregation should be. The words modules, delineate, clarity are present in the guide. We can rely on common practice and estimate that a module running in a micro-service will be better segregated than a library loaded by an executable.
Talking IEC 62304 langage: the MDSW manufacturer shall identify segregation necessary for risk control.

Usability on non-MDSW

This is a new recommendation. And this is consistent with the need of evidence of safety and effectiveness of MDSW and non-MDSW combination.
A result displayed by non-MDSW may be interpreted in a wrong manner by a user. This may be a hazardous situation leading to an unacceptable risk.

Accessory?

The guide is somewhat nice with health software editors. It doesn't re-qualify the non-MDSW displaying results as an accessory to the MDSW!

We can try to see why:
According to MDR definition of accessory: it specifically enable the medical device(s) to be used in accordance with its/their intended purpose(s). We can say that the non-MDSW, by displaying the MDSW result, enables the MDSW to be used in accordance with its intended purpose.

However, the word specifically is present twice in the definition of accessory.
It saves the health-software editors from seeing their existing non-MDSW re-qualified as accessory to the MDSW. An existing non-MDSW displaying electronic health records won't be qualified as accessory, if it displays health records containing the output of some MDSW.

However, a non-MDSW developed specifically to be used in combination with the MDSW will be qualified as accessory to the MDSW.

FDA multiple function device products

As usual, the FDA dealt with this problem long before the MDCG. The guidance on multiple function device products treats this case in a more thorough manner than the MDCG guide!
If you want to find how to document the safety and effectiveness of MDSW combined with non-MDSW, it's worth reading this FDA guidance! In other words, the content of your 510k can be reused in your CE tech file.

European Health Data Space (EHDS)

There is a long paragraph about EHDS in this MDCG guide. EHDS was published in March 2025. It's applicable to health software having electronic health records (EHR). Thus, applicable to the vast majority of health software, called EHR systems in this regulation. Editors will have to implement interoperability according to some common specifications. These specifications should be published by March 2027. The EHDS regulation will be fully applicable by March 2031 for EHR systems.

Fortunately, EHDS regulation requires "self-certification" only. No need to promise your Notified Body you will be compliant.
This paragraph is here to inform health software editors having EHR systems that this regulation will be applicable to them. Given the application dates, we have some time to implement it. However, it will collide with the dates of application of MDR, AI Act, and Cyber Resilience Act. By order of appearance on stage:

AI Act: August 2027 for MDSW with high-risk AI system (the majority, in fact, per rule 11a),
CR Act: December 2027 for the non-MDSW part of the health software,
MDR: December 2028 or December 2027 (if AIMD or Class III implantable, that's very few MDSW) ,
EHDS: March 2031 for MDSW with an EHR system,
And the NIS2 directive? That's set by every national transposition.

Lucky you, health software editors with MD, AI systems and EHR systems. You will have the feeling to be transitioning to some EU regulation forever!

Conclusion

The MDCG 2019-11 rev1 doesn't change anything to the MDSW landscape. In lots of cases, sub-rule 11.a will trap you and your MDSW in the class IIa hell. It won't discharge you until you decommission it.

Don't try to escape it!, Jun 2025

Team-NB position paper on EU AI Act

Mitch — Fri, 16 May 2025 13:53:00 +0200

The Team-NB published the V2 of a position paper on the EU AI Act in April 2025.
It's interesting to see that this paper was published just before the white paper from BSI.

This document is the V2 of a position paper published in V1 in 2021, when the AI Act was still a proposal. Thus, even if this is a V2, this is in fact the first version of the document, since the AI Act came into force.

By many aspects, this position paper could be seen as more controversial than the BSI white paper.
It raises lots of questions about the management of the AI Act by the EU Commission. On purpose.

NB designations and codes

The paper starts with Team-NB's concerns about NB designations and codes. Of course, the medical device NB's advocate for the extension of their notifications on existing codes. Rather than adding a new designation process specific to the AI Act.
Practically speaking, this is probably the best solution if stakeholders want to take as little time as possible to designate NB's able to evaluate MD with AI against the MDR/IVDR and the AI Act.

Access to data and datasets used by NB's

Annex VII paragraph 4.3 of the AI Act leaves the possibility to NB's to have a full access to datasets hold by the manufacturer.
And Annex VII paragraph 4.4 of the AI Act leaves the possibility to NB's to perform their own tests on the AI system subject to conformity assessment.

This introduces the question of having access to the right data, either those hold by the manufacturer, or those chosen by the NB to perform their test.
Especially for personal health data: where to get independent datasets? Anonymized? Pseudonymized? Patient consent? Security? These questions are wide open and are not about to be closed!

Data management process

This document, like the BSI white paper, insists on the requirement to implement robust data governance practices, including traceability and compliance with GDPR. To do so, manufacturers will have to put in place a data management process. This process is new for most of manufacturers. They can have something already in place. But not ensuring conformity to AI Act requirements.

Similarities with BSI white paper

The Team-NB position paper also raises concerns on the interpretation of Safety component, of substantial modification versus significant change. We saw similar discussions in BSI's white paper.
No controversy here, BSI's document is aligned with Team-NB's document on the need of guidance and interpretation of the grey areas present in the AI Act.

However, reading this paper between the lines, we can see the power struggles and tractations happening behind the scenes. In fact, this Team-NB document raises lots of concerns about the implementation of the AI Act, and the absence of any sign showing that the AI Act schedule will be met.

Drifting timelines

To make it short, the position paper calls into question the timelines defined in AI Act and time is really takes to get things done:

Competent Authorities designations,
NB designations,
Harmonized standards,
Access to independent test dataset,
Guidance documents on interpretation of AI Act requirements for medical devices
and probably some other topics...

Hey AI Act guys, do you know how long MDR / IVDR have been delayed?
7 years.
And that's a vertical regulation for a single industry.
And we already had harmonized standards (OK. No. Not harmonized, but harmonized all the same).
And we already had Notified Bodies (OK. No. Not notified, but bodies were already there).

So, now, you can imagine how long it will take for your AI Act, an horizontal regulation with no harmonized standards, with no notified bodies, to be fully applicable.
My optimistic guess is: 2027 for medical devices incorporating High-Risk AI will be postponed to 2030.

BSI White paper: The EU AI Act meets the MDR

Mitch — Fri, 16 May 2025 13:51:00 +0200

BSI Notified Body published in May 2025 a new White Paper on the AI Act and its interactions with MDR / IVDR.

This white paper is a summary of AI Act content and its interactions with MDR / IVDR. It contains also BSI's interpretation of AI Act requirements and interaction with MDR /IVDR.

This document is a quite interesting and practical one for people wanting to have a summary of the AI Act without reading the body text of that regulation. Examples given at the end of the document are also quite interesting and practical. For those who already read the AI Act (!), these examples will give you or confirm your interpretation.
Since it is already a summary, no need to summarise here the summary (I let other medical device news websites post a LLM-generated summary of this white paper :-).

A few remarks

Here are a few remarks picked up when reading this white paper:

In section AIA classification and Interplay with MDR

The document insists on the concept of safety function depending on the type of device that manufacturers place on the market. Especially MDSW and software which drives or influences a medical device. MDCG guidance or AI Office guidance should confirm BSI's interpretation,
Annex III of AI Act is considered as not applicable to medical devices. This is perhaps at too quick assertion. We could imagine a medical device, which intended purpose relies on performance of an AI system doing biometric categorisation or emotion recognition.

In section Quality Management System

It puts emphasis on data management and data governance. Implementing the AI Act requirements into an existing MDR-compliant QMS will require a new data management process, to cover AI Act requirements on data, as well as GDPR and other regulations referenced in the AI Act,
It references ISO 42001 and ISO 23894. It's worth noting that these standards won't be harmonized according to the CEN-CLC/JTC 21, the working group in charge of AI Act standards harmonization,
Especially, ISO 23894 uses the concept of risk defined in ISO 31000: Effect of uncertainty. This definition isn't suitable for medical devices. Thus, this standard should be left apart when seeking conformity both to AI Act and MDR.

In section Technical documentation requirements

The document mentions that the AI Act imposes more rigorous documentation and monitoring requirements, going beyond what is required by the MDR. It is right that the AI Act goes beyond the MDR on AI-specifics. But it is rather wrong that the AI Act imposes more rigorous documentation and monitoring requirements. Hey, BSI, by experience, you know how rigorous MD documentation shall be (manufacturers who chose you still recall)!

In section Product requirements

It mentions the human oversight but doesn't question the suitability of such requirement for medical devices. Take a close-loop device: Is it relevant to have human oversight the way it is required in the AI Act? Article 14 of AI Act uses the words as appropriate and proportionate to implement human oversight. Maybe BSI's intent is to stay consensual and give early guidance, in the absence of AI Office guidance or MDCG guidance,
Likewise, it mentions accessibility requirements and the need to integrate accessibility requirements to ensure inclusivity for all users. Such requirements may not be suitable, depending on the medical device intended purpose.

In section Changes to AI system

It heavily insists on the definition of significant changes according to the MDR versus the AI Act. And this remains a good question. Further guidance from the AI Office and/or the MDCG is welcome!
It also mentions the FDA PCCP guidance. It's true that state-of-the-art on changes on AI systems can be found in this FDA document. We hope that future MDR or AI Act guidance or standards will be aligned with FDA PCCP principles.

Conclusion

This white paper is a good reading if you want an introduction to AI Act requirements and its interactions with MDR.
I raises the good questions on topics subjects to interpretation, which need clarification by the AI Office and/or the MDCG.

Note: why naming a fictitious French manufacturer Reality Vieux. That sounds so awkward to fluent French speakers. Voilà, this is probably a private joke.

Team-NB questionnaire on artificial intelligence in medical devices

Mitch — Fri, 11 Apr 2025 13:32:00 +0200

The team-NB published a questionnaire on artificial intelligence in medical devices in November 2024. This questionnaire is simply a (very long) list of questions that a Notified Body may take one by one, when they review a technical file of a medical device containing AI.

We’ve seen in a previous post how the “monster” FDA guidance on AI tells manufacturer how they should design, validate and monitor such medical device. The FDA’s approach is: “You should do it this way”.
The Team-NB’s AI check list takes an opposite approach. They don’t tell how manufacturer should do things, but check if things are done. The Team-NB’s approach is then: “Show me how you did it”.

Both approaches are equivalent; at the very end, the FDA will also review what the manufacturer did. But the FDA’s approach is more didactic, by setting a common ground on how to do things.
Following the FDA guidance on IA MD allows in some way to answer to the Team-NB’s checklist!

Validation or Validation?

As a preamble, we'd like to warn the reader about the word "validation".
The checklist uses the word validation with the meaning of AI validation, after training.

E.g., with this question:
6.1.4 Input for risk management and clinical evaluation
14. Does the manufacturer assess the risks related to splitting the data into training, validation and test data?

The words shall be understood like this:

Training: is a part of AI model design (no confusion here),
Validation: is AI model validation and is a part of AI model design,
Test: is software testing phase, part of device verification.

Thus, don't misuse the word validation. To clarify it, it shall be accompanied with a context:

AI validation, part of AI model design,
Device validation, part of classical V&V.

Note: this question could be misleading:
6.4.4 Documentation
2. Can the manufacturer reproduce the test and validation results?
As this question references ISO 13485 7.3.6, we know that with are in verification, not device validation.

Overview

The Team-NB checklist is quite a long list of questions. It covers the medical device lifecycle with:

General Requirements: 7 questions
Intended use and stakeholder requirements: 33 questions
Software requirements: 24 questions
Data management: 38 questions
Model development: 30 questions
Product Development: 35 questions (including 9 in clinical evaluation)
Product release: 4 questions
Requirements for the post development phase: 18 questions

A total of 189 questions!
Questions that add up to other long checklists on the rest of Technical Files. This is a typical approach of Notified Bodies. Good luck manufacturers, if you want to reduce the burden and mark some questions irrelevant.

Regulations, guides and standards

Since there is no harmonized standard specialized on AI lifecycle, Team-NB cannot do otherwise than quoting IEC 62304, ISO 14971, and the like.

To be more precise, here is the list of regulations, guides and standards referenced by the checklist:

Reference	Comment
Regulations: Regulation (EU) 2017/745 MDR Regulation (EU) 2017/746 IVDR Regulation (EU) 2016/679 GDPR Regulation (EU) 2021/2226 e-IFU	MDR and IVDR are frequently quoted (no surprise), GDPR is quoted once on patient health data privacy, e-IFU is quoted once on availability on latest version.
Guidances: MDCG 2019-16 MDCG 2020-1 MEDDEV 2.7/1 revision 4	These guides are sparsely quoted in relevant questions about cybersecurity and clinical evaluation
Medical device standards: EN ISO 13485 EN ISO 14971 EN ISO 14155 EN/ISO 20417 EN 62366-1	These standards are without surprise frequently quoted in the checklist. Excepted ISO 14155 a few on questions related to good clinical practices, and ISO 20417 once on accompanying documents.
Medical device standards specific to software: EN 60601-1 EN 62304 EN 82304-1 EN/IEC 80001-1 IEC 81001-5-1	Not surprising: IEC 62304 and IEC 82304-1 are quoted a lot. IEC 60601-1 (only clause 14 is specific to software) is quoted once, on its clause 4.4 about expected service life. Why this clause? Because this is the only text, which defines the expected service life (try searching for lifetime or shelf life in regulations!). Cybersecurity standards are quoted on a few questions on AI security risks.
Medical device standards specific to software and AI: BS/AAMI 34971	Here we feel a bit alone. The only medical device standard on AI is this AAMI 34971 on AI risks. This is a good start. We’ll see that some other standards are to come.
Standards specific to AI, but not specific to medical devices: ISO/IEC 23894 ISO/IEC 5259-2 ISO/IEC 5259-3 ISO/IEC 5259-4 ISO/IEC 5338 ISO/IEC 5339 ISO/IEC TR 24027 ISO/IEC TR 24028 ISO/IEC TS 4213	We see here the biggest limitation of this checklist: In the absence of medical device standards, the checklist has to summon standards coming from other industries. Even if this looks like a good idea at the beginning, this leaves an impression of uncompleted work. See the discussion below.
General software standards ISO/IEC 25010 ISO/IEC/IEEE 12207	Why quoting these general software standards? Especially why quoting IEEE 12207 only once on its clause about SW architecture, and not on other clauses? Each time IEC 62304 or IEC 82304-1 is quoted, IEEE 12207 could be quoted on its equivalent clause. Likewise, why quoting ISO 25010 on SQuaRE only, and not other standards on SQuaRE method? For example: ISO 25024 about Measurement of data quality. This looks like cherry-picking some good practices from other industries. Why these ones and not other ones ? This leaves once again an impression of uncompleted work.

If you don't know standards on AI quoted above, here are their titles, and some comments:

Standard	Title	Comment
ISO/IEC 23894	Information technology — Artificial intelligence — Risk management	This standard appears twice, and especially in this question: 8. Has the manufacturer identified and evaluated the gaps between ISO/IEC 23894 and EN ISO 14971 in his risk management documentation considering the device under assessment? Is it really necessary? ISO 23894 isn't an appropriate standard for safety. The definition of risk is the one from ISO 31000: ''The effect of uncertainty on objectives'' We'd better rely on ISO 14971, AAMI 34971 and ignore ISO 23894.
ISO/IEC 5259-2	Artificial Intelligence — Data quality for analytics and machine learning (ML) — Part 2: Data quality measures	This standard appears only once as supplementary reference. This standard is an interesting one for measuring data quality and data consistency.
ISO/IEC 5259-3	Artificial intelligence — Data quality for analytics and machine learning (ML) — Part 3: Data quality management requirements and guidelines	This standard appears 7 times as supplementary reference. It is an interesting one on data quality management and setting up a data management process.
ISO/IEC 5259-4	Artificial intelligence — Data quality for analytics and machine learning (ML) — Part 4: Data quality process framework	This standard appears 4 times as supplementary reference. It is an interesting one as well on data quality management and setting up a data management process.
ISO/IEC 5338	Information technology — Artificial intelligence — AI system life cycle processes	This standard appears 6 times as supplementary reference. IEC 5338 supplements IEC 12207 on artificial intelligence. Maybe you know IEC 12207: it is referenced by IEC 62304 as supplemental source of information.
ISO/IEC 5339	Information technology — Artificial intelligence — Guidance for Al applications	This standard appears 3 times as supplementary reference. IEC 5339 supplements IEC 15288 (itself referenced by IEC 12207) on artificial intelligence.
ISO/IEC TR 24027	Information technology — Artificial intelligence (AI) — Bias in Al systems and Al aided decision making	This standard appears twice, and especially in this question: 5. Does the manufacturer examine the data sets that predicted particularly well and those that predicted particularly poorly? This technical report is an interesting one, for methods for assessment of bias and fairness Note: another standard about bias is quoted in references at the bottom of the document: ISO/IEC TS 12791 on Treatment of unwanted bias in classification and regression machine learning tasks.
ISO/IEC TR 24028	Information technology — Artificial intelligence — Overview of trustworthiness in artificial intelligence	This standard appears three times, and especially for its clause 8.8.2.23 on this question: 4. Does the manufacturer identify means to reduce the risk of training procedure related effects such as overfitting? This technical report contains lots of information. Maybe to read but not to follow by the book.
ISO/IEC TS 4213	Information technology — Artificial intelligence — Assessment of machine learning classification performance	This standard appears several times, on data collection and AI model evaluation. To read when the AI model is a classifier.

That's 11 new standards, which scope isn't medical devices! Don't buy them too quickly.
First, they're mostly quoted as supplemental source of information. Second, some MD specific standards on AI are being cooked and should be published soon. We'll see that in a further post.

The Team-NB's policy was probably to reference existing standards outside MD scope, to justify the questions and to direct the reader towards a way to bring answers.

Other remarks

Questions without reference

23 questions are left without reference, or with supplementary reference only. This raises the question: are these questions legitimate?
Obviously yes, when you read them. The absence of reference just reminds us the pressing need of having standards or guidance on AI in MD.

IEC 62304

Our good old IEC 62304 is quoted 37 times. That's a bit surprising for a standard written before the emergence of modern AI.
In the absence of MD-specific standards, we still have to rely on IEC 62304. Needless to say that IEC 62304 isn't adapted to AI. Its requirements need an extensive interpretation when they are applied to AI model lifecycle.

Performative prediction

The following question addresses a phenomenon involving software users. Its note is very interesting:
6.1.4 Input for risk management and clinical evaluation
16. Does the manufacturer assess the risks by making predictions that change the predicted outcomes themselves, if applicable?
Note: This phenomenon applies, when the model switches from being an observer to an actor. It is referred to as "performative prediction" (...).

Such concern isn't specific to AI. A classical algorithm could produce the same effects with unaware users. They could see software as an "oracle" they blindly trust.
As we've seen in the comments on the FDA guidance on MD with AI, such concerns aren't specific to AI, but are exacerbated by AI.

Data representative of target population

The following question addresses the representativeness of data:
6.3.1 Collection of the training, validation and test data sets
6. Does the manufacturer justify where it collects e.g. training, test and validation data and why it is representative of the target population? Where appropriate, has the manufacturer compared these with data from the Federal Statistical Office, scientific publications and registries?

Such concern is actually specific to AI during training and AI validation. However, data representativeness is also a concern in device validation for classical algorithms. Like the previous question, this phenomenon is pointed by the FDA guidance. Such concerns aren't totally specific to AI, but are exacerbated by AI.
For those who don't know what the Federal Statistical Office is, you'll find this institution in Germany. We see here the parenthood of this checklist with German Notified Bodies' work. The working group overlooked this reference to a local institution, when the Team-NB document was written. You can disregard and replace it by other relevant local institution.

In-field self-learning AI models

The Team-NB reminds in section 4 that in-field self-learning AI models aren't "certifiable" unless the manufacturer takes measures to ensure the safe operation of the device within the scope of the validation described in the technical documentation.
Needless to say that nobody ever tried to certify such AI model (at the time this blog post is written). It is already challenging to have a Predeterminded Change Control Plan (PCCP) accepted by the FDA. An in-field self-learning AI model is one step beyond complexity.

QMS

The section 4 also mentions that standard operating procedures (SOP) are affected by the AI lifecycle. The FDA guidance also mentions that in its section named QMS.
The Team-NB document explicitly mentions the Data Management Process. But with Customer Property in parentheses. However, data management process isn't limited to customer property. Implementing a data management process can be a way to answer to the 38 questions on data management found in the checklist.

Uncompleted work

Yes, this work isn't completed. Not in its questions, but in its references. As we said, lots of standards outside MD were summoned, in the absence of AI in MD standards. The writers of this checklist couldn't do otherwise.
Likewise, no reference is made to the AI Act. A regulation too new to allow its complete digestion by Notified Bodies.
This is exactly what the working group wrote in the V1.1 of this document, stating that it shall be updated to take into account AI Act requirements, Once European Standards are available.

On the QMS side, as of today, we only have ISO 42001 to implement an AI-aware QMS. But ISO 42001 is not considered adequate for presumption of conformity under the AI Regulation and cannot be considered as the state-of-the-art. Thus, we have to wait for new AI MD-specific standards, for product standards and for QMS standards.

That leaves the reader in an uncomfortable situation: should I read and buy these standards? Should I wait for MD specific standards?
We cannot answer to this question right now. Some of these new standards may or may not reference the ones currently quoted in this document.
But if a manufacturer is requested by an Notified Body to fill-in this checklist, they will be left with no choice but following at least the clauses referenced in these standards, if they don't have strong alternative references.

This situation is the result of the AI technological breakthrough in MD, leaving regulators, certification bodies, and standardization committees behind. Standards take time to write. In the absence of standards, the Team-NB just tries to fill the gap with this checklist.

Note: ISO 42001, despite lacking requirements on design and made to manage risks according to ISO 31000 is still a good start to define new AI-aware SOPs. A white paper on the impact of ISO 42001 on an ISO 13485-compliant QMS is available here.

Conclusion

This checklist is quite long, with 189 questions. It will also cost a lot of man-hours to fill-in for manufacturers. Imagine you need half an hour per question, that makes 95 hours, or approx. 12 days (8 hours per day). This checklist isn't lowering the already exagerated costs of CE marking.

One solution would be to add it to reference documents in a technical file, without following it by the book. However, that solution may not be accepted by some Notified Bodies.

IEC 62304 2nd Edition draft ready for public comments

Mitch — Fri, 07 Feb 2025 15:21:00 +0100

BS EN IEC 62304 Ed2. Medical device software - Software life cycle processes is published for public comments on BSI website!

This new draft version of the standard is really an improvement, compared to the 20215 version!

The picture would be perfect if it weren't plagued by the design specification of this standard. It states that 2 software process levels replace the 3 software safety classes. These two levels are misaligned with FDA guidances and with current SW development practice. I already analysed the situation in a previous post.
My concerns about this misalignment are still valid with this draft version.
In its current state, and if the definition of sw levels is kept as is, this standard bring unnecessary overhead on manufacturers with software in class B.
This design specification was the input data of the working group. Thus, they couldn't do otherwise.

Don't hesitate to comment on BSI website. You won't have much time. Comments are closed by the 12th February 2025!