EU AI Act and Healthcare AI: Obligations for Medical Device Integrations

If you build AI for healthcare — diagnostic imaging tools, clinical decision support systems, patient triage algorithms, remote monitoring software — you are operating under two concurrent EU regulatory frameworks simultaneously. The EU AI Act and the Medical Device Regulation (MDR 2017/745) or In Vitro Diagnostic Regulation (IVDR 2017/746) both apply to your product. Their requirements overlap, interact, and in some places reinforce each other. Getting them right in parallel is one of the most complex compliance challenges in EU digital health regulation.

This article explains exactly how the dual framework applies, what your specific obligations are under each regime, and where a single compliance process can satisfy both.

Why Healthcare AI Faces a Dual Regulatory Framework

The EU AI Act classifies AI systems as high-risk through two distinct routes. The first route applies to AI systems that are safety components of products already regulated under specific EU product safety legislation. The second route applies to standalone AI systems in certain high-impact domains listed in Annex III.

For healthcare AI, the first route is the most consequential.

Article 6(1) of the AI Act states that an AI system is high-risk when it is a safety component of a product covered by EU harmonisation legislation listed in Annex I of the AI Act, and when that product itself is required to undergo third-party conformity assessment under that sector legislation. The MDR and IVDR are both explicitly listed in Annex I.

The consequence: any AI system embedded in a regulated medical device is automatically classified as high-risk under the AI Act, regardless of what the AI does. You do not need to assess whether it meets any additional criteria. If the device is regulated under MDR or IVDR, the embedded AI is high-risk.

For standalone clinical AI that is not embedded in a physical device but provides decision support, triage, or diagnostic outputs, the path runs through Article 6(2) and Annex III. Annex III §5(b) covers AI used in administration of critical digital infrastructure. More relevantly for clinical software, AI systems used in employment, education, and essential services — including health services — may fall within scope depending on their specific function and the population they affect. Regulatory guidance on where clinical decision support software sits on this spectrum continues to develop.

AI as a Medical Device: The Dual Classification

The term "AI as a Medical Device" (AIaMD), sometimes called Software as a Medical Device (SaMD) with AI capability, describes software whose primary purpose is medical and which uses AI or machine learning to achieve that purpose.

Under the MDR and IVDR, the classification of a medical device determines the intensity of the conformity assessment required:

Device class	Examples	MDR conformity pathway
Class I	Basic non-invasive tools	Self-certification (no notified body)
Class IIa	Low-to-medium risk diagnostic support	Notified body involvement required
Class IIb	Higher risk diagnostics, monitoring systems	Full notified body QMS audit
Class III	High risk, life-sustaining	Notified body design examination
IVDR Class C/D	High-risk in vitro diagnostics	Full notified body assessment

AI-based diagnostic software typically lands in Class IIa or higher under the MDR risk classification rules. AI used in in vitro diagnostics for companion diagnostics, cancer screening, or infectious disease detection typically reaches IVDR Class C or D.

The MDCG (Medical Device Coordination Group) has issued guidance on the qualification and classification of software under MDR, which the European Medicines Agency (EMA) has supplemented with AI-specific considerations for regulated products. Manufacturers should consult both MDCG 2019-11 (software classification) and MDCG 2021-6 (AI/ML in medical devices) when determining classification.

How the AI Act and MDR/IVDR Conformity Assessments Interact

This is the question that keeps medtech compliance teams awake: does a company need to run two entirely separate conformity assessments, or can they be coordinated?

The answer, under Article 43(1) of the AI Act, is that they can be coordinated — and in many cases, should be.

For an AI system that is a safety component of a medical device (Article 6(1)), the AI Act requires third-party conformity assessment via a notified body. The MDR and IVDR already require notified body involvement for Class IIa, IIb, III, and equivalent IVDR classes. The AI Act explicitly allows the same notified body that conducts the MDR/IVDR assessment to also cover the AI Act conformity assessment requirements for Annex I products.

In practical terms: if your notified body is assessing your Class IIb AI diagnostic device under MDR, that same notified body can and should also review your AI Act compliance documentation at the same time. This is not automatic — the notified body needs to be designated under both regimes and you need to structure your documentation accordingly — but it eliminates the need for two entirely separate assessment tracks.

For AI systems falling under Annex III §1 (biometric identification), a separate notified body AI Act assessment is required regardless of sector regulation. Diagnostic AI that does not use biometric identification does not fall under this sub-provision.

The CE Marking Pathway

An AI system embedded in a medical device carries the CE mark under MDR or IVDR. The AI Act does not create a separate CE mark for Annex I products — the single CE mark from the sector legislation covers the product, provided the AI Act obligations have been met as part of the conformity assessment.

For standalone clinical AI software that falls within AI Act Annex III (not via Article 6(1)), a separate AI Act conformity assessment and CE marking process applies.

Full AI Act Obligations for Healthcare AI Providers

Meeting the "high-risk AI system" classification means the full suite of Articles 9 through 15 obligations applies. Here is what each requires in a clinical context.

Article 9: Risk Management

You must establish, implement, document, and maintain a risk management system for the lifecycle of the AI system. In a clinical context, this means integrating your AI Act risk management process with your existing MDR/ISO 14971 medical device risk management framework. The two are compatible but use different terminologies — MDR focuses on clinical safety risks, while the AI Act requires a broader lens including fundamental rights risks, discrimination risks, and risks from foreseeable misuse.

Article 10: Training, Validation, and Testing Data

Healthcare AI training data requirements under the AI Act are among the most demanding in any sector. Your training datasets must:

Be relevant, representative, free of errors, and complete
Account for characteristics specific to the intended geographical, contextual, or functional setting
Be examined for possible biases that could affect the health or safety of natural persons
Reflect diversity of the patient population in terms of age, sex, ethnicity, and other relevant demographic variables

The interaction with GDPR Article 9 special category data is critical here. Health data is special category data under GDPR. Processing it for AI training requires either explicit consent or a specific legal basis under Article 9(2) GDPR — typically scientific research or public interest under national law. The AI Act's training data requirements do not modify GDPR requirements; both apply simultaneously. A dataset that is AI Act compliant but lacks a valid GDPR processing basis is still non-compliant.

Article 11: Technical Documentation

Technical documentation for healthcare AI must follow Annex IV requirements and include detailed descriptions of the system architecture, the training methodology, the validation datasets and performance metrics disaggregated by relevant patient subgroups, known limitations, and foreseeable edge cases in clinical use.

For AI diagnostic tools, performance metrics disaggregated by patient demographics — including performance differences by age group, sex, and where applicable ethnicity — are not optional. Regulators and notified bodies reviewing healthcare AI technical files will specifically examine whether the system has been validated across the patient populations where it will be deployed.

Article 12: Record-Keeping and Logging

High-risk AI systems must automatically log events relevant to system performance and decision traceability. For clinical AI, this means maintaining an audit trail of AI-generated outputs that is accessible to clinical staff reviewing a care decision. Where an AI system generates a diagnostic suggestion, that suggestion and the inputs used to generate it must be recorded and retained in a form that allows retrospective review.

Article 13: Transparency and Information to Deployers

Providers must supply deployers — hospitals, clinics, healthcare networks — with an instructions for use document that includes the system's intended purpose, performance characteristics, known limitations, required expertise for users, and circumstances in which the system might produce unreliable outputs. In clinical settings, this information must be integrated into clinical training and workflow documentation.

Article 14: Human Oversight

Healthcare AI systems must be designed to allow clinical staff to meaningfully oversee, interpret, and where necessary disregard the AI's output. This is not satisfied by a generic "human in the loop" checkbox. The AI Act requires that the human oversight function be technically implemented in the system design — meaning clinical staff can intervene, override, or flag outputs, and those interventions are logged.

Black-box models that provide a clinical recommendation without any indication of the factors driving that recommendation create both an Article 14 compliance problem and a clinical governance problem. Explainability mechanisms are not legally mandated by name in the AI Act, but the oversight obligation functionally requires that clinical staff can make an informed judgment about whether to follow the AI's recommendation.

Article 15: Accuracy, Robustness, and Cybersecurity

The system must achieve appropriate levels of accuracy for its intended purpose and must be resilient against errors, inconsistencies, and attempts at adversarial manipulation. In healthcare contexts, this requires documented performance thresholds established during clinical validation, drift monitoring post-deployment, and a defined process for suspending the AI system if performance falls below acceptable clinical thresholds.

Article 27: FRIA Obligations for Healthcare Deployers

The Fundamental Rights Impact Assessment (FRIA) is one of the AI Act's most significant innovations for deployers. Under Article 27, any deployer who is either a public body or an entity providing a service of general public interest must conduct a FRIA before deploying a high-risk AI system.

Hospitals are public bodies in most EU member states. This means they are required to conduct a FRIA before deploying any high-risk AI system, including AI diagnostic tools, patient triage systems, and clinical decision support software.

Private clinics and healthcare providers delivering services under public health contracts or providing services accessible to the general public are likely to fall within the "public interest service" category as well. The boundary is not yet definitively interpreted across all member states, but healthcare providers should apply the FRIA requirement unless they can clearly establish that their services are entirely private and outside any public health framework.

A FRIA for healthcare AI must assess the likely impact of the system on fundamental rights, including the right to health, the right to non-discrimination, the right to privacy, and the rights of vulnerable populations including children, elderly patients, and persons with disabilities.

Key Risks in Healthcare AI: What Regulators Will Look For

Algorithmic bias in diagnostics is the highest-profile risk that regulators and notified bodies will scrutinise. Studies have documented significant performance disparities in AI diagnostic tools for dermatology, radiology, and cardiology across racial and demographic groups. A technical file that does not address these disparities will face scrutiny in both MDR and AI Act assessment processes.

The interaction between clinical validation datasets and real-world deployment populations is a second key risk area. A system validated on a dataset from a single hospital in one country may perform significantly differently in a different patient population. The AI Act's post-market monitoring obligation and the MDR's post-market surveillance requirements both address this, but companies must actively track real-world performance disaggregated by relevant demographic variables.

Practical Implementation Guidance

For health tech companies building AI-enabled medical devices or clinical software:

Determine your MDR/IVDR device class first. This determines whether a notified body is required and under which regime.
If a notified body is required under MDR/IVDR, engage them early about their AI Act assessment capability. Not all notified bodies are yet designated or equipped for AI Act Annex I assessments.
Structure your technical documentation to address both MDR Annex II/III requirements and AI Act Annex IV requirements in a single integrated document set. Duplication is unnecessary; gaps are costly.
Integrate your AI Act risk management process with your ISO 14971 process from the beginning of the design phase. Retrofitting AI Act requirements onto a completed MDR risk management file is far more expensive.
If you are deploying AI in a hospital setting, conduct the FRIA before go-live. The FRIA is a deployer obligation, not a provider obligation — but providers who offer FRIA-ready documentation to hospital customers have a significant procurement advantage.

DilAIg generates the four mandatory documents that both providers and deployers need — including the FRIA. If you are building healthcare AI and need to produce your Annex IV technical documentation, Declaration of Conformity, FRIA, and Transparency Notice, start your 50-question audit at dilaig.com.

FAQ: EU AI Act and Healthcare AI

Q: Does every clinical AI tool need a notified body assessment under the AI Act? Not necessarily. If the AI is embedded in a medical device that requires notified body assessment under MDR/IVDR (Class IIa and above), that assessment covers the AI Act requirement under Article 6(1). Standalone clinical software that does not qualify as a medical device under MDR may need its own AI Act conformity assessment, depending on Annex III classification. Lower-risk clinical software tools that do not meet Annex III criteria can self-certify.

Q: Can a hospital use an AI diagnostic tool that does not yet have a CE mark under the AI Act? Until August 2026 (postponed to 2 December 2027 under the AI Omnibus agreement), the AI Act's high-risk system obligations are not yet fully applicable for new systems on the market. However, systems placed on the market from the applicable date onwards must meet full AI Act obligations before deployment. Hospitals procuring AI diagnostic tools now should include AI Act compliance timelines in their procurement criteria and contracts.

Q: What is the difference between a FRIA and an AI Act conformity assessment? A conformity assessment is conducted by or on behalf of the provider (the company that built the AI system) to demonstrate the system meets AI Act technical requirements. A FRIA is conducted by the deployer (the hospital or clinic) to assess the specific impact of deploying that system in their context on fundamental rights. Both are required, by different parties, for the same high-risk system.

Q: Does the AI Act apply to AI used purely for administrative purposes in a hospital? AI used for administrative tasks — scheduling, billing, resource allocation — is not automatically high-risk under the AI Act just because it is used in a hospital. It needs to meet an Annex III criterion or be an Article 6(1) product safety component to be classified as high-risk. Administrative AI that does not make decisions affecting individual patients' care falls outside the high-risk classification in most cases.

Q: How does GDPR interact with AI Act training data requirements? The AI Act Article 10 training data requirements and GDPR operate in parallel. A healthcare AI company training on patient data must comply with both. GDPR requires a valid legal basis for processing special category health data — typically explicit consent, scientific research grounds, or substantial public interest under national law. The AI Act adds requirements for data quality, representativeness, and bias examination. Neither regulation supersedes the other.

Key Takeaways

Healthcare AI faces two concurrent regulatory frameworks: the EU AI Act and MDR/IVDR. Both apply simultaneously and must be addressed together.
Under Article 6(1) of the AI Act, any AI system embedded in a regulated medical device is automatically classified as high-risk, regardless of what the AI does.
The same notified body can conduct both the MDR/IVDR conformity assessment and the AI Act assessment for Annex I products, enabling a coordinated rather than duplicated process.
Articles 9 through 15 impose specific obligations for healthcare AI on risk management, training data diversity, demographic performance disaggregation, logging, human oversight, and clinical robustness.
Hospitals are public bodies and are therefore required to conduct a FRIA (Article 27) before deploying any high-risk AI system.
Algorithmic bias across patient demographics is the highest-scrutiny area for both regulators and notified bodies reviewing healthcare AI.
Providers who integrate AI Act and MDR documentation from the design phase avoid costly retrofitting and create a procurement advantage with hospital customers.