The 'AI Check-Up': How Lifelong Exams for Medical AI Could Protect Your Health Before a Silent Glitch Ever Reaches You

Imagine your doctor passed a licensing exam in 2015 and was never assessed again. No continuing education, no peer review, no one checking whether their judgment had slipped as medicine changed around them. You would be alarmed, and rightly so. Yet for most of the past decade, that is roughly how many AI tools in medicine have been treated: tested carefully once before launch, then trusted to keep performing indefinitely.

That model is starting to crack. In the first weeks of October 2026, regulators on three continents moved, almost in unison, toward a new idea: medical AI should get regular check-ups for as long as it is in use, much like the clinicians who rely on it. On October 6, the UK government accepted all 44 recommendations of a national commission on regulating AI in healthcare, including a shift to monitoring AI devices throughout their working life. On October 19, the U.S. Food and Drug Administration closed public comments on a discussion paper that floats evaluating generative AI devices more like doctors than like machines. And on October 8, South Korea's health ministry published its first guidelines for people using generative AI for their mental health.

None of this makes headlines the way a breakthrough drug does. But it may matter more to your safety than any single new algorithm, because it decides who notices when an AI tool quietly gets worse.

The Problem Nobody Sees: AI That Drifts

Traditional medical devices are fairly predictable. A blood pressure cuff that works on launch day will, barring damage, work the same way five years later. AI is different. Its performance depends on the data it sees, and real-world data never stops changing.

Researchers call this drift. A model trained on images from one brand of scanner may stumble when a hospital upgrades its equipment. A risk score built on pre-pandemic patients may misjudge a population whose habits and illnesses have shifted. A tool tuned for adults in one country may perform unevenly on older patients, children, or different ethnic groups elsewhere. Nothing visibly breaks. There is no error message. The tool simply becomes a little less accurate, and in medicine a little can be the difference between a caught tumour and a missed one.

Generative AI raises the stakes further. A chatbot or documentation assistant built on a large language model can produce an almost unlimited range of answers. You cannot test every possible question before launch, and the underlying foundation models are often updated by their developers on a schedule the device maker does not fully control.

The scale is no longer small. The FDA's public list of AI-enabled medical devices now runs well past a thousand authorized products, the large majority in radiology. Most were evaluated in a one-time review. The question regulators are now asking is simple: who is watching them afterwards?

Britain's Blueprint: From One-Time Approval to Lifelong Oversight

The UK's National Commission into the Regulation of AI in Healthcare was set up by the Medicines and Healthcare products Regulatory Agency (MHRA) in September 2025. Chaired by Professor Alastair Denniston, with England's Patient Safety Commissioner, Professor Henrietta Hughes, as deputy chair, it gathered evidence from more than 12,000 people over a year before publishing its recommendations on September 10, 2026.

The government's response, released on October 6, accepted every one of the 44 recommendations. The core finding is that AI-enabled devices should be assessed and monitored across their entire working life rather than relying on a single assessment at the start. Concretely, the plan includes:

• Draft MHRA guidance by December 2026 on managing changes to AI devices as they adapt over time.
• A consultation in 2027 on how AI-enabled devices should be qualified and classified as medical devices in the first place.
• Staged authorisation pathways, letting promising tools be used in the NHS earlier but under close supervision, with evidence building as they go.
• Phase 3 of the AI Airlock, the MHRA's regulatory sandbox, now focused specifically on post-market surveillance and lifecycle regulation, backed by three more years of government funding. The first wave of participants is due to be selected in November 2026.
• A commitment to tell patients when AI is used in their care, alongside better access to redress when care falls short.

Health Innovation Minister James Frith framed the balance plainly: "innovation must never come at the expense of patient safety." A full implementation roadmap is promised by spring 2027.

America's Twist: Credentialing AI Like a Clinician

Across the Atlantic, the FDA is wrestling with the same problem from a different angle. In August 2026, its Center for Devices and Radiological Health released a discussion paper on generative AI-enabled medical devices, inviting comments under docket FDA-2026-N-7874 until October 19. It is not formal guidance, but it signals where the agency's thinking is heading.

The most striking idea is a competency-based review. Instead of trying to test every possible input and output, which is impossible for a system that can say almost anything, the FDA would assess whether a device demonstrates core competencies, much as medical boards assess a physician. Those competencies include recognizing safety-critical situations, communicating uncertainty honestly, clinical proficiency, and consistency across different patients and settings. That nonclinical benchmarking would then be paired with clinical confirmation, ranging from analysis of real patient records to full randomized trials depending on risk.

The paper also sketches a two-axis risk framework. One axis measures how independently a device acts; the other measures how badly someone could be harmed if they relied on a wrong answer. Notably, tools that speak directly to patients may be treated as higher risk than those used by professionals, because clinicians are better placed to spot an error.

Crucially, the FDA suggests accepting somewhat more uncertainty before launch in exchange for stronger oversight afterwards. The options it lists read like a continuing-education programme for software:

• Periodic re-benchmarking against the same thresholds used at approval, on a schedule and whenever the model changes.
• Sample-based clinician review, in which independent experts audit a slice of real-world inputs and outputs.
• Drift monitoring to catch performance decay as patient populations or data environments change.
• Supervisory AI agents that watch other AI systems, an idea the agency raised as a question rather than a plan.
• Foundation model master files, a voluntary way for model developers to share confidential technical details that device makers can reference.

The FDA's fiscal 2027 guidance agenda, published in early October, reinforces the direction. Finalizing its guidance on lifecycle management for AI-enabled device software sits on the top-priority list, as does new draft advice on evidence for generative AI conversational devices aimed at mental health conditions, a topic that was absent from the previous year's list.

Asia, Canada and Europe: Different Routes, Same Destination

South Korea: protecting the user, not just the product

On October 8, ahead of World Mental Health Day, South Korea's Ministry of Health and Welfare and the Korean Neuropsychiatric Association issued guidelines for using generative AI for mental health, with separate versions for adults and for young people and their guardians. Each carries 21 recommendations: do not treat AI answers as a diagnosis, be careful with personal information, and watch for over-reliance. The motivation is striking. In a survey of 1,019 users aged 15 to 79, nearly 69 percent said they used generative AI for emotional purposes, and about one in six said they had found it hard to stop a conversation with an AI. Where Britain and the U.S. focus on monitoring the tool, Korea is also coaching the human using it.

Canada: change plans written into the licence

Health Canada has already laid groundwork through its pre-market guidance for machine learning-enabled medical devices. It allows manufacturers to submit a predetermined change control plan describing how a model will be updated and validated, and lets the regulator attach terms and conditions to a device licence to keep watching safety and effectiveness after it reaches the market. Canada, the FDA and the MHRA have also published shared guiding principles for these change plans, which makes cross-border alignment more realistic for companies selling in all three markets.

Europe: high-risk rules on a longer runway

In the European Union, AI built into medical devices remains classified as high-risk under the AI Act, layered on top of the Medical Device Regulation's existing post-market surveillance duties. Following a provisional political agreement in spring 2026, the compliance date for those obligations is slated to move from August 2027 to August 2028. The substance, including risk management, data governance and human oversight, is unchanged; only the clock moved.

The Gulf and wider Asia

Health systems in the Gulf states, Singapore and elsewhere that import AI tools approved abroad will feel these shifts directly. If London and Washington start demanding ongoing performance evidence, buyers in Riyadh, Abu Dhabi or Singapore can reasonably ask for the same monitoring data, and for proof that a tool performs well on their own populations rather than only on the patients it was trained on.

The Benefits: Why Lifelong Check-Ups Could Save Lives

• Problems get caught sooner. Continuous monitoring can flag a slipping algorithm in weeks rather than years, before a pattern of missed diagnoses builds up.
• Fairness becomes measurable. Real-world audits can reveal when a tool underperforms for specific age groups, ethnicities or regions, which one-time trials often miss.
• Good tools reach patients faster. Staged authorisation means a promising tool need not wait years for perfect evidence, provided it is closely watched once deployed.
• Updates become safer. Pre-agreed change plans let developers improve models without starting from scratch, while staying inside guardrails the regulator has approved.

The Risks: What Could Go Wrong

• Weaker gatekeeping up front. Early commenters on the FDA paper warned that post-market monitoring cannot replace solid pre-launch evidence where harms are severe or irreversible.
• Who pays for the watching? Ongoing audits, clinician reviewers and drift dashboards cost money. Smaller hospitals and clinics, often in rural or lower-income areas, may struggle to take part, creating a two-tier system of oversight.
• AI watching AI. Using software agents to supervise other software is appealing at scale but has no clear regulatory precedent, and it risks automating away the human judgement it is supposed to support.
• Privacy pressure. Monitoring real-world performance means collecting and reviewing real patient interactions. In the U.S., HIPAA governs how hospitals handle that data, while in Canada PIPEDA and provincial health privacy laws set the rules for private-sector organizations; many consumer health apps fall partly or wholly outside such protections. Monitoring programmes will need strong de-identification and clear consent.
• Patchwork rules. With the UK, U.S., EU, Canada and Korea moving on different timelines, global developers face overlapping demands, and patients may get different levels of protection depending on where they live.

What This Means for You

You will not see a regulator's dashboard, but you can apply the same thinking to your own care:

• Ask whether AI is involved. The UK is committing to tell patients when AI is used in their care. Wherever you live, it is fair to ask your clinician whether a scan, risk score or message was produced or assisted by AI.
• Ask how it is checked. A good follow-up question is whether anyone reviews the tool's results over time. Clinicians may not know every detail, but the question signals that patients care.
• Treat chatbot answers as a starting point. Korea's guidelines offer universal advice: a general-purpose AI is not a diagnosis, and it is not a therapist. If a conversation leaves you more anxious, or you find it hard to step away, talk to a person.
• Guard your details. Think twice before typing names, health card numbers or detailed medical histories into consumer AI tools, which may not be covered by health privacy law.
• Report what seems off. If an AI-generated result or message seems wrong, tell your care team. Real-world feedback is exactly the signal lifecycle monitoring depends on.

For Healthcare Professionals and Leaders

• Inventory your AI. List every AI-enabled tool in use, including features quietly switched on inside existing software, with its version, vendor and intended use.
• Set local baselines. Measure how each tool performs on your own patients at go-live, so you have something to compare against when drift appears.
• Write monitoring into contracts. Ask suppliers for change notifications, performance reports broken down by patient subgroup, and visibility into the foundation models they depend on.
• Name an owner. Create a governance group, or extend an existing one, with clinical, data, privacy and patient-safety voices and clear authority to pause a tool.
• Watch the calendar. Key dates include MHRA draft guidance on adaptive AI by December 2026, the FDA's comment window on its fiscal 2027 priorities through November 30, and the EU's high-risk obligations for medical device AI from August 2028.

Wrapping Up

For years, the debate about medical AI has centred on the moment of approval: is this tool good enough to launch? The developments of early October 2026 shift the spotlight to a harder and more important question: is it still good enough today, for these patients, in this hospital?

Britain is building a lifelong oversight system, the U.S. is exploring credentialing AI almost like a clinician, Canada is writing change plans into device licences, Europe is preparing high-risk rules, and South Korea is reminding users that a chatbot is not a cure. The approaches differ, but the destination is shared. The safest medical AI will not be the one that passed a single exam brilliantly. It will be the one that keeps getting checked, and that someone is ready to switch off the moment it slips.