What medicine can learn from incident reporting, recurrent training, and human judgment.
In a recent article, I wrote about a risk raised at the Cleveland Clinic AI Summit: artificial intelligence can improve a clinician’s performance while it is available, yet leave that clinician less prepared when it is not.
One example came from a 2025 observational study of colonoscopy practice. After clinicians began routinely using AI-assisted polyp detection, their adenoma detection rate during unassisted procedures fell from 28.4 percent to 22.4 percent. The study was limited to four centers and does not establish that every clinical AI system will weaken human skill. But it describes a pattern that another safety-critical industry has confronted for decades.
A presenter at the summit described a similar concern among medical students: AI could help them complete the immediate assignment while leaving them less prepared for an assessment without it. The details and circumstances matter, but the broader warning is familiar. Assistance can improve today’s performance without necessarily building tomorrow’s capability.
Advanced automation transformed aviation. It improved navigation, situational awareness, efficiency, workload management, crew performance, and safety. It also changed the pilot’s job. When crews misunderstood, mismanaged, degraded, or unknowingly disengaged automation, they were less prepared to recognize what was happening and more likely to make basic mistakes under pressure. They had become dependent on the automation.
Healthcare is entering a similar transition. As caregivers rely more on computing and AI, we will increasingly practice medicine through or alongside extraordinarily complex machines. Aviation’s most useful lesson isn't just extensive, comprehensive simulator training. It is the safety system that surrounds automation: learn from failures and near misses, share those lessons, and turn them into recurring, demanding, and required education.
Automation changes the expert’s job
Pilots once spent much more of a flight directly manipulating the controls. Modern flight crews often supervise layers of automation, verify what the aircraft is doing, anticipate what it will do next, and intervene when the system’s behavior no longer fits the situation.
That arrangement works extremely well most of the time. The danger appears when automation is partially unavailable, in the wrong mode, fed bad information, confronted with a condition its designers did not anticipate, or experiencing partial or complete failures. In those moments, the crew must detect the mismatch, reconstruct what is happening, choose the appropriate level of automation, and sometimes return to basic flying, thinking, analytics, and other decision-making skills under intense pressure.
Aviation researchers and regulators have examined the resulting risks: reduced manual proficiency, weak awareness of automation modes, overreliance, confusion during transitions, and poor performance when surprise sharply increases workload. FAA guidance has therefore emphasized manual flight practice, proficiency assessment, training with different automation combinations, and scenarios that move beyond routine checklists to much higher-order thinking and decision-making.
The lesson is not that automation was a mistake. Commercial aviation became extraordinarily safe and efficient as it adopted it. The lesson is that introducing automation changes the competencies people need. Training, reporting, and oversight have to change with the job.
Aviation built a learning system, not just a training device
Although I am a physician and scientist, I have also spent decades taking and teaching recurrent pilot training. For commercial pilots, the training is required on a recurring schedule. It is serious, technically demanding, and intentionally uncomfortable. Pilots commonly describe it as going back to school to get beat up. The objective is to expose weaknesses and failure scenarios in a controlled setting, not discover them for the first time with passengers aboard.
The scenarios do not come from imagination alone. Aviation has spent decades building feedback loops that turn experience into shared learning.
When a major accident occurs in the United States, the National Transportation Safety Board conducts an independent investigation. It examines probable cause, contributing factors, human performance, technology, procedures, organizational decisions, and the surrounding system. Its reports and safety recommendations become part of the industry’s collective memory and can drive changes in equipment, operations, regulation, and training.
Aviation also needed a way to learn from events that did not become accidents. The Aviation Safety Reporting System, administered by NASA, allows pilots, controllers, flight attendants, dispatchers, maintenance technicians, and others to voluntarily and confidentially report unsafe occurrences, errors, and hazardous situations. NASA de-identifies the reports, analyzes them, issues alerts where appropriate, and makes safety information available to the aviation community. Under defined conditions, a timely report may also support an FAA waiver of a sanction. It is not blanket immunity; it is a carefully bounded incentive to disclose and learn.
Pilots often refer casually to filing a “NASA form,” although they can now submit reports electronically. The important feature is not the form. It is the trusted pathway from an individual observation to system-wide learning.
Accident investigation, confidential near-miss reporting, operational changes, and recurrent training reinforce one another. That is the parallel healthcare should study.
Healthcare has pieces of the system, but not yet the computing-AI learning loop
Healthcare is not starting from zero. Hospitals have internal event-reporting programs. Patient Safety Organizations can receive protected patient-safety information. Regulators, accreditors, specialty societies, manufacturers, and health systems all collect portions of the picture.
But the pieces do not yet amount to a broadly trusted, cross-organizational learning loop for clinical AI or computer-enabled health care operations. Too many errors, omissions, and near misses remain known only to the people directly involved, or perhaps to a department or health system. Even when an incident is reported, the lesson may not travel far enough to change product design, workflow, policy, or continuing education elsewhere.
That is especially dangerous with computing and AI because the same failure pattern can appear in many forms. A model may be correct but lack critical context. An answer may be wrong yet persuasive. A useful finding may be routed to the wrong person. A clinician may stop checking a task the machine usually handles. A team may assume someone else owns the next step. Harm can arise from the model, the human, the interface, the data, the workflow, communication (or its lack), or the interactions among them. Problems are rarely studied or tracked when they involve patient care that crosses more than one health care organization or the communication, coordination, and collaboration between them
AHRQ’s review of patient-safety event reporting makes a related point: collecting reports is not enough. Effective systems need a supportive reporting environment, broad participation, timely dissemination, structured review, and action. The purpose is not to accumulate incident forms. It is to turn what people observe into safer practice.
The first warning may be loss of independent skill
The colonoscopy study is valuable because it examined what happened when clinicians exposed to AI performed without it. The reported decline was not an AI error in the usual sense. The system did not hallucinate a finding or fail during the procedure. The concern was that repeated assistance changed human performance.
That distinction matters. Most AI governance programs focus on whether a model is accurate, secure, fair, and compliant at deployment. Those are essential questions, but they do not tell us how prolonged use will affect the people working alongside it.
We should expect some skills to change. Clinicians no longer need to memorize every fact that can be retrieved reliably, just as pilots no longer need to manually calculate every parameter that avionics can determine more quickly and accurately. The goal is not to preserve every historical task forever. The harder question is which human capabilities must remain strong because they are necessary to detect error, manage exceptions, address ethical and deeply human issues, communicate effectively, and assume control.
In medicine, those capabilities include clinical judgment, the ability to integrate incomplete and contradictory information, awareness of what the system cannot see, communication with patients and families, and responsibility for the final decision. An AI may identify a pattern or propose a solution. It does not bear the consequences of acting on it.
The response to my first article sharpened the issue
Several comments on my earlier article helped clarify what is at stake. One reader argued that a human being or human enterprise remains responsible for everything a computer is asked to do and for the quality of the result. Another observed that discussions of AI’s value often omit ethical principles.
Both points belong in the continuing-education conversation. A caregiver should never be uncertain about who owns the decision after AI enters the workflow. Organizations also need clear boundaries for appropriate use, escalation, disclosure, oversight, and review. Technical proficiency without accountability is not safe practice.
Another reader described watching a family member with advanced cancer wait weeks to enter a clinical trial while medical records moved slowly and required manual handling. His comment returned the discussion to the problem that motivated my first article. Even when AI identifies the right answer or the most effective pathway, the patient benefits only if the right people receive the information and act in time.
Education cannot stop at evaluating a model’s output. It must cover the full operational chain: what the finding means, who is responsible, where the information must go, how it should enter the receiving workflow, who acts, and how the organization confirms that someone acted.
What an aviation-style safety system for healthcare should do
Healthcare does not need to copy aviation word for word. A cockpit, an intensive care unit, a primary care practice, and an oncology program have different hazards, timelines, and operating models. But several principles transfer remarkably well.
First, make reporting safe and worthwhile. Clinicians, staff, patients, vendors, and others should have trusted ways to report AI- and non-AI-related errors, close calls, confusing behavior, workflow failures, and signs of overdependence. Reports should be protected and de-identified where appropriate, while still allowing responses to reckless or intentional misconduct.
Second, investigate the whole system. Reviews should examine not only model accuracy but also data quality, interface design, automation assumptions, staffing, timing, communication, accountability, and what happened after the output appeared.
Third, share the lessons. De-identified patterns should cross organizational, vendor, operational, and geographic boundaries. Health systems cannot improve from risks they never see, and caregivers should not have to rediscover the same failure on their own.
Fourth, turn incidents into recurrent education. Clinicians should practice with realistic failure modes: incomplete records, misleading confidence, automation bias, conflicting recommendations, unavailable tools, poorly timed alerts, accurate results routed to the wrong person, and systems that change behavior after an update.
Fifth, preserve the independent skills required for safe intervention. We should identify which capabilities must remain available when AI is absent or wrong, then assess them periodically without assistance.
Sixth, rehearse the handoff from insight to action. An AI-generated finding should have an explicit owner, an appropriate destination, enough clinical context, a defined next step, and a way to confirm follow-through.
Finally, train leaders and instructors, not only end users. Aviation has learned that instructors need specific knowledge of the failures in and shortcomings in automation, human factors, error recognition, and intervention. Healthcare will likewise need people who understand both clinical work and AI well enough to design scenarios, evaluate performance, and improve the program as technology and evidence change. AI itself may help surface patterns across errors, omissions, and near misses, but people and institutions must decide what to do with those signals.
Recurrent preparation for a changing system
One-time training will not be enough. AI systems change too quickly, and both operational systems and human behavior change with repeated use. Health systems need a recurrent model that revisits known risks, introduces new failure modes, tests performance, and uses actual incidents and near misses to improve future training.
This doesn't need to be a grand-slam effort to do everything at once. The information security space had a similar problem where information about organizations that were successfully hacked was typically hidden and not shared. I was part of the creation of the very successful DBIR effort (Data Breach Investigation Report), which initially combined the investigation data from more than a dozen and now nearly 100 different contributing organizations worldwide, including large digital forensics firms, cyber insurance companies, law enforcement, national computer emergency response teams, vendors, and others. The most recent report aggregates data from more than 31,000 incidents and is routinely republished, viewed, downloaded, summarized, analyzed, taught, and used by thousands of organizations, informing policy and education and generating millions of impressions annually.
This is one reason I believe organizations will need a formal AI education function. Stanford Medicine’s appointment of Jonathan Chen, MD, PhD, as Director for Medical Education in Artificial Intelligence offers one example of the institutional capability that may be required. Someone must own the continuing education of the people who use these tools, not merely the procurement and technical governance of the tools themselves.
The objective should not be to make clinicians distrust AI. It should be to develop warranted trust: confidence that expands or contracts according to the system, the evidence, the patient, and the situation.
Aviation did not respond to automation risk by returning to an earlier era of flight. It built ways to discover failures, report close calls, share lessons, and train people to work with powerful machines while staying ready to recognize trouble and take control.
Healthcare with AI now needs its aviation safety moment. We need trusted reporting, rigorous investigation, shared learning, and recurrent education that turns real experience into better care. The point is not to prepare clinicians for a single dramatic failure. It is to build a system that keeps learning before the next patient is harmed.
Sources and editorial references
Budzyn K, Romanczyk M, Kitala D, et al. Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study. The Lancet Gastroenterology & Hepatology. 2025;10(10):896–903. doi:10.1016/S2468-1253(25)00133-5.
National Transportation Safety Board. Safety Recommendations. https://www.ntsb.gov/investigations/Pages/safety-recommendations.aspx
NASA Aviation Safety Reporting System. ASRS Program Briefing and program overview. https://asrs.arc.nasa.gov/docs/ASRS_ProgramBriefing.pdf
Federal Aviation Administration. Advisory Circular 00-46F: Aviation Safety Reporting Program. September 23, 2024.
Federal Aviation Administration. Air Carrier Training Aviation Rulemaking Committee Recommendation 16-9: Manual Flight Operations. 2016.
Agency for Healthcare Research and Quality Patient Safety Network. Reporting Patient Safety Events. Last reviewed March 2025. https://psnet.ahrq.gov/primer/reporting-patient-safety-events
Agency for Healthcare Research and Quality. Patient Safety Organizations Program. Last reviewed November 2024. https://www.ahrq.gov/cpi/about/otherwebsites/pso.ahrq.gov/index.html
Stanford Medicine. Jonathan H. Chen, MD, PhD, faculty profile.
Contact careMESH today to learn how careMESH can accelerate your product's interoperability strategy.