Hackensack’s AI Monitoring Work Becomes the First Seal

Hackensack Meridian Health became the first U.S. health system certified for responsible AI use on July 29, 2026. The seal scores whether the New Jersey network can govern and watch the tools it already runs, not whether any one model is clever.

That is the bill from a warning Bommae Kim, then lead data scientist at the system, gave at HIMSS24 on March 12, 2024. Hospitals were pouring effort into building models, she said, and starving the work that starts after go-live.

The Monitoring Gap Named at HIMSS24

Kim’s session, “Monitoring the Health and Real-World Impact of AI Applications,” ran from 4:15 to 4:45 p.m. in Room W307A in Orlando. She holds a PhD in quantitative methods and a master’s in behavioral science, and she was not talking about another accuracy slide.

The focus around AI, she said, stays on technology and performance during development. Adoption and impact after deployment get less attention, even as in-house tools and vendor products move into clinical and business workflows. Her team’s answer was a four-area monitoring framework built for both homegrown software and third-party products.

To address this gap, we developed a robust monitoring framework covering four key areas: product pipeline, model performance, user behaviors, and business impact.

Bommae Kim, lead data scientist, Hackensack Meridian Health, HIMSS24

The stated aims were to catch trouble, stop critical errors, and measure whether a program actually worked. Hackensack designed a dashboard template with shared metrics and a unified data model so product teams, stakeholders, and leadership could argue from the same numbers. An automated alert path was meant to flag odd patterns and keep bad outputs from reaching the applications clinicians see.

Kim put the cultural problem in one line that still reads like 2026 procurement language. “Although AI is a hot topic, unfortunately, the importance of monitoring is often overlooked in contrast to the development and deployment of AI solutions,” she said. The framework, she added, was supposed to shape workflows and user adoption, not just model training.

The same week, chief executive officer Robert C. Garrett opened HIMSS24 with a broader pitch: responsible AI with humans always in the loop, including tools to help radiologists rank critical cases and to spot advanced kidney disease earlier. The data-science talk in W307A was the unglamorous half of that speech. Someone still had to watch the tools after the keynote lights went out.

Hackensack Took the First National AI Seal

Joint Commission launched its Responsible Use of AI in Healthcare certification on June 1, 2026, after September 17, 2025 guidance written with the Coalition for Health AI. The Oakbrook Terrace, Illinois, group, founded in 1951, said it evaluates more than 23,000 healthcare organizations and programs. The new credential is voluntary, and it certifies how a provider uses AI, not the products on its inventory list.

Hackensack, an 18-hospital network based in Edison, New Jersey, first health system in the nation to say it had earned the seal, on July 29, 2026, two years and four months after Kim’s session. Joel Klein, M.D., executive vice president and chief digital and information officer, said the review ran three weeks and looked like other Joint Commission inspections, down to named staff interviews and a tool inventory. Microsoft Dragon Ambient eXperience for documentation and some Epic AI features were on that list.

Klein had already been talking like a skeptic. In January 2026 he said the system was “dialing up our scrutiny level” and wanted hard outcomes before pouring money onto a hyped tool. On the seal itself he said Hackensack was proud of its process and still wanted to know whether that process could live up to Joint Commission standards and come out tighter.

Garrett’s announcement line stayed on care, not dashboards. “AI has the potential to help clinicians detect diseases earlier, personalize treatment for every patient, improve the efficiency of our hospitals, and reduce administrative work that contributes to burnout,” he said. “Most importantly, it enables physicians, nurses and team members to spend more time caring for patients.” The press text underneath that quote is the part Kim would have recognized: safeguards, monitoring processes, education, and accountability structures.

THE PATH FROM A CONFERENCE TALK TO A SEAL

1. March 12, 2024: Kim presents the four-area monitoring stack at HIMSS24 while Garrett keys the same meeting on responsible AI.
2. September 17, 2025: Joint Commission and the Coalition for Health AI release the first shared guidance on responsible use, after convening more than 20 coalitions and groups.
3. June 1, 2026: Joint Commission opens the voluntary Responsible Use of AI in Healthcare certification, with monitoring written in as a major domain.
4. July 29, 2026: Hackensack announces it is the first U.S. health system to earn the seal, after a three-week review.
5. August 18, 2026: The FDA asks for comments on how to regulate generative AI devices, including how to watch them after they ship.

Jonathan B. Perlin, M.D., Ph.D., president and CEO of Joint Commission, tied the program to how widely the tools have already spread. With more than 80% of physicians using AI in professional settings, he said, citing an American Medical Association survey, health systems need a shared blueprint. Klein made the competitive point in plainer words: the vast majority of health systems already live under Joint Commission accreditation, so an AI add-on from that body will not stay a New Jersey curiosity.

What the Five Certification Domains Require

Joint Commission organizes the seal around five areas: governance; effective data management; risk and bias reduction; monitoring, evaluating, and validating safety performance, effectiveness, and responsible use; and transparency, education, and training. The program page says it recognizes organizations that can show governance, safeguards, monitoring, and education. It does not bless a scribe, a sepsis score, or a chatbot.

Set those five domains next to Kim’s four pillars and the 2024 dashboard looks less like an internal science project and more like a draft exam. Product pipeline maps onto data management. Model performance sits inside the monitoring-and-validation domain. User behavior becomes training and transparency. Business impact becomes board-level governance. The extra Joint Commission column is explicit risk and bias work, which Kim had folded into “fair” care rather than named as its own scored box.

HOW THE 2024 STACK MAPS ONTO THE SEAL

The commission also wants quality monitoring and voluntary reporting of AI safety-related events. That is closer to drug safety reporting than to a one-time validation study. A hospital can pass a lab test on a frozen dataset and still fail this exam if nobody is watching live use, overrides, and downstream harm.

Hackensack’s public AI page now talks the same language. The system says innovation without intention is not enough, points to an AI governance program, and lists NIST-aligned “trustworthy AI” as a design goal. The 2026 seal did not invent that posture. It graded it.

Quiet Drift After a Clean Launch

The reason monitoring has to outlive the launch party is boring, and it is why patients get hurt without a headline outage. A model that never changes its files can still degrade because the hospital around it changed: new scanners, older patients, a different mix of nursing-home transfers, a pandemic, a lab that starts reporting a test in a new way.

A prognostic study in JAMA Network Open followed 143,049 patients across seven hospitals in Toronto and used a label-agnostic pipeline to catch those shifts in demographics, hospital type, admission source, and lab assays such as brain natriuretic peptide and D-dimer. Transfer learning and drift-triggered retraining recovered performance, including during COVID-19. The study’s practical finding is the one Kim was selling in 2024: waiting for a labelled outcome to prove the model is sick is too slow.

WHAT BREAKS WHEN NOBODY IS WATCHING

• Population shift: The patients in front of the model are not the patients it was trained on, even if the software version never moved.
• Workflow mismatch: An alert that helps in one unit becomes noise in another, and human-use problems in FDA reports have been tied to harm more often than pure technical faults.
• Chatbot misuse: ECRI, the device-safety nonprofit, ranked misuse of AI chatbots as its No. 1 health technology hazard for 2026, ahead of digital blackouts and fake medical products.
• Closed-loop failure: A correct warning still fails when the only person allowed to act is busy and the system never turns the override into follow-up labs or a second set of eyes.

That last failure is not a model-accuracy story. It is a staffing and authority story. Hospitals that buy a high-scoring tool and skip the watch floor will look fine on a vendor slide and still ship a silent miss. The Toronto work shows the miss can be found with input-data monitoring before the mortality chart moves. ECRI’s 2026 list shows the other path: staff and patients already treating unregulated chatbots as if they were devices.

Vendors Now Answer to Hospital Dashboards

Because the seal certifies the hospital, not the app, the pressure moves upstream. A third-party model that cannot export the metrics a monitoring template needs becomes harder to keep. Kim’s 2024 brief already treated vendor tools and in-house tools as the same monitoring problem. Joint Commission just gave that demand a paid, inspectable form.

The losers are not only shaky startups. They are also large vendors that sell a go-live and then disappear into a business-associate agreement, and small or rural hospitals that cannot hire the people Hackensack already had in 2024. Large academic networks can stand up governance committees, local validation, bias review, and 24-hour alerting. A 25-bed hospital that shares a CIO with a county system cannot clone an 18-hospital data-science bench. If the seal becomes how payers, plaintiffs, and boards define ordinary care, that gap turns into a two-speed safety market.

WHO FEELS THE MONITORING BILL

• Vendor product teams: They now owe live performance feeds, version logs, and a way to roll back when a hospital dashboard trips.
• Hospital quality desks: They inherit AI events the way they inherited infection and medication events, including voluntary safety reports.
• Front-line nurses and physicians: Their clicks, overrides, and workarounds are no longer anecdotal; they are a scored user-behavior stream.
• Smaller providers: They face the same five domains without the bench that let Hackensack finish a three-week review.

Fees for the certification vary by organization type and size, which is another filter. A voluntary program that the dominant U.S. accreditor sells will not stay cosmetic once boards start asking why a peer has the mark and they do not. Klein’s “vast majority” point is the second-order mechanism: most U.S. hospitals already organize their safety work around Joint Commission. An AI module from that body sets a default, even for systems that never apply.

The FDA Opened a Generative AI Docket

Hospital accreditation is only half of the watch regime. On August 18, 2026, the FDA’s Digital Health Center of Excellence posted a discussion paper on generative AI-enabled medical devices and asked for comments under docket FDA-2026-N-7874 by October 19, 2026. The paper is not guidance and does not change the rules. It does put postmarket monitoring for generative AI devices on the same page as risk scoring and premarket tests.

The FDA is seeking public input on considerations for the regulation of generative AI-enabled medical devices. We’ve put out a discussion paper, and we’re inviting feedback by October 19, 2026. https://t.co/s2L3cCqkyb pic.twitter.com/8XvwdUkiP6

— U.S. FDA (@FDA) August 18, 2026

The agency already has a tool for planned updates to AI devices, the predetermined change control plan, with current final recommendations posted in August 2025. That plan assumes a maker can describe future changes, the protocol for making them, and the impact on safety. Generative systems that take open-ended prompts and give variable answers strain that logic, which is why the August 18 paper exists. Periodic benchmarking, sample clinician review, and degradation tracking are the monitoring ideas on the table.

A hospital seal and a device docket sound like different worlds until a scribe or a triage chatbot sits in both. Hackensack has to watch whatever it deploys, including tools that may never need an FDA file. Makers that do need a file will be asked how they will keep watching the model after clearance. Kim’s four pillars, written for a health-system dashboard, now show up in both conversations.

The 2024 talk was easy to file under conference content. The 2026 seal is a priced exam on the same homework, and the FDA comment window is still open for the generative half of the toolkit. Hospitals that treated monitoring as optional are no longer arguing with a data scientist in Orlando. They are arguing with an accreditor that already grades their operating rooms.

Frequently Asked Questions

Does Joint Commission AI Certification Approve a Hospital’s Software Tools?

No. The program certifies the organization’s governance and use of AI, and Joint Commission says it does not validate or certify individual products. A hospital can hold the seal while running a mix of vendor and homegrown tools, each of which still follows its own FDA or local-validation path.

Can a Health System Apply if It Is Not Already Joint Commission Accredited?

Yes. The June 1, 2026 launch notice said interested organizations do not need Joint Commission accreditation to apply. They must be in the United States, operated by the U.S. government, or operated under a congressional charter, and they are expected to meet applicable federal participation rules such as CMS conditions.

What Did Bommae Kim’s 2024 Monitoring Stack Measure Day to Day?

Beyond the four named pillars, Hackensack used a reusable dashboard template, a unified data model, and automated alerts designed to stop concerning outputs from reaching end-user applications. The operational goal was shared numbers for product teams, executives, and clinical leaders, not a one-off accuracy report at launch.

What Is a Predetermined Change Control Plan for an AI Device?

It is an FDA-reviewed plan, submitted with a marketing file, that describes planned modifications, the protocol for making and checking those modifications, and an impact assessment. Changes that stay inside an authorized plan can proceed without a new 510(k), De Novo, or PMA supplement; changes that leave the plan still need a fresh filing.

Hackensack can hang the first seal because it built the watch floor before anyone sold a test for it. Every other system now has a public example of what “good” looks like, and a calendar that runs through October 19, 2026 for the generative rules still being written.