The majority of Australian hospitals and clinics are experimenting with artificial intelligence – yet fewer than one in eight of those experiments reaches the production floor. New research shows 60% of healthcare organisations are piloting AI, but only 12% have deployed the technology across multiple clinical or administrative functions. The bottleneck is not the algorithms: it’s that boards, executives, clinicians and IT staff are struggling to govern a technology that can influence diagnosis as easily as it drafts a letter.

What actually happened – AI left the lab and walked into the consultation room

Across Australia, ambient AI scribes now listen to patient conversations and draft clinical notes. Predictive models surface sepsis risk and flag abnormal imaging. Large language models summarise correspondence and propose care plans. These tools are no longer research projects; they are appearing inside Microsoft 365, in browsers, on smartphones and through subscription services purchased by individual practitioners.

Dr Monica Trujillo, Chief Health & Risk Officer at Telstra Health, describes the speed of adoption as a clinical leadership crisis. Traditional medical software arrived through formal procurement, validation and integration. Generative AI, she writes in Health Services Daily, “can enter through a web browser, smartphone application, Microsoft 365 add-in or subscription purchased by an individual practitioner.” That accessibility bypasses the safety checkpoints hospitals spent decades building.

A pilot can succeed with hand-picked clinicians, close supervision and a narrow patient group. Production deployment must survive staff turnover, software updates, unusual presentations, network outages, inconsistent documentation and thousands of interactions never seen in testing. The 88% gap between pilot and production is not slow procurement, Trujillo argues – it is the difficulty of translating a controlled demonstration into a safe, repeatable service.

What the governance failure means for you

If you are a patient, you may not know that an AI system contributed to your consultation. A scribe may have drafted the note your GP signed, or a model may have prioritised your imaging study. The consent you gave – perhaps a hurried click on a tablet – may not have explained where your audio recording is stored, who can access it, or whether it is used to train a product. Refusing the technology may be impractical if it means finding another provider.

For clinicians, the danger is quieter. An ambient scribe does not transcribe verbatim; it interprets, selects and restructures. It can confuse a historical condition with a current diagnosis, omit a negative finding, or convert a discussion of a possible medication into an active prescription. Reviewing the output means more than scanning for spelling mistakes. Under pressure, clinicians may accept polished, grammatically correct notes that are clinically wrong. Liability sits with the practitioner who signs the note, not the algorithm that drafted it.

Windows and IT administrators face a different set of risks. Healthcare AI depends on endpoints, browsers, cloud identity, EMR integrations, microphones and APIs. If a nurse uses a personal phone to access an unsanctioned AI service because the approved tool is too clunky, patient data leaks outside managed controls. Stolen credentials can turn a clinical recommendation engine into a vector for manipulating care. “Every AI action should be attributable to an authorised person or service,” Trujillo emphasises. Shared accounts, broad permissions and persistent integration tokens scramble that audit trail.

How we got here – from drug-interaction checkers to ambient scribes

Healthcare AI has moved through distinct waves. Early rules-based systems checked drug interactions and flagged abnormal lab results. Machine learning found patterns in imaging, pathology, electronic health records and population data. Generative AI changed the equation because it is easier to access and appears more flexible. A clinician no longer needs a specialised dashboard; they can ask Copilot to summarise a discharge summary.

That apparent simplicity is deceptive. A note-typing assistant that seems like familiar office software can become a de facto diagnostic suggestion engine without anyone deciding to upgrade it. This use drift – where a tool validated for one purpose drifts into another – is invisible to a procurement office. It happens when a clinician discovers a helpful new prompt or workflow and shares it with colleagues.

Australia’s regulatory landscape is evolving to catch up. The Therapeutic Goods Administration regulates software that meets the legal definition of a medical device, based on intended purpose rather than the technology label. A digital scribe that merely transcribes may fall outside TGA oversight, but if it starts generating diagnosis suggestions or treatment recommendations, it crosses into regulated territory. The Australian Commission on Safety and Quality in Health Care’s 2026 National Model for Clinical Governance explicitly places digitally enabled care, including AI-supported decision-making, under board and executive responsibility. That means governance cannot be delegated to an innovation team.

What to do now – practical steps for every stakeholder

For healthcare boards and executives

  1. Classify every AI use case by risk and intended purpose. A low-risk scheduling assistant does not need the same governance as a model influencing diagnosis, but neither should bypass privacy and security review.
  2. Demand evidence beyond aggregate accuracy. When a vendor says the model is 95% accurate, ask: does performance differ by age, language, location or cultural background? Where do false negatives cluster?
  3. Write suspension criteria into deployment decisions. Predefine the conditions – a serious adverse event, sustained accuracy drop, unresolved data breach – under which the tool must be paused. A governance process that can only approve is incomplete.
  4. Commission independent validation in your own environment. Testing must involve your workflows, your devices, your patient population. Don’t rely on a curated demonstration.

For clinicians

  • Review AI-generated notes as thoroughly as you would a trainee’s work. Look for omissions, false statements and misplaced certainty. A grammatically perfect sentence can be clinically wrong.
  • Understand the tool’s intended purpose and limitations. If a vendor says the scribe is “not for diagnosis,” do not use it to generate differential diagnosis lists.
  • Protect your professional judgement. Under time pressure, it is tempting to accept automated suggestions. Track how often you override the system. If that number is implausibly low, automation bias may be creeping in.

For Windows and IT administrators

  • Inventory active AI capabilities across your estate. AI features now appear inside Windows, Edge, Microsoft 365, Teams and third-party clinical software. Create a living list of what is live and what data flows where.
  • Lock down identity as the control plane. Enforce Microsoft Entra ID, conditional access, MFA and separate privileged accounts. Machine-to-machine integrations should use narrowly scoped credentials, not broad service accounts.
  • Set explicit endpoint policies for AI services. Use app allow-listing, browser restrictions and endpoint DLP to block unapproved consumer AI tools that could ingest patient data. Pair blocking with an approved, usable alternative, or staff will circumvent security.
  • Plan for logs you can clinically investigate. Beyond authentication and file access, you may need prompt history, output versions, model identifiers and source data snapshots. Safeguard those logs because they contain ePHI.
  • Test fallback procedures under realistic conditions. If the AI goes offline, can your clinicians revert to manual documentation and decision-making without patient harm?

For patients

  • Ask your provider if AI tools are used during your care. A legitimate question for any consultation: “Is an AI system listening or contributing to my record?”
  • Request a plain-language explanation of consent. You have a right to know what information is captured, where it is processed, how long it is kept, and whether it trains a product. If you are uncomfortable, ask for an alternative.

What to watch next

The 2026 National Model for Clinical Governance gives boards a stronger lever to treat AI as core business rather than an IT experiment. Expect accreditation bodies to integrate AI governance into their standards, forcing organisations to show operational controls, not aspirational policies. Privacy regulators are already scrutinising ambient scribe consent practices; formal enforcement action is likely. And the market needs independent evidence comparing AI-enabled care with standard practice on safety, equity, workload and cost – not just documentation speed.

Australia has a narrow window to build a model where boards own the quality of digitally enabled care, clinicians retain meaningful judgement, patients receive genuine transparency, and IT teams protect the infrastructure connecting every decision. The alternative is a slow, invisible embedding of systems that are poorly understood, lightly supervised and extremely difficult to unwind when they fail.