Posted by:
Paul Muir, Chief Customer Officer at risr/
I have this week been at ANZAHPE 2026 in Christchurch, which had the theme “Weaving Connection”, where the health professions education (HPE) community gathered with generative AI sitting, once again, squarely on the agenda. The core message was stated plainly and early. GenAI and large language models are reshaping health professionals education, and educators now need pedagogy that makes students responsible users, not merely enthusiastic ones.
But the line I keep coming back to was said in passing early one morning and it has the useful quality of sounding obvious while being nothing of the sort.
“If it works, you should be able to demonstrate it.”
Hold that thought…
1. Being the human in the loop
The conference opened with a workshop titled “Being the Human in the Loop: Testing and Evaluating AI-Based Tools in Health Professions Education”. Regular readers will know I have been circling this phrase for weeks. In edition 5, from the eAA Conference in London, Tim Burnett ‘retired’ Human in the Loop and handed us a sharper one: Expert in the Loop. The distinction is not cosmetic. As this blog has stated before, a human in the loop can be anyone with a login and a deadline. An expert in the loop is the person who can actually catch the error the machine made, because they know the domain well enough to see it. The safeguard was never the human. It was the expertise.
A workshop on testing and evaluating AI tools is that argument made operational. Being the human in the loop is not a governance checkbox you tick once the tool is bought. It is the ongoing, unglamorous work of deciding whether the thing does what it claims. Which returns us to that line. If it works, you should be able to demonstrate it. In health education and assessment, where the downstream consequence of a wrong inference is not a bad grade but a harmed patient, “demonstrate it” is not a nice-to-have. It is the whole job.
2. The ethics were not an afterthought
Where some conferences tend to treat AI ethics as a closing slide, here a problem-posing workshop took the concerns seriously and specifically.
- Bias and misrepresentation.
- Ownership, copyright and intellectual property.
- Data protection and the legal risk of mishandling patient data.
- The fair concern that the benefits of GenAI concentrate in wealthier populations.
And the environmental footprint the sector too often waves through as ‘tomorrow’s problem’. One presentation noted that data centres already consume around 3.5 billion litres of Sydney’s drinking water a year.
The concern most specific to this part of the world was GenAI and Indigenous Peoples, and it deserves more than a passing mention. The workshop named cultural flattening; 250 diverse languages and cultures across Australia rendered, by a generative model, into one generic and less diverse image. It examplared AI-generated images of Indigenous Peoples used in the “No” campaign during the Voice referendum, misinformation that shaped a political decision. This is not bias as an abstraction. It is the ‘Attribution Gap’ made concrete. An output that looks authoritative, but is detached from any accountable source, doing significant damage in the real world.
3. Competence is inferred, and the machine unsettles the inference
One of the sharpest methodological threads of the week was the tension between subjective judgement and objective evidence, and one claim in particular. That competence is inferred, and LLMs unsettle that inference. I think that is exactly right, and it is worth being precise about why.
We almost never measure competence directly. We observe a performance, a consultation, a written note, a diagnosis, and we infer competence from it. That inferential chain is the whole edifice of assessment validity. Michael Kane’s argument-based framework names its links, from scoring to generalisation to extrapolation to the decision the score is used to justify. An LLM does not break the top of that chain. It corrupts the bottom. When a student can generate a fluent, structurally impeccable clinical note without the reasoning that used to be required to produce it, the observed performance stops licensing the inference we used to draw from it. The note still looks like competence. It no longer reliably signals it.
I am not alone in this. A live body of work, Corbin, Dawson, Bearman and colleagues, and Roe’s notion of “assessment twins”, is circling the same problem from the GenAI side. It argues that we must redesign assessment around exactly which inference AI threatens, rather than reaching for a better detector. The honest conclusion is uncomfortable. Much of what we assess was always a proxy. AI has simply made the proxy cheap to fake, and forced the question we should have asked years ago. What were we actually trying to infer?
4. Enthusiasm on the slides, hesitancy in the room
A striking graph put the AI adoption picture up in lights. 63% of global educators using AI, 75% of Australian educators and 94% of US based-educators profess to using AI weekly. The use cases skew towards the ‘safe’ adoptions: brainstorming, editing, literature searches, or efficient administration. Mapped onto the old adoption curve, a large conservative majority sits between the evangelists and the objectors.
Read those numbers next to the mood in the room and you find us very close to the theme of the week. High usage, but cautious uses. Seeing enthusiasm on the slides, but hesitancy in the practitioners. That gap is not irrational. It is expertise doing its job. Our true experts in the loop!
5. Assessing communication, and the dental provocation
I saw plenty of GenAI used to assess communication, in dentistry, mental health and physiotherapy among others. This is close to home. risr/’s AI Coach lets learners rehearse real conversations with an AI role-player and get instant, rubric-based feedback against an organisation’s own criteria, with a clinical variant built for medical consultations. Realistic practice, at a scale the majority of faculties could match in the short-term. That is the ‘Mirror Moment’. Not a machine that passes for the examiner, but feedback no examiner could produce that fast, with the human still holding the judgement.
I’ll briefly focus on the dental session’s content, which was on e of the highlights on the conference. It called out two opposing view points on AI:
Promising with AI: more targeted practice, faster formative feedback, greater availability of cases.
Uncertain with AI: transfer to independent clinical performance, long-term retention, generalisability, safety, bias, privacy and over-reliance.
Did you notice which grouping carries the weight? Everything promising is about the practice. Everything uncertain is about whether the practice becomes competence. A tool can generate a case, but the student still has to demonstrate durable reasoning and safe judgement. We are, so far, measuring engagement and confidence and calling it learning. Confidence is the one thing we least want a clinician to have in excess of their competence.
6. The merging of VR and AI
A quieter, faster-moving front was seen in some of the later sessions. The convergence of virtual reality and AI, increasingly with wearables feeding live physiological data into simulations. This is where re-innovation stops being rhetorical. Bolt AI to a flat case and you have an accessory. Build an adaptive, sensor-aware simulation that responds to a learner in real time and you have something the old learning and assessment model could not do at any price. The first is impressive. The second is genuinely new, and the demonstration bar is correspondingly higher.
The Measured Response
The key takeaways discussed at one of the final sessions stated: GenAI can scale clinical education safely, that academic design is essential, that virtual simulation complements real-world practice, and that a framework supports adoption. I agree with every word, and every word is a promissory note. “Can”, “supports”, “complements”. Each one is a claim about the future written in the present tense.
So I will return to the beginning of this blog. If it works, you should be able to demonstrate it. Not that it is engaging. Not that learners feel more confident. That it produces competence that transfers to a patient who never agreed to be part of the pilot.
Health professions education has a virtue the rest of the sector should borrow. It already knows that the burden of proof sits with the claim, not the doubt. The hesitancy I saw was probably not the sector lagging. It was the sector doing to AI exactly what it does to every intervention. Show me the evidence.
That’s the Measured Response.
Over to you…
The homework (for the comments section): when an AI tool boosts a learner’s confidence and engagement but nobody has yet shown it improves competence that transfers to practice, is adopting it good pedagogy, or a bet dressed as one?
The enthusiasm is real, and so is the hesitancy. In HPE, right now I would rather trust the hesitancy.
Until next week,
Paul
