Posted by:
Paul Muir, Chief Customer Officer at risr/
Last week I asked the question I always ask of a good event. Not whether it would impress you, but whether it would convince you and what, exactly, would have to happen in the room for the difference to show. Three days at the e-AA Conference later, I’ll return the verdict on my own challenge.
1. The regulator on a tightrope
The theme, The Trust Imperative, got one of its most authoritative readings from Michael Hanton, Ofqual’s Deputy Chief Regulator, whose keynote on innovation in high-stakes assessment set out to balance opportunity, risk and trust in a single breath. It was, rightly, all about equity and fairness. The proposition that innovation which widens the gap between candidates isn’t innovation at all, it’s just newer unfairness.
Nobody in the room argued with that. Which is the tell I flagged last week: when everyone agrees loudly on the diagnosis, watch the prescription.
So here is my question, and it is a sincere one. A regulator’s instinct, the one I set out in edition 4, is to move really fast on threats and really slowly on the technology, and there is real wisdom in that asymmetry when the stakes are a young person’s future. But caution carries a cost we almost never put a number on.
“Will risk-averse regulation slow the sector’s adoption of AI so far that we forfeit the very gains, in access, in fairness, in cost, that the technology was meant to deliver?” The candidate harmed by a reckless tech rollout is easy to picture. The candidate harmed by the tool that arrived five years too late is invisible, but no less real.
For what it’s worth, the exhibition floor offered one answer to the reflexive “but we can’t trust it”. Trust you can actually audit, in the shape of assessment and proctoring platforms now certified to ISO/IEC 42001 for responsible AI management now exist. A small, but significant step, as a certificate is a good deal more than a slide claiming you’re careful. Balancing opportunity, risk and trust is exactly the right brief. My worry is that “balance,” in practice, too often resolves into “wait.”
2. Accessory or architecture: the 25-year question, answered in the room
The AI Symposium panel session I was part of took the long view, and the re-innovation test. Do you treat AI as an accessory you bolt on, or as architecture you build from, turned out to be a serviceable lens for the entire week, not just our forty minutes.
We asked the audience to tell us what they thought assessment would look like in 1, 10 and 25 years. Their 25 year responses hinted at a future where assessment is likely to be more continuous and integrated, but concerned about who would potentially own those assessments? GCSEs or Apprenticeships owned and powered by Anthropic or Meta in 2040?
Most of what was on show was still what I class as ‘AI-as-an-accessory’. The same exam, the same item bank, the same delivery model, now with a model stapled to the side and a confident noun in front of it. Useful, often impressive, rarely re-innovation. As I said in the panel, our last 25 years have seen us go from 2001 with candidates sitting in an exam hall, taking exams on pen and paper. In 2026, we see candidates sitting in an exam hall, taking exams on pen and paper…
Will the next 25 years, ‘doped’ by AI, finally make a difference?
3. From considerations to practice, and the considerations mostly survived
The Human Oversight panel was billed as the move from considerations to practice, and the honest finding is that the considerations held up under contact better than I feared, but the failure mode I warned about is alive and well. Theatrical oversight is everywhere: the reviewer rubber-stamping a thousand machine decisions an hour, the human in the loop who is really just automation bias with a job title.
What’s changed since we wrote the white paper is the deadline. Article 14 has stopped being a thought experiment, and a deadline concentrates the mind in a way a principle never does. The room knew it. The question now isn’t whether oversight should be meaningful, but who pays for the version that actually is, and that conversation has barely started.
A conversation after the panel put it very well to me. There is a mis-understanding in the sector as a whole and the concept of Human-in-the-Loop is being applied too uniformly. We are seeing that worrying check-box approach.
However, there was a view on Human ‘involvement’ in AI, that really captured the minds of many in the conference, with it being repeated multiple times across the 3 days.
Step forward, Expert in the Loop…
4. Expert in the loop and the vibe coding question
A session on vibe coding is not where you’d expect the oversight debate to be settled. But in his, Tim Burnett ⚡ challenged us to retire a phrase we have all leaned on too hard, “Human-in-the-Loop” and handed us a better one: Expert in the Loop.
It sounds like a small edit, but I genuinely don’t think it is. A human in the loop can be anyone with a login and a deadline, which is precisely how you end up with the ‘automation bias with a job title’ I warned about earlier. An expert in the loop is the person who can actually catch the error the machine made, because they know the subject well enough to see it. The safeguard was never the human; it was the expertise. Tim’s phrase says out loud what the oversight conversation keeps implying and never quite commits to. A warm body is not a control.
The same session posed the provocation I have not been able to put down: could vibe coding revolutionise the assessment industry? I’ll argue it both ways, because it deserves both.
For the motion: Yes, and sooner than we’re comfortable with. Vibe coding collapses the distance between an assessment idea and a working build. The psychometrician who can describe an adaptive simulation can now make one, without joining the back of a development queue. The performance tasks and scenario engines that were always too expensive to commission become cheap to prototype. That is re-innovation made affordable, not AI bolted to the side of the old exam, but the old build cost, the very moat that protects legacy formats, quietly draining away. Lower the cost of building something genuinely new and you get more of it.
Against the motion: No, and the theme of the week is the reason. Assessment’s hard part was never the code, it was the construct. Vibe coding speeds the build, not the thinking, and “by feel” is the exact opposite of the validity, reliability and auditability a high-stakes score demands. Lower the barrier to building and you potentially also lower the barrier to building badly. A flood of plausible, untested, psychometrically hollow artefacts that nobody can quite account for when the appeal lands. You cannot defend a score you cannot explain, and you cannot explain what you waved through on a vibe.
So which is it? Both. Vibe coding will revolutionise the production of assessment whether we plan for it or not. Whether it revolutionises the discipline turns entirely on Tim’s distinction. It is a revolution only if the expert stays in the loop. Not signing off the build, but owning the construct.
Removing the expert and vibe coding is simply a faster way to deliver things you possibly cannot defend. Keep them in, and it may be the most useful tool the field has been handed in a decade.
5. Guilty until proven human
The last of my three sessions, with Isabelle Gonthier and Claire McCauley, was the one I most wanted to test in front of a live audience, because the argument is uncomfortable. We have drifted, almost without deciding to, into a model that presumes the candidate cheated until they prove they didn’t. Purely on the word of detection tools wrong often enough that the false positive has become a feature rather than an edge case.
At ATP in March 2026, 73% of people in the audience agreed that we should start from a position of “Guilty until proven Human”. At our session at eAA, this moved to 85%. Wow…
If your security model depends on cleanly sorting humans from AI, you have already lost, because the line is dissolving while we argue about where to draw it. As I argued at the Cambridge Assessment Network conference in April, the robust move is to design the construct so the distinction matters less, security as a stack, not a single tripwire at the door. Reverse the burden of proof and you don’t catch more cheats; you teach honest candidates that the system regards them as suspects. Nothing drains trust faster.
6. The student who doesn’t use AI
Here is the one of the most useful things I heard all week, and it came from the person in the building with the least experience of our sector. On Tuesday, an A-Level student told the RM keynote panel, plainly, that they don’t use AI. At all.
Sit with that for a moment. We have built an entire panic detection arms race, integrity summits, the burden-of-proof inversion I spent a session arguing against, on the assumption that every teenager runs every piece of coursework through an AI tool as a matter of routine. And here was one, to our faces, suggesting the picture in our heads might be a caricature. Are we overestimating how much young people actually use this stuff? Are we busy designing security for a candidate who doesn’t exist, while the real one sits there wondering why the test still feels like 2009? Or was this student the anomaly? RM Assessment ‘s keynote on bridging generational expectations made the same point from the other direction: we keep mistaking our assumptions about a generation for the generation itself.
The contradiction sharpened the instant you stepped outside the conference. Well, virtually anyway… Because in the same week, across town, the Prime Minister stood up at London Tech Week and promised AI tutors for the 450,000 of the most disadvantaged children in the UK. Explicitly, in his words, to “close the attainment gap.” So the policy direction is unambiguous. AI in the lesson, personalised, state-funded, aimed squarely at the some of the most disadvantaged. And yet at the other end of the same journey we still deliver virtually no K-12 assessment digitally, and on the sector’s own (regulated) timeline we won’t at any meaningful scale until way beyond 2030.
Read those two facts together and wince. We are preparing to teach/support a generation with AI tutors and then assess them with a pencil. The learning gets the future; the exam hall keeps the past. If you wanted a single image of a sector out of step with the world it serves, there it is.
7. Losing the story
Which brings me to the session I suspect mattered most and will trend least. Geoff Chapman and Oliver Chesher made a case the sector should have tattooed somewhere it can’t avoid reading. Stop expecting the celebrity misinformants to play fair, and stop answering them with footnotes.
Their argument, in essence (in my view), was that while we publish earnest, rational explainers on accessibility and ship clever new features almost nobody uses, the public conversation about digital assessment is being written for us by other, less-informed people. And we are losing that argument before we have noticed we’re in a fight. Digital Assessment, they warned, is at risk of becoming politically toxic and not because digital is bad, but because it is badly explained, by a sector with little sustainable advocacy. What’s missing, they argued, are competing stories that feel equally human.
Enter, with exquisite timing, Hugh Grant. This very week he took to X to amplify an essay on resisting AI in schools, with a verdict of his own attached: using AI in schools, he declared, is “a betrayal of children.” Five words. No caveat, no footnote, no accessibility annex. The man is a gifted actor and a long-standing campaigner against screens in the classroom, so the intervention was entirely in character and a flawless demonstration of the exact problem Geoff and Oliver had just diagnosed. That single line will travel further than every accessibility paper the sector has recently published. And there is a particular irony in a celebrity who could hire any tutor in London calling it a betrayal to hand the free one to the children who can’t.
That’s the asymmetry, in one news cycle. The misinformants have stories (and amazing PR). We have slide decks. And a story beats a spec sheet every single time, until the sector learns to tell better ones.
The Measured Response
So, convinced or merely impressed?
My honest verdict is that the week was long on impression and a little shorter on conviction and that isn’t a complaint, it’s the diagnosis. A sector that agrees loudly on the problem and quietly on the cure is a sector that knows it has work left to do.
But the deeper worry isn’t inside the room, it’s outside it. We are extraordinary at the internal conversation. The oversight spectrum, the construct, the expert in the loop, the validity of the score. We are losing the external one, to politicians with budgets and celebrities with reach, while the regulator inches along its tightrope and the public watches an entirely different show, or has changed the channel 50 times to find Hugh’s latest movie.
And I keep returning to the question I closed with last week. Whose trust are we talking about? After three days, the honest answer is that we spent most of our energy earning the trust of regulators and awarding bodies, a little on the candidate, and almost none on the public who will ultimately decide whether digital assessment is allowed to grow up at all. Integrity was never the brake on innovation, it’s the specification. But a specification you cannot explain, to a worried parent, to a sceptical student, to an actor with a megaphone, is a spec that loses anyway.
That’s the Measured Response.
Over to you…
Here’s the homework (for the comments section):
What is the one human story about digital assessment you could tell a worried parent? In a sentence, no acronyms, and who will you tell it to before the politicians and the celebrities finish telling it for you?
Be honest. The sector that wins the next decade won’t be the one with the best technology. It’ll be the one with the better story.
Until next week…
Paul
Link to the article from Hugh Grant’s ‘tweet’ is here: https://annelutzfernandez.substack.com/p/resisting-ai-in-education-part-iii
