Mind the Gap – Measured Response #3

Welcome to the third edition of Measured Response

Last week I asked whether we’re in a cheating pandemic and concluded, with apologies to anyone hoping for reassurance, that “pandemic” is an accurate diagnostic frame rather than hyperbole.

My time this week was spent in two very different rooms that turned out to be describing the same symptom. Two webinars. One an exam-security summit panel, the other an assessment podcast. Different formats, (subtly) different audiences. And yet both, independently, arrived at an idea I haven’t been able to shake off. We are looking in the wrong place.

So this week, ‘Mind the Gap’.

Two rooms, one blind spot

The first was the AI-DOME Exam Security Summit, on a panel with the deliberately uncomfortable title “Why the Most Dangerous Cheating Now Hides Between Exams, Not Within Them.” I shared it with Liberty Munson of Microsoft and Isabelle Gonthier, PhD, ICE-CCP of ETS/PSI and Rajeev Menon on of AI DOME .

Link here: https://www.youtube.com/watch?v=q1J0v2NEIqg

The second was Isabelle’s own Tried and Tested podcast (yes, the same Isabelle, twice in one week; the assessment world is reassuringly small!) on Test Security, Agentic AI, and the Future of Assessment and the ATP community.

Link here: https://www.psiexams.com/knowledge-hub/test-security-agentic-ai-and-the-future-of-assessment/

I went in thinking I’d be making two different arguments. I came out realising I’d been making one.

Five examples, five different parts of the sector. Same pattern.

The crime scene is empty

The AI Dome panel’s central provocation: we call between-exam cheating “the exception” only because our instruments are built to look inside the session. A clean session feels like proof of integrity. Increasingly, it’s proof the session simply had the wrong focus.

Take three things we usually file as separate problems.

• Impersonation and proxy rings, where the imposter is often genuinely competent, the session passes every check, and the fraud lives upstream in the identity chain.
• Item harvesting, which is designed to be distributed across many sittings so no single one looks abnormal
• Collusion-as-a-service, the Telegram groups and paid feeds where each session is spotless but the operation is a network.

Three problems? No. One. A program-level adversary, met by a session-level defence. We catch the amateur who panics in the session, and miss the professional who never really enters one. And the threat I think we underplay most is the laundering of harvested content back into “legitimate”, AI-regenerated prep. It erodes validity silently; no session ever looks dirty.

The answer isn’t “buy a better camera” or “layer on more and more security.” It’s to grow up from detection into intelligence, which sounds glamorous and absolutely is not. Intelligence here is a cross-session pattern layer (exposure drift, response-time anomalies, centre and IP clustering, recurring identities) and the decisions it produces are gloriously mundane: rotate the form early, hold three results, refresh the content that’s leaking. It’s just the unglamorous discipline of acting on weak signals. Aggregation tells you what happened; intelligence tells you what to do, and stakes its reputation on occasionally being wrong. If your dashboard never once forces a hard decision, it isn’t a security tool. It’s a reporting tool wearing a security badge.

There’s a sting in the tail, though. The moment we move to program-level intelligence, the power asymmetry changes: a session can wrong a candidate once, but a program builds a profile across years. Liberty’s idea of a ‘security credit score’ is a really interesting one and one I’m sure the EU AI Act would have a few questions about! I’d love to explore more though…

So my line on the day was this: the more we can see, the more we owe. More transparency, more recourse, more restraint, not less. That’s the ethical stance and duty of care we owe the honest majority whose credential/assessment the cheats are busy devaluing.

“Guilty Until Proven Human”

If the panel was about where we look, a key component of my discussion on Isabelle’s Tried and Tested podcast was about who is actually sitting there.

2026 is the year AI stopped being a tool and became a teammate, or, less comfortably, a threat-actor. The risk I keep returning to is something I’ve called the “Digital Double”: an autonomous agent that doesn’t merely look an answer up, but inhabits a candidate’s digital identity and performs the exam from start to finish. We spent years worrying about a cheat sheet. We’re now up against a proxy. The mouse, as I may have said once or twice this year, now has a jetpack.

Two consequences follow. First, the static item bank is, I’m sorry to report, as dead as a Norwegian Blue parrot. An AI agent can scrape, solve and redistribute your entire bank in roughly the time it takes to read this sentence.

The opportunity hiding inside that Monty Python themed obituary is real though; generative, performance-based tasks built in real time, where we stop testing what a candidate remembers and start testing how they reason. It’s much harder to buy the answers to an exam that didn’t exist until the moment you logged in. Secondly, if identity itself is now contestable, verification can’t be a one-time check at the door. It has to be continuous, watching behavioural and biometric signatures across the whole interaction rather than staging one high-stress checkpoint and hoping for the best.

What struck me most was the mood. The panic phase of AI I believe is over. Last year the sessions of ATP’s Innovations conference were full of “how do we block AI.” This year it’s “how do we orchestrate this responsibly”, with humans/experts in the loop as the ethical sanity check, rather than the automation bias that quietly hands over the keys to save a few Pounds/Dollars/Euros. (Other currencies are available.)

The Measured Response

So here’s what two rooms in one week taught me: our deepest problem isn’t detection, or proctoring, or even AI. It’s that we’ve been measuring integrity as a moment when it has quietly become a pattern.

Both the between-exam threat and the “Digital Double” exploit the same gap, the space our security instruments don’t watch: between sessions, between checkpoints, between “you logged in” and “you’re still you.” Fix either in isolation and you’ve treated a symptom. Close the gap and you’ve started eradicating the ‘disease’.

Which folds back into last week’s blog. Security as a stack, not a product; cross-session intelligence is simply one more tier of it. Construct validity as the security question; because if an agent that has never seen a patient, a courtroom or an IT server can ace your exam, your security problem is always downstream of a measurement problem. This is what I mean by re-innovation rather than redesign: not a better lockdown browser bolted onto a format already living on borrowed time, but an assessment an AI agent genuinely can’t fake, which conveniently, is also one infinitely more worth taking.

That’s the Measured Response.

Over to you…

Three deliberately provocative ones this week:

• A clean session is the single most dangerous assumption in our industry right now. Agree, or is this too dramatic?
• “Guilty until proven human”: necessary realism for the agentic AI era, or a candidate-experience disaster we’ll regret building?
• Should security features be a tool of competition for vendors, or simply one that we as an industry collaborate upon? It’s where the role of ATP is particularly interesting!

Comments are open.

Until next week,
Paul Muir