RCGP piloted risr/’s AI Clinical Coach to explore whether artificial intelligence could provide scalable, on-demand, formative consultation practice aligned to the Simulated Consultation Assessment (SCA) – the licensing examination for general practice.
Since the launch of the SCA in November 2023, GP registrars have consistently expressed interest in additional opportunities to practise consultation skills. Existing RCGP preparation resources and courses are well regarded by candidates and continue to form an important part of the preparation landscape. However, consultation skills development benefits from repeated, flexible practice opportunities that are challenging to deliver at scale through traditional formats alone.
The AI Clinical Coach was developed to complement existing preparation resources by providing scalable, on-demand formative consultation practice. The platform combines a simulated AI patient with an optional AI coach, drawing on the SCA’s detailed case design, role-player briefs, and structured assessment domains to support realistic consultation rehearsal and generate feedback aligned to the style and expectations of the assessment.
Participants engaged meaningfully with the platform throughout the pilot, completing an average of three cases each, with substantial time spent interacting both with the AI patient and the optional AI coaching component. The findings suggested that AI-supported simulation may offer valuable opportunities for repeated consultation practice, reflection, and feedback. It also highlighted that technology is only one part of the solution. Strong educational assessment design, clinical oversight, and quality assurance need to be in place for the experience to be robust and authentic.
Q1. Was RCGP facing any challenges with the existing SCA prep opportunities offered to candidates before exploring AI-based solutions?
Yes. Since launching the SCA in November 2023, GP registrars requested more preparation resources and training opportunities. RCGP’s preparation courses are highly-valued by candidates but they are resource-intensive to deliver and difficult to scale. This means that not all candidates who like to attend are able to access them. As a result, many candidates seek additional preparation through commercial providers at significant cost. There was also recognition that effective SCA preparation benefits from repeated practice, which traditional courses don’t easily provide with on-demand availability.
Q2. How would you summarise the experience and impact of risr/ AI Clinical Coach during the pilot?
While not all volunteers who initially expressed interest went on to participate fully in the pilot, those who did engage with the platform used it meaningfully and often repeatedly. Participants didn’t just try a single case; they completed multiple cases and spent meaningful time with both the AI patient and the AI coach. On average, participants completed three cases each, with about 10 minutes 45 seconds in consultation with the AI patient and a further 4 minutes 45 seconds with the AI coach. This suggests genuine educational value and willingness to use the platform repeatedly as part of their preparation.
Q3. What stood out most about the AI Clinical Coach compared to traditional methods?
The standout was the combination of scalability and repeatability. High-quality, in-person training courses are valuable but limited in reach and repeat availability.
The AI platform offered repeated consultation practice and structured feedback in a flexible, scalable way.
The SCA’s detailed case design, briefs, guidance, and structured mark schemes translated well into prompts for the AI patient and coaching components, enabling the AI to simulate consultations and generate feedback aligned to assessment domains.
Q4. How did participants respond to interacting with the AI Clinical Coach?
Participants showed notable engagement with both the simulated consultations and the coaching. After each consultation, they spent an average of 4 minutes 45 seconds interacting with the AI coach, even though coaching was optional.
Importantly, 95% of survey respondents agreed that using the tool increased their insight into their own strengths and weaknesses, indicating the tool supported formative learning rather than just being a novelty.
Q5. Was there any feedback suggesting the AI might enhance or change the candidate assessment experience?
The feedback remained focused on the potential value of the tool as a formative preparation resource. It was not viewed as something that would change or influence summative assessment decisions.
Q6. Can you share any specific feedback from the pilot where participants identified where the AI solution could add value?
International medical graduates, who may be less familiar with UK general practice communication and consultation style, found the tool particularly useful. It allowed them to repeatedly practise consultations, receive structured feedback. and refine their consultation approach over time.
Participants also provided helpful feedback on areas where the experience could be improved further, notably latency in AI responses and consultation flow, and that at times the AI patient offered too much information spontaneously. There was also evidence that the AI patient sometimes adapted responses based on expected consultation patterns (e.g., quantifying pain), which occasionally reduced the realism of the interaction. Overall, feedback emphasized refinement and optimisation rather than concerns about the platform’s educational value.
Q7. What would you say to other institutions considering piloting and adopting AI for formative assessments?
The potential of AI for exam preparation is significant, but the key is strong educational design. The pilot benefited from the SCA’s existing, detailed case design, role-player briefs, consultation structure, and mark schemes, which translated effectively into the AI prompts and scaffolding.
It’s important to articulate a clear purpose for the tool—as a formative resource for practice, reflection, and feedback rather than a substitute for summative assessment. There is potential for broader use in CPD and ongoing professional development, but that requires equivalent educational scaffolding and clinical oversight to maintain educational integrity and realism. The AI solution can provide scalable, repeated practice and reflection, but success depends on preserving educational validity and robust quality assurance.
Want to learn more?
If you’d like to know more, we’d love to give you a straightforward look at how we can help you drive better learning with institutional access and pilot opportunities.
