Proof or it didn’t happen: Assess it like it’s research

The views expressed here are my own and do not represent those of the University of Oxford.

If assessment were a research paper most of our methods sections would get rejected.

Currently we’re trying to certify what students can do using approaches that don’t actually produce proof. The old problems (contract cheating, ghostwriting) were already munching away at validity but generative AI has just made it impossible to pretend that bans plus detection are effective enforcement mechanisms for unsupervised tasks.

This isn’t a lament. It’s a call to bring our assessments up to the standard we demand in our labs and journals: proof over promises. Ideally, we don’t publish on vibes, so we shouldn’t grade on them either.

The problem

I work as an AI consultant specialising in education and in my role at the University of Oxford’s AI Competency Centre, people regularly come to me for advice on how to address AI in learning and assessment. Oxford’s approach to assessment integrity currently rests on two mechanisms: declarations and detection. Students are asked to report their AI use; assessors are encouraged to spot “AI hallmarks” in submitted work.

Neither mechanism produces proof.

Declarations

AI-use declarations assume compliance rather than verify it. They’re an honour system in which we ask students to self-report, then treat those reports as evidence. But self-reporting only catches honest students or careless ones. It was never designed to and cannot produce defensible proof of authorship or independent capability.

The result is enforcement theatre: declarations create the appearance of control while providing no actual verification. We’ve shifted responsibility onto students without doing the hard work of redesigning assessments to generate observable evidence. And when declarations inevitably fail to catch misconduct, we compensate with ever more detailed guidance documents and rubrics, while quietly relying on post-hoc detection to do the real enforcement work.

I hope wouldn’t publish findings based on ‘we asked participants if X occurred’ when X is objectively verifiable but unobserved. We’d observe X directly or design around the limitation. Assessment should work the same way.

Hallmark-spotting

Lately, I’ve also started seeing a pattern: assessor “vigilance” and lists of “AI hallmarks” are being promoted as ways to identify AI use in student work. These hallmarks include features like formulaic structure, inconsistency, wordiness, surface-level reasoning, repetition, unnatural flow, North American spellings, even both “inaccuracy” and “accuracy.”

There are two major problems with this approach.

Firstly, it’s both unsound and unfair. Those “tells” are also the hallmarks of novice academic writing. Of students learning to structure an argument, use evidence, and write within conventions. So if your only evidence is “it reads like a tidy, five-paragraph essay with stock transitions that has factual mistakes in it,” then you don’t have evidence of misconduct; you have evidence of developmental writing, and in that case, mark the work at face value against the outcomes. Don’t turn a growth stage into a tribunal.

Secondly^¹, it doesn’t even do what it says on the tin. Hallmark-spotting can only ever catch unsophisticated AI use. The students who are already good at prompting, iterating, revising, and defending their choices can (and will) evade “tells.” The result is a system that over-polices the least experienced writers and misses the most capable AI users which is precisely backwards if your aim is fairness or assurance of learning.

Under real conditions there’s also no ground truth for text provenance, which means “detector scores” or “similar-to-an-LLM answer” comparisons aren’t verifiable evidence. Good integrity processes require evidence that meets a standard; probabilistic guesses and stylistic vibes do not. So here’s a simpler rule of thumb for markers: if your only “evidence” is a list of stylistic features that also occur in human novice writing, just grade the work on its merits.

We are fortunate that Oxford’s education policy explicitly does not endorse digital AI detectors, however we should also ensure that we don’t backslide into other forms of performative vibes-based detection while trying to do the right thing.

Why both approaches fail together

Together, declarations and detection create a system that can only catch honest students or incompetent AI users. We ask students to report their own misconduct (compliance), then use stylistic tells to verify when they don’t (detection), but neither mechanism produces defensible evidence of what the student can actually do. We’re running quality assurance on promises, not proof.

Is that the evidential standard we want from an institution like Oxford?

Assessment as research design (methods > promises)

One of the most important functions of assessment is to make a defensible inference: this student can X. Cheating only matters insofar as it threatens that inference, and when we treat integrity as a style-policing problem, we undermine validity and equity at the same time. However, if we instead approach it as a methodology problem, we can actually begin working towards solutions.

This is why AI use declarations are theatre. They shift responsibility to students while providing no verification and in essence we’re asking students to report their own misconduct, then grading their honesty rather than their capability. Tools like traffic-light frameworks and institutional guidance statements can be useful for signalling expectations and for illustrating the different ways AI can be integrated into the writing process (for better or worse), but they can’t produce robust or defensible evidence. In unsupervised settings, relying on honour systems and voluntary compliance creates an “enforcement illusion“: we compensate with endless guidance rather than doing the hard work of redesigning assessments, and inevitably end up leaning on vibe-checks.

The shift we require is from rules to proof.

A better backbone: Invigilation • Interaction • Integration

Soon after ChatGPT-3.5 was unleashed onto education, I realised that there were only three realistic ways to approach assessment. After starting my role here at Oxford and working with Dominik Lukeš I discovered he had come to the same conclusion, but with a much catchier title - Invigilation · Interaction · Integration. I’ve adopted his framing here and expanded each in turn:

1) Invigilation

Secure what must be certified

When the claim is high-stakes individual capability, design the task so the evidence is produced under observed conditions. That can be an exam, a lab practical, an in-person performance, an authenticated live task. Oxford is already strong in this domain: written exams and practicals are visible, auditable evidence. However, they need to be used strategically, not ubiquitously – and always with careful attention to accessibility and inclusion.

2) Interaction

Test thinking, not writing

For unsupervised work, add oral checks (vivas, oral essays) that ask students to verbally reproduce, explain, and extend their own arguments or outputs. This verifies authorship and understanding without turning everything into a written exam. Oxford already models this in the tutorial system, so consider scaling that logic with short, consistent checkpoints.

3) Integration

Assume AI use; assess judgement

For take-home tasks, assume AI exists and that most students will use it. The job, then, isn’t to police linguistic markers; it’s to teach effective use of AI tools that don’t undermine learning. This approach demands good pedagogical practice and awareness of AI capabilities: being explicit about what “knowing, thinking, learning, and arguing” mean in your subject, why the assigned task exists, and how that relates to which parts of the cognition the student should be doing themselves rather than offloading to AI. Then align the brief and rubric so that appropriate tool use naturally produces work that meets your criteria, while inappropriate use doesn’t. In other words: guiding students when to use AI, when not to, and how to know the difference because sooner or later they’ll hit a secure assessment checkpoint (or the job market) where their knowledge has to stand on its own.

To make this work in practice, assessments also need to be worth a student’s intellectual effort. The strongest designs give students something worth saying: a real problem to interrogate, a position to defend, a messy case to make sense of. When tasks demand judgement and an original stance rather than mere reproduction, students are more likely to be motivated to use AI as a lens for exploring ideas and a tool for clearer communication instead of a shortcut.

Why this is fairer (and keeps standards high)

Structural validity protects both rigour and equity, while declarations and vibe-based “hallmarks” policing is noisy and biased: it over-flags multilingual, neurodivergent, and racialised students, and it penalises conventional academic structures students are learning to master. Designing for evidence avoids that trap.

This approach also aligns with several of the Russell Group AI principles: ensuring academic rigour and integrity, supporting staff and students to use AI effectively and appropriately, and allowing equal access to generative AI tools. At Oxford, the latter commitments are already being realised through the university-wide rollout of enterprise-licensed generative AI tools, and through the AI Competency Centre’s training for staff and students.

Where we still have work to do is on rigour and integrity. That expectation does not rest solely on how students complete their work; it also extends to all of us designing assessments, because rigour that isn’t evidential is only opinion dressed as certainty.

Examples of concrete reform: program-level design, multiple path/lane approaches, and secure checkpoints

Some institutions are already moving from unenforceable rules to concrete structural change. The University of Sydney’s pragmatic two-lane model is a particularly useful and realistic template. It separates Lane 1: secured, program-level checkpoints that validate learning under supervised conditions that may or may not include AI use — from Lane 2: open, scaffolded tasks that teach and normalise productive AI use. That structure enables both formative learning (where AI can be taught and used) and summative assurance (where independent capability is demonstrably evidenced). Not every model will suit every discipline, but the two-lane framing gives a clear, workable logic for where to place secure checkpoints and where to design for process evidence instead of policing surface “hallmarks.”

Additionally, Australia’s TEQSA guidance maps out three pathways for assuring learning (program-level reform, module/subject assurance, or a hybrid of the two), and it argues that assessment design should create multiple, meaningful points of evidence rather than rely on post-hoc detection which it clearly states is “all but impossible”.

TEQSAs guidance also says that to meet standard 1.1.4 of the HESF, providers must either assess:

1. in a way that incorporates AI use, remembering any AI use is full AI use.

2. in a way where AI use is irrelevant.

3. where AI use is prevented via direct supervision.

What next?

This is hard work. It takes design time, staff development, and quality assurance. But we’re nearly three years into the GenAI era. The real risk now isn’t moving too fast; it’s drifting into a credibility gap where our degrees claim things our methods can’t prove. And there are other benefits: doing this work won’t just address the GenAI challenge. It will also force us to finally fix the assessment security issues we’ve been ignoring for years that have just gotten too uncomfortable to overlook since ChatGPT arrived. So, start the conversations this term to redesign for the next academic year.

Stop policing. Start proving.

Image generated by ChatGPT

1. Does it read like AI when I use these discourse markers? I hear that’s another hallmark.