Research
AI marking accuracy research
We measure accuracy as the gap between DeepMark’s mark and the definitive mark. Every figure is dated, names its engine version and states how many answers it rests on.
Each study marks real student answers that exam boards publish with the mark they awarded, using the same marking engine teachers use, with mark schemes read from the original documents. We publish every mark, not just the averages, and we test the same answers again each time the engine changes.
Key facts
Same mark on all 5 runs
AI marking repeatability
Repeatability: 264 of 264 school answers received the same mark on all 5 runs of the same engine version.
264 answers, 76 papers, 185 questions, 48 teachers, 1,320 runs. Engine version 2026.11.04, tested 5 November 2026.
Read the study
Answers by marks away from the definitive mark
GCSE Geography AI marking, 1 to 9 marks
Accuracy: Matched the definitive mark exactly on 71 of 104 exemplars and was within 1 mark on 94. Average gap of 0.4 marks.
104 exemplars on questions worth 1 to 9 marks. Engine version 2026.11.04, tested 6 November 2026.
Read the study
GCSE History AI marking, up to 16 marks
Accuracy: Average gap of 1.3 marks from the definitive mark. 37 of 57 within 1 mark and 19 of 57 matched exactly.
57 exemplars on questions worth 4, 8 and 16 marks. Engine version 2026.11.04, tested 8 November 2026.
Read the study
GCSE English Language reading AI marking, 15 marks
Accuracy: Average gap of 1.5 marks from the definitive mark. 13 of 31 matched exactly and 23 of 31 were within 2 marks.
31 exemplars on 15 mark reading questions. Engine version 2026.11.04, tested 6 November 2026.
Read the study
GCSE Business AI marking, whole mock papers
Accuracy: 23 of 23 GCSE Business mock scripts were within 4 marks of an independent examiner on a 43 mark paper. Average gap of 2.3 marks.
23 handwritten mock scripts on a 43 mark paper. Engine version 2026.11.04, tested 6 November 2026.
Read the study
Teacher oversight band
Questions where DeepMark is a strong first pass and a teacher should check the marks before they go back to students.
Study pages
- AI marking repeatability264 answers, 76 papers, 185 questions, 48 teachers, 1,320 runs
- GCSE Geography AI marking, 1 to 9 marks104 exemplars on questions worth 1 to 9 marks
- GCSE History AI marking, up to 16 marks57 exemplars on questions worth 4, 8 and 16 marks
- GCSE English Language reading AI marking, 15 marks31 exemplars on 15 mark reading questions
- GCSE Business AI marking, whole mock papers23 handwritten mock scripts on a 43 mark paper
- A level History AI marking, 25 marks44 exemplars
How to compare marking accuracy claims
These questions apply to any marking accuracy claim, including ours.
1.Who set the reference mark?
An exam board's published mark, one teacher, or several teachers agreeing are different standards.
2.How many answers, and how were they chosen?
Exemplars chosen to illustrate each level are clearer than a typical class.
3.Were the test answers kept separate from tuning?
A system adjusted on the same answers it is tested on will look better than it is.
4.Is the average gap reported?
A correlation measures ranking, not accuracy. It can be high while every mark is several marks out.
5.Is a correlation or a gap turned into a percentage?
A correlation of 0.89 does not mean 89% accurate.
6.Was each answer marked more than once?
One run does not show whether the same answer gets the same mark.
7.Are the marks whole marks?
A teacher awards 27, not 27.8. An average of several runs hides how much a single run varies.
8.Is the figure dated, with a version?
Results can change when the system changes.
9.Were answers typed or read from handwriting?
Reading handwriting is part of the job in a real classroom.
10.Is the question size stated?
A 2 mark gap means more on a 16 mark question than on a 40 mark one.
Method and limits
- Engine version
- Every study names the engine version it was tested on. The version is the date of the last change to the code that decides marks, so a figure always points to the exact marking it measured. When that code changes, we test again.
- Definitive mark
- The mark or level the exam board published for that answer.
- Exemplars
- Real student answers that exam boards publish with their marks to show teachers each level. They are chosen to illustrate the levels, so they include more clear cut answers and more full marks than a typical class.
- Sample sizes
- 264 answers for repeatability. For accuracy, 104 GCSE Geography, 57 GCSE History, 31 GCSE English Language and 44 A level History exemplars. Each figure states its own count.
- Production marking
- Every answer went through the same marking engine teachers use, with mark schemes read from the original documents. The engine was not changed in response to these results, and no marks were written to any teacher's account.
- Whole marks
- Every mark reported is a whole mark from a single run, the mark a teacher would see.
- Handwriting
- Handwritten exemplars were read from scans or typed up before marking. Each study page says which.
- Published material
- No student writing or mark scheme text is reproduced on these pages.
Questions teachers ask
How accurate is DeepMark's marking?
It depends on the question. On GCSE Geography, DeepMark matched the definitive mark exactly on 71 of 104 exemplars and was within 1 mark on 94. On GCSE History questions worth up to 16 marks, the average gap was 1.3 marks, with 37 of 57 within 1 mark. Each study page shows every mark.
How do you measure marking accuracy?
We mark exemplars, real student answers that exam boards publish with the mark they awarded, and compare DeepMark's mark with that definitive mark. We report the average gap, how many matched exactly and how many were within 1 mark. A ranking correlation is shown separately because it measures order, not accuracy.
Does DeepMark give the same mark every time?
Yes. In our repeatability study, 264 of 264 school answers received the same mark on all 5 runs of the same engine version, and every essay stayed in the same level.
Can DeepMark mark handwritten scripts?
Yes. Handwritten exemplars in these studies were read from scans by artificial intelligence before marking, so the accuracy figures include reading the handwriting.
Where should a teacher still check the marks?
DeepMark placed 17 of 20 top level A level History essays correctly, but mid level essays tend to be placed higher, so we recommend a teacher checks those. On GCSE English Language reading, the strongest answers tend to be marked lower.
How often are these figures updated?
Every figure names the engine version and the date it was tested. When we change how DeepMark marks, we test the same exemplars again and publish the new result beside the old one, so you can see how accuracy changes over time.