GCSE History AI marking, up to 16 marks

Across 57 GCSE History exemplars, DeepMark’s marks were an average of 1.3 marks from the definitive mark. 37 were within 1 mark, 45 were within 2 marks, and 19 matched the definitive mark exactly. Every answer received the same mark on all 3 runs.

1. 1We collected real GCSE History answers that the exam board publishes with the mark it awarded. These are exemplars, and the published mark is the definitive mark.
2. 2We loaded each question paper and its mark scheme into DeepMark from the original documents, through the same path a teacher uses.
3. 3Handwritten answers were read from the scans by artificial intelligence before marking.
4. 4Each answer was marked 3 times by the same marking engine teachers use. Nothing was tuned on these answers.
5. 5We compared DeepMark's whole mark with the definitive mark for every answer.

Short questions are the closest: on 4 mark questions DeepMark matched the definitive mark exactly on 6 of 8. On 8 mark questions, 7 of 8 were within 1 mark.

29 of 41 essays were within 2 marks of the definitive mark, with an average gap of 1.5 marks. Ranking correlation: 0.89. This measures how closely DeepMark puts the essays in the same order as the definitive marks; the average gap is the accuracy figure.

Where marks differ, DeepMark tends to be slightly cautious: marks sit 0.5 marks below the definitive mark on average, so a teacher reviewing them is more likely to move a mark up than down.

Every exemplar in the study, with the definitive mark and DeepMark’s mark. We publish marks only, never student writing.

Average gap

How far DeepMark's mark was from the definitive mark, on average, ignoring direction. A gap of 1.3 marks means a typical answer was a little over 1 mark away.

Matched exactly

DeepMark gave the same whole mark as the definitive mark. This is the strictest test, and only possible because every DeepMark mark is a whole mark.

Within 1 mark

DeepMark's mark was the definitive mark or 1 mark either side.

Ranking correlation

How closely DeepMark puts answers in the same order as the definitive mark, from 0 to 1. It measures order, not accuracy, so we always show it beside the average gap, never instead of it.

• AI marking repeatability264 answers, 76 papers, 185 questions, 48 teachers, 1,320 runs
• GCSE Geography AI marking, 1 to 9 marks104 exemplars on questions worth 1 to 9 marks
• GCSE English Language reading AI marking, 15 marks31 exemplars on 15 mark reading questions
• GCSE Business AI marking, whole mock papers23 handwritten mock scripts on a 43 mark paper
• A level History AI marking, 25 marks44 exemplars