All research

Study

AI marking repeatability

Repeatability: 264 of 264 school answers received the same mark on all 5 runs of the same engine version.

Answers
264
Papers
76
Questions
185
Teachers
48
Marking runs
1,320
Engine version
2026.11.04
Tested
5 November 2026

By DeepMark Labs Limited. Last updated 8 November 2026.

The result

We marked 264 real school answers 5 times each with the same engine version: 1,320 marking runs in all. Every one of the 264 answers received the same mark on all 5 runs, and all 156 essays stayed in the same level.

That held across 7 subjects, short answers and long essays, and questions worth 2 to 40 marks.

264 of 264 answers received the same mark on all 5 runs
Answers given the same mark on every run, by subject
SubjectAnswer typeSame mark on all 5 runs
English LanguageEssays
50 of 50
BusinessEssays
36 of 36
BusinessShort answers
37 of 37
Religious StudiesEssays
29 of 29
Religious StudiesShort answers
37 of 37
English LiteratureEssays
20 of 20
HistoryEssays
12 of 12
HistoryShort answers
17 of 17
ScienceEssays
9 of 9
ScienceShort answers
12 of 12
OtherShort answers
5 of 5

By question size

Long essays are where a marker is most likely to drift. DeepMark gave the same mark every time on the largest questions as well as the smallest.

Question worthAnswersSame mark on all 5 runs
2 to 4 marks8484 of 84
5 to 9 marks8282 of 82
10 to 16 marks4848 of 48
20 to 40 marks5050 of 50

How we tested it

  1. 1We took 264 answers that teachers had already marked with DeepMark, from 76 papers, 185 questions and 48 teachers.
  2. 2We chose them by fixed rules before marking anything: a spread of subjects, short answers and essays, and no more than a few answers from any one question, script or paper.
  3. 3We marked each answer 5 times with the same engine version and the same answer text, writing nothing back to any teacher's account.
  4. 4We compared the 5 marks, and for essays the 5 levels, for every answer.

Why it matters

A department needs the same answer to receive the same mark whoever presses the button and whenever they press it. Without that, marks cannot be compared across classes, moderated or trusted in a data drop.

DeepMark is built for this. Marking runs with randomness switched off, and every mark is a whole mark from a single run, the mark a teacher sees, not an average of several attempts.

What this study covers

  • The same answer text, marked by the same engine version. Every figure on these pages carries a date and an engine version because marks can move when the engine is updated.
  • Consistency, not accuracy. How close the marks are to the definitive mark is measured in the accuracy studies.

Questions teachers ask

Does DeepMark give the same mark if I mark an answer twice?

Yes. In our repeatability study, 264 of 264 school answers received the same mark on all 5 runs of the same engine version, and all 156 essays stayed in the same level.

Why does repeatability matter for a department?

If two teachers mark the same answer through DeepMark, they need the same mark back. Without that, marks cannot be compared across classes or moderated.

Does this cover every subject?

The study covers 7 subjects, including English Language, English Literature, Business, Religious Studies, History and Science, on questions worth 2 to 40 marks.

Other studies