Overview
HieraticBench is a benchmark for measuring whether AI models can recognize and read hieratic, the cursive form of Egyptian hieroglyphs. Its central test uses a sentence handwritten by an Oxford Egyptologist whose translation has not been published, reducing the possibility that a model can answer through memorization.
The project focuses on genuine reading ability rather than recognition of familiar museum imagery. Its reported experiments show a sharp distinction between identifying full ancient documents and understanding newly written signs.
Benchmark Structure
The evaluation is organized as four progressively harder tasks:
- Identify the writing system from an image without prior hints.
- Map individual hieratic signs to standard Gardiner sign-list codes.
- Transliterate the writing using Egyptological notation.
- Translate the text into English.
After the identification stage, later tasks disclose that the writing is hieratic. This allows sign-reading ability to be assessed even when a model cannot initially name the script. Identification and sign tasks receive automated scores, while the sealed sentence is not scored for transliteration or translation because its answer is not stored.
Dataset and Results
The dataset contains 268 items: 87 real hieratic documents, 29 controls in other Egyptian scripts, 150 individual signs, and two sealed sentences. Public images are openly licensed and credited, while the sealed samples are available for model evaluation.
Across the reported comparison of 13 models, none identified the newly written sentence in 78 attempts. The strongest models recognized hieratic in real documents 95% of the time, yet the best reported single-sign score was 13%, highlighting the difference between recognizing a document's visual context and reading its component signs.
Research Purpose
Hieratic was used for more than three millennia on papyrus, pottery fragments, and wooden boards for letters, records, medical texts, mathematics, literature, and other everyday writing. Because only a small group of specialists can read it fluently and scribal forms vary across periods and writers, the benchmark targets a low-resource recognition problem where useful progress requires generalizing beyond page-level appearance.