EEG-EditBench: Probing Visual Information in EEG–Image Retrieval Models with Controlled Image Edits

Kaifan Zhang1, Lihuo He1,*, Yuqi Ji1, Junjie Ke2, Lukun Wu1, Tianhao You3, Xinbo Gao1
1School of Electronic Engineering, Xidian University, Xi'an, China
2School of Software, Tsinghua University, Beijing, China
3Chongqing University of Posts and Telecommunications, Chongqing, China
*Corresponding author
Paper Code Dataset EEG Data
Overview of EEG-EditBench

This work presents EEG-EditBench. Beyond 200-way retrieval, it asks a more direct question: can EEG-to-image models distinguish the viewed image from an edited version in which only one visual element has been changed?

Abstract

Recent EEG-to-image retrieval models have achieved strong performance in identifying viewed images from semantically diverse candidates. Yet such success does not reveal what visual information supports the match. A model may readily identify a cheetah among tools, plants, and vehicles, but can it still distinguish the viewed cheetah from the same scene with the cheetah replaced by a dog? Motivated by this question, we introduce EEG-EditBench, a diagnostic benchmark that examines this question through controlled edits of object identity, attributes, background, and object presence. Built from the 200 THINGS-EEG2 test images, EEG-EditBench contains 2,137 quality-controlled edits and evaluates eight representative EEG visual decoding models. Our results show that strong standard retrieval does not consistently transfer to edit-based evaluation, with fine-grained attribute changes presenting the greatest challenge. EEG-EditBench reveals model behavior hidden by aggregate retrieval accuracy and provides a controlled basis for studying what visual information EEG–image models preserve. The code and complete dataset are available through the project repository.

Controlled Comparisons on Object Identity

Controlled Comparisons across Edit Types

Dataset Construction

A vision-language model first profiles each source image, then proposes edit targets and prompts for four edit families. An image-editing model generates candidate edits, which are quality-controlled and filtered to form the final benchmark.

EEG-EditBench dataset construction pipeline

Acknowledgements

EEG-EditBench is built on the THINGS-EEG2 project. The recordings can be downloaded from the official OSF repository.

We thank the authors of the following open-source projects:

Citation

@misc{zhang2026eegeditbench,
  title         = {EEG-EditBench: Probing Visual Information in EEG--Image Retrieval Models with Controlled Image Edits},
  author        = {Kaifan Zhang and Lihuo He and Yuqi Ji and Junjie Ke and Lukun Wu and Tianhao You and Xinbo Gao},
  year          = {2026},
  eprint        = {2607.27857},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2607.27857}
}