Deep LLM layers expose passage-level context in EEG during silent reading

Key points

• Here we ask how EEG-decodable structure is distributed over the full depth of the language model.1
• We trained one decoder per layer of Llama-3.1-8B, with and without a causal transformer over the sequence of EEG windows.1
• We distinguish word-level decodability from a context-tracking component.1
• The results locate word identity in the early layers of the language model, discourse state in the deeper layers, and make the choice of target layer a design dial for EEG language decoders.1

Sources (1)

• [1]Deep LLM layers expose passage-level context in EEG during silent reading
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 03:41 PM

  In a previous report we showed that a decoder can recover open-vocabulary word-level information from 19-channel EEG recorded during silent reading, by aligning short EEG windows with hidden-state embeddings of the presented word taken from a large language model (LLM).

  Here we ask how EEG-decodable structure is distributed over the full depth of the language model.

Extractive summary: sentences quoted from the sources.