Semantic Alignment of EEG and Text: A Contrastive Learning Framework for Decoding Imagined Speech

Key points

• We adapt a multimodal contrastive learning framework to imagined speech, aligning EEG trials with language representations in the embedding space of a frozen pretrained text encoder.1
• An EEG encoder is trained with a composite objective combining semantically reweighted EEG-sentence alignment and text-prototype-based class discrimination, encouraging semantic structure while preserving separability among 39 intent categories.1
• We also investigate maximum EEG-text cosine similarity (MaxCosine) as an intrinsic indicator of decoding reliability and probe semantic structure beyond category labels using five evaluation-time semantic axes and a qualitative Semantic Gallery Probe (SGP) with previously unseen candidate sentences.1
• The contrastive framework achieved decoding performance comparable to a task-optimized closed-set classifier while mapping EEG into a frozen language embedding space.1

Sources (1)

• [1]Semantic Alignment of EEG and Text: A Contrastive Learning Framework for Decoding Imagined Speech
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 06:14 PM

  Imagined speech decoding from EEG offers a promising route toward naturalistic brain-computer interfaces.

  We adapt a multimodal contrastive learning framework to imagined speech, aligning EEG trials with language representations in the embedding space of a frozen pretrained text encoder.

Extractive summary: sentences quoted from the sources.