Mapping and Decoding Language Representations from Human Cortex

Is it possible to read the content of human thought out using recordings of brain activity? We use non-invasive functional MRI and machine learning methods based on large language models to investigate the relationship between brain activity and the content of thought. These methods reveal complex spatial and temporal patterns of activity that relate to specific semantic categories. We show that this information can be read out as language, even when the stimulus evoking it is from another modality. We also use self-supervised speech models to show that fine temporal information can be deduced even from slow non-invasive recordings. These results point to a future of neuroscience that strongly integrates modern neural network models.