Inter-individual and inter-site neural code conversion without shared stimuli

• Neural code conversion enables transfer of decoders across subjects and sites without shared stimuli.¹
• This approach supports scalable BCI and multi-center trials by reducing calibration burden.¹
• Inter-individual and inter-site conversion supports generalizability of decoders.^{1 1}

Gardner updates

• Neural code conversion enables transfer of decoders across subjects and sites without shared stimuli, reducing calibration burden and supporting generalizability for scalable BCI and multi-center trials (Nature). ¹

Weekly enrichment (2026-07-20)

• The study was published in Nature Computational Science (2025, DOI 10.1038/s43588-025-00826-5) by the Kamitani Lab, and introduces a “content-loss-based” functional alignment that converts fMRI brain activity between people without any paired or shared stimuli.^{2 3}
• Rather than minimizing differences between paired brain patterns directly, the converter is optimized so that the deep-neural-network feature content decoded from a source subject’s converted activity matches the actual stimulus content — using hierarchical VGG19 features as the latent content representation.^{2 3}
• Analyses spanned three natural-image fMRI datasets totaling 114 subject pairs: the Deeprecon dataset (5 subjects, 6,000 training and 2,000 test samples each; 20 pairs) plus the THINGS dataset (3 subjects, 8,640 training/1,200 test) and the Natural Scenes Dataset (4 subjects, ~25,000 training/300 test), forming 94 inter-site pairs.^{2 3}
• Conversion accuracy with content loss (no shared stimuli) was statistically comparable to the traditional brain-loss method that requires shared stimuli, across whole visual cortex and subareas (Levene’s test P > 0.05 for all ROIs except V2 at P = 0.039).³
• Inter-site conversion is notable because the 94 pairs involved different subjects, different stimuli, and different MRI scanners/resolutions, yet converted activity decoded through the target’s pre-trained decoders produced image reconstructions rivaling within-individual decoding.^{2 3}
• A “non-overlapping” control split Deeprecon training data by stimulus category so source and target shared no training stimuli or categories; accuracy remained comparable to the overlapping condition, confirming the method does not secretly rely on shared content.³
• The approach generalized beyond vision: applied to the DeepSoundRecon auditory dataset it achieved robust inter-individual conversion of auditory cortical representations, though profile correlations dropped more than in the visual domain.³
• BCI relevance: by porting one person’s decoder to another (or across sites) without recollecting paired calibration data, the method reduces per-subject training burden and supports multi-center generalizable decoders; it builds on the lab’s original 2015 inter-subject neural code converter, which required shared stimuli.^{4 5}

Footnotes

1. https://news.google.com/rss/articles/CBMiX0FVX3lxTE9BcW9tZHFNOHBRaUprZmx2ZmRmM2k4NDY2WHVBSzRMR0ptdkFMVTRMS0NrQ19kdUVJODBlMVNKcTVHcUFPWlNqcjdwOHBWYW9QSDZWS3lrSjB2LTlFajVZ?oc=5 ↩ ↩² ↩³ ↩⁴ ↩⁵

2. https://www.nature.com/articles/s43588-025-00826-5 ↩ ↩² ↩³ ↩⁴

3. https://pmc.ncbi.nlm.nih.gov/articles/PMC12286860/ ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

4. https://arxiv.org/html/2403.11517v1 ↩

5. https://pubmed.ncbi.nlm.nih.gov/25842289/ ↩