Bidirectional Mapping with Contrastive Learning on Multimodal Neuroimaging Data

Authors

Kai Ye
Haoteng Tang, The University of Texas Rio Grande Valley
Siyuan Dai
Lei Guo
Johnny Yuehan Liu
Yalin Wang
Alex Leow
Paul M. Thompson
Heng Huang
Liang Zhan

Document Type

Conference Proceeding

Publication Date

10-2023

Abstract

The modeling of the interaction between brain structure and function using deep learning techniques has yielded remarkable success in identifying potential biomarkers for different clinical phenotypes and brain diseases. However, most existing studies focus on one-way mapping, either projecting brain function to brain structure or inversely. This type of unidirectional mapping approach is limited by the fact that it treats the mapping as a one-way task and neglects the intrinsic unity between these two modalities. Moreover, when dealing with the same biological brain, mapping from structure to function and from function to structure yields dissimilar outcomes, highlighting the likelihood of bias in one-way mapping. To address this issue, we propose a novel bidirectional mapping model, named Bidirectional Mapping with Contrastive Learning (BMCL), to reduce the bias between these two unidirectional mappings via ROI-level contrastive learning. We evaluate our framework on clinical phenotype and neurodegenerative disease predictions using two publicly available datasets (HCP and OASIS). Our results demonstrate the superiority of BMCL compared to several state-of-the-art methods.

Comments

Reprints and permissions

https://rdcu.be/dxGmx

Recommended Citation

Ye, K. et al. (2023). Bidirectional Mapping with Contrastive Learning on Multimodal Neuroimaging Data. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14222. Springer, Cham. https://doi.org/10.1007/978-3-031-43898-1_14

Publication Title

Lecture Notes in Computer Science

DOI

10.1007/978-3-031-43898-1_14