获取途径

作者

Jianqiao Zhang Hao Xiong Qiangguo Jin Tian Feng Jiquan Ma Ping Xuan Peng Cheng Zhiyuan Ning Zhiyu Ning Changyang Li Linlin Wang & Hui Cui

作者单位

¹⁴Universitat de Barcelona, Barcelona, Spain ¹⁵Helmholtz Munich, Technical University of Munich and King’s College London, Munich, Germany ⁵Department of Computer Science and Technology, Heilongjiang University, Harbin, China ²Centre for Health Informatics, Macquarie University, Sydney, Australia ⁷School of Computer Science, The University of Sydney, Sydney, Australia ⁹Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Jinan, China ¹⁰Children’s National Hospital/George Washington University, Washington, DC, USA ⁴School of Software Technology, Zhejiang University, Hangzhou, Zhejiang, China ⁸Sydney Polytechnic Institute, Sydney, Australia ¹²Technical University of Denmark, Kgs Lyngby, Denmark ⁶Department of Computer Science, Shantou University, Shantou, China ³School of Software, Northwestern Polytechnical University, Xi’an, Shaanxi, China ¹Department of Computer Science and Information Technology, La Trobe University, Melbourne, Australia ¹¹The Chinese University of Hong Kong, Hong Kong, China ¹³Imperial College London, London, UK

会议名称

Medical Image Computing and Computer Assisted Intervention – MICCAI 2024

召开年

2024

摘要

Esophageal fistula is a critical and life-threatening complication following radiotherapy treatment for esophageal cancer . Albeit tabular clinical data contains other clinically valuable information, it is inherently different from CT images and the heterogeneity among them may impede the effective fusion of multi-modal data and thus degrade the performance of deep learning methods. However, current methodologies do not explicitly address this limitation. To tackle this gap, we present an adap...更多

Esophageal fistula is a critical and life-threatening complication following radiotherapy treatment for esophageal cancer . Albeit tabular clinical data contains other clinically valuable information, it is inherently different from CT images and the heterogeneity among them may impede the effective fusion of multi-modal data and thus degrade the performance of deep learning methods. However, current methodologies do not explicitly address this limitation. To tackle this gap, we present an adaptive multi-information dual-layer cross-attention model using both CT images and tabular clinical data for early-stage EF detection before radiotherapy. Our MDC model comprises a clinical data encoder, an adaptive 3D Trans-CNN image encoder, and a dual-layer cross-attention module. The Image Encoder utilizes both CNN and transformer to extract multi-level local and global features, followed by global depth-wise convolution to remove the redundancy from these features for robust adaptive fusion. To mitigate the heterogeneity among multi-modal features and enhance fusion effectiveness, our DualCrossAtt applies the first layer of a cross-attention mechanism to perform alignment between the features of clinical data and images, generating commonly attended features to the second-layer cross-attention that models the global relationship among multi-modal features for prediction. Furthermore, we introduce a contrastive learning-enhanced hybrid loss function to further boost performance. Comparative evaluations against eight state-of-the-art multi-modality predictive models demonstrate the superiority of our method in EF prediction, with potential to assist personalized stratification and precision EC treatment planning.收起

文献类型

会议论文