MOAB: MULTI-MODAL OUTER ARITHMETIC BLOCK FOR FUSION OF HISTOPATHOLOGICAL IMAGES AND GENETIC DATA FOR BRAIN TUMOR GRADING
Published in 20th International Symposium on Biomedical Imaging (ISBI-2023), 2023
Our Contributions
- We introduce a novel Multi-modal Outer Arithmetic Block (MOAB) that fuses latent representations of different modalities using arithmetic operations.
- We demonstrate MOAB’s ability to capture rich representations of fused data in a brain cancer application using a histology image and genetic data. Our method outperforms the previous state-of-art by producing more discriminative fused features for classification. This is also demonstrated with t-SNE plots, which visualize the improved separability of classes thanks to MOAB.