Mots-Clés
Multiple instance learning
machine learning
benchmark
multi-modal model
Multi-modal model Oncology
deep learning
Pathomics
Transcriptomics
Image Analysis
Description
Project description:
Nowadays, there is an emergency in oncology: accurate prognostic and predictive models are urgently needed to guide patient treatment. This project aims to systematically benchmark multimodal integration strategies using histology, radiomics and genomics to disentangle real predictive gains from potential overfitting effects, and to provide guidelines for when and how multimodal approaches should be preferred over unimodal models.
Context:
High-grade ovarian cancer is an aggressive disease, characterized by significant cellular heterogeneity, rapid progression, and highly variable response to treatment. Its management generally relies on cytoreductive surgery combined with platinum-based (carboplatin) and taxane-based (paclitaxel) chemotherapy, sometimes complemented by targeted therapies such as PARP inhibitors in patients with BRCA mutations or homologous recombination deficiency (HRD). Given the massive influx of data in the field, there is a need to aggregate these datasets in order to analyze them more effectively.
Prognostic models in oncology are most often developed from a single data source, typically histopathology or genomics. While these unimodal approaches have provided important insights, they fail to address the potential benefits of integrating complementary information from multiple modalities. Recent studies have demonstrated that radiological data, such as computed tomography, contain valuable prognostic features that can be exploited using machine learning. However, the capacity of multimodal integration—combining histological, genomic, and radiological features—to truly improve prediction of treatment response remains poorly understood.
A major challenge in this setting is that multimodal machine learning models are more prone to overfitting, since available multimodal datasets are usually small while the models themselves involve a larger number of parameters. Consequently, reported improvements in prediction performance may partly reflect artefacts rather than genuine biological or clinical signals.
Objective:
We propose to investigate several key questions: Do models integrating medical images, histopathological slides, and routinely collected clinical data outperform models trained on clinical data alone? Are multimodal models prone to overfitting on small datasets? Can transcriptomic data provide added value, and which modality contributes most to predictive performance? The goal is to provide practical guidelines for the use of multimodal machine learning models in oncology, specifically applied to ovarian cancer, and to benchmark available models.
Methodology:
The project will compare classical statistical models, such as logistic regression with LASSO regularization on multivariate features, with more complex machine learning approaches. In particular, foundation models will be used to extract high-level features from each data modality. Multiple feature extraction strategies will be evaluated to assess their impact and stability on model performance. Multimodal integration strategies, including Multi-Head Attention (MHA) architectures and ensemble-based multimodal models, will be compared with unimodal models to determine the added value of combining data types.
We will explore both early fusion approaches, where features from different modalities are combined before modeling, and late fusion approaches, where predictions from each modality are combined at the decision level. This will allow us to systematically compare the performance and robustness of different integration strategies.
To further investigate model robustness and potential overfitting, experiments will be performed by progressively increasing the dataset size through the introduction of random or simulated variables. This will allow observation of how prediction performance evolves and help distinguish genuine predictive signal from artefacts caused by model complexity or limited data.
We have a cohort of 80 whole-slide images (20X – 0.25 µm²/pixel) from HGSOC patients classified as short-term and long-term responders. In addition, we have 30 external patients (NCT-06084195) with spatial transcriptomics data. Other groups have made their datasets available upon reasonable request, which can be used to further extend and validate our analyses.
Clinical impact and expected outcome:
This approach will provide a better understanding of multimodal models and their applicability in a clinical context. By systematically benchmarking integration strategies, it will help determine whether the reported improvements in prediction performance reflect genuine biological signal or methodological artefacts. In oncology, and particularly in high-grade serous ovarian cancer (HGSOC), this work is expected to yield several outcomes: the establishment of a reproducible benchmarking framework for multimodal data integration, the identification of robust biomarkers and predictive features associated with treatment response and survival, and the development of practical guidelines for the use of multimodal machine learning models in clinical research.
Together, these advances will contribute to improving prognostic and predictive tools, ultimately supporting more personalized treatment strategies for patients.
Program of the student
• Conduct a comprehensive review of existing literature on machine learning and deep learning models applied to medical data for survival analysis and patient outcome prediction – Attention Based Deep Learning models
• Benchmark existing tools for multimodal integration.
• Design and implement a framework of simulated datasets to serve as a reference for the robust evaluation and comparison of such models.
Skills:
Coming from an engineering school or holding a master’s degree in computer science, data science or biostatistics, you are looking for a research internship and have the following skills:
• Proficiency in Python with ML and Deep learning libraries such as Sklearn and Pytorch
• Ability to write clean, maintainable code in a Git project
• Comfort with data and statistical analysis, Machine and Deep learning fundamentals, linear algebra, image processing, etc
Contact:
Applications (CV and motivation letter) should be sent to jean-philippe.villemin@inserm.fr. Contact details for referees and master’s transcripts with grades are appreciated.
The internship is a 6-month full-time position starting in February 2026 at the Institut de Recherche en Cancérologie de Montpellier (IRCM – Inserm U1194), located on the Val d’Aurelle Hospital Campus of the Montpellier Cancer Institute (ICM).
References:
1. Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer.Boehm & al, Nature Cancer (2022)
2. Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer. Vanguri & al, Nat Cancer (2022)
3. Pan Cancer integrative histology-genomic analysis via multimodal deep learning. Mamood & al, Cancer Cell (2022)
4. Integration of clinical, pathological, radiological, and transcriptomic data improves prediction for first-line immunotherapy outcome in metastatic non-small cell lung cancer Captier & al, Nat Comm (2025)
5. Foundation model-driven multimodal prognostic prediction in patients undergoing primary surgery for high-grade serous ovarian cancer. Jiwey & al, NPJ Oncology (2025)
6. A multi-modal model integrating MRI habitat and clinicopathology to predict platinum sensitivity in patients with high-grade serous ovarian cancer: a diagnostic study. Jiwey & al International Journal of Surgery (2025)
7. Harnessing multimodal data integration to advance precision oncology. Boehm & al, Nature Reviews Cancer (2021)
8. Histology and spatial transcriptomic integration revealed infiltration zone with specific cell composition as a prognostic hotspot in glioblastoma Fidon & al, BioRxiv (2025)
9. Multiple instance learning with spatial transcriptomics for interpretable patient-level predictions: application in glioblastoma. Grouard & al, BioRxiv (2025)