Pré-Publication, Document De Travail Année : 2024

MixMAS: A Framework for Sampling-Based Mixer Architecture Search for Multimodal Fusion and Learning

Résumé

Choosing a suitable deep learning architecture for multimodal data fusion is a challenging task, as it requires the effective integration and processing of diverse data types, each with distinct structures and characteristics. In this paper, we introduce MixMAS, a novel framework for sampling-based mixer architecture search tailored to multimodal learning. Our approach automatically selects the optimal MLP-based architecture for a given multimodal machine learning (MML) task. Specifically, MixMAS utilizes a sampling-based micro-benchmarking strategy to explore various combinations of modality-specific encoders, fusion functions, and fusion networks, systematically identifying the architecture that best meets the task's performance metrics.
Fichier principal
Vignette du fichier
2412.18437v1.pdf (436) Télécharger le fichier

Dates et versions

hal-04946294 , version 1 (13-02-2025)

Identifiants

Citer

Abdelmadjid Chergui, Grigor Bezirganyan, Sana Sellami, Laure Berti-Équille, Sébastien Fournier. MixMAS: A Framework for Sampling-Based Mixer Architecture Search for Multimodal Fusion and Learning. 2025. ⟨hal-04946294⟩
0 Consultations
0 Téléchargements

Altmetric

Partager

More