Hierarchical Feature Fusion with Differential Attention Enhancement for Multimodal Sarcasm Detection

Di He

Abstract: Multimodal sarcasm detection requires effective modeling of semantic interactions between textual and visual information. Existing approaches often struggle to distinguish sarcasm-relevant cues from redundant noise during cross-modal feature learning. To address this issue, this paper proposes a multimodal sarcasm detection framework based on hierarchical feature fusion and differential attention enhancement. The proposed method incorporates differential attention into a vision-language dual-encoder architecture to suppress noisy attention patterns and emphasize salient sarcasm-related features. A bidirectional cross-modal interaction module is further introduced to capture semantic correspondences and contradictions between text and images. Finally, a hierarchical fusion strategy progressively integrates multimodal information at the feature, semantic, and decision levels. Experiments conducted on the MMSD2.0 dataset demonstrate that the proposed framework achieves 88.10% accuracy, 85.85% precision, 89.20% recall, and 87.29% F1-score, outperforming several existing state-of-the-art methods. The results confirm the effectiveness of differential attention and hierarchical fusion for multimodal sarcasm detection.

Keywords: Multimodal Sarcasm Detection, Differential Attention, Vision-Language Model, Hierarchical Feature Fusion, Cross-Modal Learning, Vision-Language Pretraining