Dual-Domain Aligned Deep Hierarchical Matrix Factorization Method for Micro-Video Multi-Label Classification
Fugui Fan, Yuting Su, Liqiang Nie, Peiguang Jing, Daozheng Hong, Yu Liu
Abstract
Recently, with the growing popularity of micro-videos, multi-label learning has attracted increasing attention due to its potential commercial value in different scenarios. However, existing methods place more emphasis on the alignment between explicit semantics and visual features, while neglecting the exploration of interactions at fine-grained semantic levels. To address this problem, we propose a novel dual-domain aligned deep hierarchical matrix factorization (DADHMF) method for micro-video multi-label classification. Specifically, we construct a dual-stream deep matrix factorization framework to explore implicit hierarchical semantics and corresponding intrinsic feature representations in top-down and bottom-up ways, respectively. On this basis, we leverage the intralayer alignment strategy to narrow the semantic gap between label and instance domains by introducing adaptive semantic-aware embeddings. Moreover, we further utilize the inverse covariance estimation module to automatically capture latent semantic correlations, and project the structural information into the semantic-aware embeddings to ensure the stability of the intralayer alignment. Extensive experiments on two available micro-video multi-label datasets demonstrate that our proposed method outperforms the state-of-the-art methods.