Fine-Tune the Pretrained ATST Model for Sound Event Detection
Nian Shao, Xian Li, Xiaofei Li
Abstract
Sound event detection (SED) often suffers from the data deficiency problem. Recent SED systems leverage the large pretrained self-supervised learning (SelfSL) models to mitigate such restriction, where the pretrained models help to produce more discriminative features for SED. However, the pretrained models are regarded as a frozen feature extractor in most systems, and fine-tuning of the pretrained models has been rarely studied. In this work, we study the fine-tuning method of the pretrained models for SED. We introduce frame-level audio teacher-student transformer model (ATST-Frame), our newly proposed SelfSL model, to the SED system. ATST-Frame was especially designed for learning frame-level representations of audio signals and obtained state-of-the-art (SOTA) performances on a series of downstream tasks. We then propose a fine-tuning method for ATST-Frame using both (in-domain) unlabelled and labelled SED data. Our experiments show that, the proposed method overcomes the overfitting problem when fine-tuning the large pre-trained network, and our SED system obtains new SOTA results of 0.587/0.812 PSDS1/PSDS2 on the DCASE challenge task 4 dataset.