Setting BirdNET confidence thresholds: species-specific vs. universal approaches

Abstract

BirdNET is widely used in avian acoustic research, providing species predictions alongside confidence values that represent the algorithm’s certainty in species identification. Setting thresholds for these confidence values can increase precision (i.e., the percentage of true positives out of all the identified predictions) but may decrease the predictions retained and exclude true positives that fall below the threshold. This study evaluates two methods for setting confidence thresholds using a two-year audio dataset from western Canada, focusing on 19 target species: (1) a universal threshold of 0.7 across all species and (2) species-specific thresholds defined as the minimum confidence required to achieve a precision of at least 0.9. The universal threshold yielded precision ranging from 0.7 to 1.0 across species but retained only 17 ± 14% (SE) of BirdNET predictions. In contrast, species-specific thresholds ensured precision above 0.9 while retaining 70 ± 37% (SE) of predictions. Species-specific thresholds varied across species but were generally lower than 0.35. Confidence values associated with BirdNET predictions were found to be species specific, but no clear link was observed between BirdNET’s performance and species' song/call complexity, defined as song duration, bandwidth, and number of inflections. Our results confirm that species-specific thresholds offer higher precision and retain more predictions compared to a universal threshold. We provide a step-by-step workflow, including R code, to help researchers define species-specific thresholds that ensure reliable interpretation of BirdNET outputs. Additionally, we discuss how our workflow aligns with and complements previously proposed approaches for setting BirdNET thresholds.

Topics & Concepts