Predicting respiration rate in unrestrained dairy cows using image analysis and fast Fourier transform
R.R. Mantovani, Guilherme Lobato Menezes, J.R.R. Dórea
Abstract
The respiratory rate (RR) is commonly employed for identifying animals experiencing heat-stress conditions and respiratory diseases. Recent advancements in computer vision algorithms have enabled the estimation of the RR in dairy cows through image-based approaches, with a primary focus on standing positions, thermal imaging, and deep learning techniques. In this study, our objective was to develop a system capable of accurately predicting the RR of lying Holstein cows under unrestrained conditions using RGB and infrared night vision (IR) images. Thirty lactating cows were continuously recorded for 12 h per day over a 3-d period, capturing at least one 30-s video segment of each cow during lying time. A total of 95 videos were manually annotated with rectangular bounding boxes encompassing the flank area (region of interest; ROI) of the lying cows. For future applications, we trained a model for ROI identification using YOLOv8 to avoid manual annotations. The observed RR was determined by visual counting of breaths in each video. To predict the RR, we devised an image processing pipeline involving: (1) capturing the ROI for the entire video, (2) reshaping the pixel intensity of each image channel into a 2D object and calculating its per-frame mean, (3) applying Fast Fourier Transform (FFT) to the average pixel intensity vector, (4) filtering frequencies specifically associated with respiratory movements, and (5) executing inverse FFT on the denoised data and identifying peaks on the resulting plot, with the count of peaks serving as the predicted RR per minute. The evaluation metrics, Root Mean Squared Error of Prediction (RMSEP), and R2 yielded values of 8.3 breaths/min (17.1% of the mean RR) and 0.77, respectively. To further validate the method, an additional data set comprising pre-weaned dairy calves was utilized, consisting of 42 observations from 25 calves. The RMSEP and R2 values for this data set were 13.0 breaths/min and 0.73, respectively. The model trained to identify the ROI exhibited a precision of 100%, a recall of 71.8%, and an F1 Score of 83.6% for bounding box detection. These are promising results for the implementation of this pipeline in future studies. The application of FFT to signals acquired from both RGB and IR images proved to be an effective and accurate method for computing the RR of cows in unrestrained conditions.