Crowd Counting by Using Top-k Relations: A Mixed Ground-Truth CNN Framework
Dong Li, Haijun Zhang, Kai Yang, Dongliang Zhou, Jianyang Shi, Jianghong Ma
Abstract
Crowd counting has important applications in the environments of smart cities, such as intelligent surveillance. In this paper, we propose a novel convolutional neural network (CNN) framework for crowd counting with mixed ground-truth, called top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relation-based network (TKRNet). Specifically, the estimated density maps generated in a coarse-to-fine manner are treated as coarse locations for crowds so as to assist our TKRNet to regress the scattered point-annotated ground truth. Moreover, an adaptive top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relation module (ATRM) is proposed to enhance feature representations by leveraging the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> dependencies between the pixels with an adaptive filtering mechanism. Specifically, we first compute the similarity between two pixels so as to select the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relations for each position. Then, a weight normalization operation with an adaptive filtering mechanism is proposed to make the ATRM adaptively eliminate the influence from the low correlation positions in the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relations. Finally, a weight attention mechanism is introduced to make the ATRM pay more attention to the positions with high weights in the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relations. Extensive experimental results demonstrate the effectiveness of our proposed TKRNet on several public datasets in comparison to state-of-the-art methods.