# Accuracy-Efficiency Optimization for Multi-Stage Small Object Detection in Surveillance Video with Collaborative Frame Sampling

Year: 2024, Pages: 403-413

DOI Bookmark: [10.1109/CLUSTER59578.2024.00042](https://doi.ieeecomputersociety.org/10.1109/CLUSTER59578.2024.00042)

#### Authors

[Chunhong Du](/content/csdl/search/default?type=author&givenName=Chunhong&surname=Du/index.html), College of Intelligence and Computing, Tianjin University, China  
[Shanjiang Tang](/content/csdl/search/default?type=author&givenName=Shanjiang&surname=Tang/index.html), College of Intelligence and Computing, Tianjin University, China  
[Song Meng](/content/csdl/search/default?type=author&givenName=Song&surname=Meng/index.html), College of Intelligence and Computing, Tianjin University, China  
[Jiekai Gou](/content/csdl/search/default?type=author&givenName=Jiekai&surname=Gou/index.html), College of Intelligence and Computing, Tianjin University, China  
[Ce Yu](/content/csdl/search/default?type=author&givenName=Ce&surname=Yu/index.html), College of Intelligence and Computing, Tianjin University, China  
[Yusen Li](/content/csdl/search/default?type=author&givenName=Yusen&surname=Li/index.html), School of Computer Science and Technology, Nankai University, China  
[Hao Fu](/content/csdl/search/default?type=author&givenName=Hao&surname=Fu/index.html), National Supercomputing Center of Tianjin, China  
[Ye Tian](/content/csdl/search/default?type=author&givenName=Ye&surname=Tian/index.html), College of Intelligence and Computing, Tianjin University, China  
[Ding Yuan](/content/csdl/search/default?type=author&givenName=Ding&surname=Yuan/index.html), College of Intelligence and Computing, Tianjin University, China

#### Abstract

In video analytics, accuracy and efficiency are two important metrics and there tend to be a tradeoff between each other. In this paper, we consider accuracy-efficiency optimization for small object detection in surveillance video, which is important and has been widely used in many scenarios such as license plate detection in the traffic domain. Given that small objects tend to be attached to big objects, multi-stage object detection is supposed to be an effective approach to achieve high accuracy for small objects by detecting big objects first and then small objects within the ROIs (Regions of interests) of big objects. However, existing studies considered the accuracy-efficiency optimization for small object detection only within the single-stage scenario by changing the frame resolution or sampling rate configuration of video data, which are not suitable for multi-stage detection given that its accuracy-efficiency result is determined by the results of all stages jointly. In this paper, we propose an Adaptive and Collaborative frame Sampling approach named ACS for accuracy-efficiency optimization in the multi-stage small object detection. To improve the efficiency significantly while guaranteeing a given accuracy threshold, ACS dynamically adjusts the sampling rates of all stages collaboratively and periodically using the Karush-Kuhn-Tucker (KKT) condition based on the Lagrangian multiplier method. Additionally, we introduce a tuning knob to allow users to flexibly balance accuracy and efficiency, while ensuring a given accuracy threshold λ. Extensive experiments demonstrate the effectiveness of our approach in improving detection efficiency while guaranteeing diverse accuracy requirements.