Evaluation and Comparison

Evaluation metrics

The evaluation complies with the classic standard of one-pass evaluation (OPE) [1], including success rate and precision. Success rate reflects intersection over union (IoU) score, and area under the curve (AUC) of success plot is used to rank the trackers in the experiment. As for precision, it is concerned with the center location error (CLE) between the estimated bounding box and ground truth. Since application of UAM tracking methods confronts severe scale variation, IoU is more important to demonstrate tracking robustness and more accounted in comparison.

[1] Y. Wu, J. Lim, and M.-H. Yang, “Online Object Tracking: A Benchmark,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 2411–2418.

Results

1. Overall performance

Overall performance evaluation on UAMT100 benchmark.

2. Attribute-based performance

AttributeARCBCLIOBOVPOCSVSOBUAM-AVCWD
DaSiamRPN [2]0.3620.3760.3600.4140.3510.4830.4070.3870.4790.3940.449
SiamFC++ [3]0.4700.4510.4440.5180.4470.6290.5010.4840.6270.5020.445
SiamRPN++ [4]0.4300.4630.4410.4840.4420.5650.4810.4700.6080.4660.483
SiamAPN [5]0.4540.4210.3970.5060.4140.5970.4760.4670.6630.4550.417
SiamSA (ours)0.5270.5080.4860.5500.4720.6150.5330.5240.7090.5190.500

Attributed-based performance on UAMT100. Based on AUC score, the best performance is denoted by bold font.

[2] Z. Zhu, Q. Wang, B. Li, W. Wu, J. Yan, and W. Hu, “Distractor-Aware Siamese Networks for Visual Object Tracking,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 101–117.

[3] Y. Xu, Z. Wang, Z. Li, Y. Yuan, and G. Yu, “SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), vol. 34, no. 07, 2020, pp. 12 549–12 556.

[4] B. Li, W. Wu, Q. Wang, F. Zhang, J. Xing, and J. Yan, “SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 4282–4291.

[5] C. Fu, Z. Cao, Y. Li, J. Ye, and C. Feng, “Siamese Anchor Proposal Network for High-Speed Aerial Tracking,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 510–516.