Detecting phone-related pedestrian distracted behaviours via a two-branch convolutional neural network

Authors

Humberto Saenz, The University of Texas Rio Grande Valley
Huiming Sun
Lingtao Wu
Xuesong Zhou
Hongkai Yu

Document Type

Article

Publication Date

2020

Abstract

The distracted phone-use behaviours among pedestrians, like Texting, Game Playing and Phone Calls, have caused increasing fatalities and injuries. However, the research of phonerelated distracted behaviour by pedestrians has not been systemically studied. It is desired to improve both the driving and pedestrian safety by automatically discovering the phonerelated pedestrian distracted behaviours. Herein, a new computer vision-based method is proposed to detect the phone-related pedestrian distracted behaviours from a view of intelligent and autonomous driving. Specifically, the first end-to-end deep learning based Two-Branch Convolutional Neural Network (CNN) is designed for this task. Taking one synchronised image pair by two front on-car GoPro cameras as the inputs, the proposed two-branch CNN will extract features for each camera, fuse the extracted features and perform a robust classification. This method can also be easily extended to video-based classification by confidence accumulation and voting. A new benchmark dataset of 448 synchronised video pairs of 53,760 images collected on a vehicle is proposed for this research. The experimental results show that using two synchronised cameras obtained better performance than using one single camera. Finally, the proposed method achieved an overall best classification accuracy of 84.3% on the new benchmark when compared to other methods.

Comments

© 2020 The Authors. IET Intelligent Transport Systems published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology

Recommended Citation

Saenz, H, Sun, H, Wu, L, Zhou, X, Yu, H. Detecting phone‐related pedestrian distracted behaviours via a two‐branch convolutional neural network. IET Intell Transp Sy. 2020; 1– 12. https://doi.org/10.1049/itr2.12012

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.

Publication Title

IET Intelligent Transport Systems

DOI

10.1049/itr2.12012