Detecting Hazards at Rail Grade Crossings Using Computer Vision and AI

Authors

• Silvia Flores-Osuna, The University of Texas Rio Grande Valley
• Gasser G. Ali, The University of Texas Rio Grande Valley
• Darren Espinoza, The University of Texas Rio Grande Valley
• Felix Chavez, The University of Texas Rio Grande Valley
• Constantine Tarawneh, The University of Texas Rio Grande ValleyFollow

Document Type

Conference Proceeding

Publication Date

8-27-2026

Abstract

The Federal Railroad Administration reported a total of 2,250 highway-rail incidents during 2024. Continuous efforts by transportation agencies aim to lower these numbers by spreading awareness, improving infrastructure, and installing warning devices. Efforts have been made to enhance safety at these crossings using a variety of innovative technologies; however, there remains a need for a robust and cost-effective system capable of detecting hazards at various crossings under diverse climate and lighting conditions. This paper explores the application of artificial intelligence and deep learning to develop a Convolutional Neural Network (CNN) for image-level multi-class classification of hazards at grade crossings, including stalled vehicles, pedestrians, and animals. An extensive dataset of grade crossing images was collected from publicly available sources and manually labeled. A MobileNetV3-based CNN model was trained using transfer learning across 13 hazard classes. Model performance was evaluated on unseen data by assessing balanced accuracy, F1 score, and Area Under Curve (AUC). Results indicate that the proposed model achieves balanced accuracy exceeding 90% for most hazard classes, with F1 scores greater than 0.8 and AUC values above 0.95, demonstrating reliable hazard classification performance. Overall, the findings support the potential of AI-driven computer vision systems to enhance safety at grade crossings.

Comments

© 2026 American Society of Civil Engineers.

Recommended Citation

Flores-Osuna, Silvia, Gasser G. Ali, Darren Espinoza, Felix Chavez, and Constantine Tarawneh. "Detecting Hazards at Rail Grade Crossings Using Computer Vision and AI." In Construction Research Congress 2026, pp. 137-146. https://doi.org/10.1061/9780784486962.014

First Page

461

Last Page

495

Publication Title

Construction Research Congress 2026 Advanced Technologies Artificial Intelligence and Data Analytics in Construction Selected Papers from Construction Research Congress 2026

DOI

https://doi.org/10.1061/9780784486962.014