This presentation will start by reviewing the history of vision or imaging-based structural (surface condition or damage) inspection and discuss the notion of structural inspection automation. Besides robotic platforms for operation, this conceived automation process must feature 3-dimensional (3D) integrated structural element and damage (or any anomaly) detection, quantification, and mapping amid complex scenes. Further, the results of such a process should be readily fused with existing lifecycle 3D BIM or digital-twin models (DTMs) for ultimate decision-making.

The presentation plans to discuss that machine vision with optical sensors is the most viable approach as a core component to realizing structural inspection automation. Using an analogy of Tesla cars that rely entirely on vision sensors without using active Radar or LiDAR sensors, the presentation will then elaborate on six automation levels for structural inspection. However, our civil infrastructure stakeholders are much different from those from private automobile sectors, which have poured tremendous investment into collecting and creating semantically rich datasets for developing machine learning algorithms and AI-based software frameworks. On the other hand, large-scale semantically annotated datasets from civil engineering sectors are not expected to be available in the near future. Considering such constraints and aiming at the goal of realizing cost-effective structural inspection automation, the presentation will introduce recent efforts in the following three topics:

1. Low-cost 3D structural element and damage data collection and deep learning based algorithmic benchmarking

2. Human-in-the-loop based structural data collection using augmented reality headsets.

3. Visual Simultaneous localization and mapping (SLAM) enabled optimal structural element and damage mapping using virtual reality-based robotic drones.

The presentation will conclude by sharing the vision and opportunities about the future of this research area.