Automatic geometric inspection of MEP elements using mixed reality with RGB-D sensing

Abstract

The construction industry is undergoing a significant transformation with the adoption of digital technologies, aimed at improving the efficiency and accuracy of processes such as geometric inspection and quality control. Among these technologies, Building Information Modelling (BIM) has emerged as a critical tool, offering semantically-rich 3D representations of construction elements. However, the integration of as-planned BIM models with real-time on-site data for progress monitoring and inspection presents substantial challenges. In particular, traditional reality capture methods like Terrestrial Laser Scanning (TLS) and photogrammetry, while accurate, are time-consuming and not conducive to real-time decision-making on construction sites.

This research addresses these limitations by developing an automated, Mixed Reality (MR)- based system for real-time geometric inspection, specifically targeting Mechanical, Electrical, and Plumbing (MEP) systems. The complexity of MEP networks, with their intricate configurations and overhead installations, makes accurate real-time inspection difficult using manual or traditional methods. The system leverages the semantic information embedded in BIM models and processes data captured in real-time through MR devices. By integrating MR glasses with image-based and 3D mesh-based processing techniques, this research facilitates automatic inspection and identification of discrepancies between as-built and as-planned structures.

This work makes several key contributions. First, it introduces the MR-MEP dataset, the first benchmark pairing MR-acquired point clouds with BIM-derived annotations for MEP components. Second, it develops and validates a hybrid inspection framework that integrates 3D mesh analysis, 2D image detection, and deep learning–based verification. The framework is explicitly designed to be scalable and adaptable, allowing the verification module to be upgraded as more advanced classification models emerge, thereby ensuring long-term applicability and continuous improvement. Finally, the framework is validated on real construction site data, demonstrating its feasibility for practical deployment and offering actionable recommendations that support the development of real-time MR-based inspection workflows in construction environments.