AI-Based Building Inspection for Large Structures |
A new approach to construction progress monitoring
| Journal | Industry 4.0 Science |
| Issue | Volume 42, 2026, Edition 5, Pages 78-84 |
| Open Access | https://doi.org/10.30844/I4SE.26.5.9 |
| Bibliography | Share | Cite | Download |
Abstract
Keywords
Article
The construction of complex large structures such as buildings, production facilities, or ships requires continuous monitoring to detect deviations early and respond in a targeted manner. Construction progress monitoring (CPM) involves the systematic documentation of processes, planning and work steps during production to ensure compliance with technical specifications, quality standards, and legal requirements. This includes data collection and comparison with a defined reference entity [1]. Interpretation, evaluation, and derivation of corrective actions then takes place on this basis.
CPM encompasses several inspection dimensions:
- the geometric dimension (presence and position of components),
- the temporal dimension (deadlines and sequences), and
- the qualitative dimension (condition).
Depending on the timing of the inspection, the process may serve to verify a completed work step, prompt acceptance of a component, or document overall progress. Across all industries, CPM faces key challenges: manual data collection, subjective assessment, and heterogeneous data sources. To address these, intensive research is being conducted into automated digital data collection, analysis, and evaluation, including digital models such as building information modeling (BIM) [2].
In shipbuilding, CPM is of particular importance. Ships are manufactured as one-of-a-kind vessels in a shipyard environment where numerous trades work in parallel on a stationary structure [3]. High product complexity, combined with strict regulatory requirements (e.g., from classification societies), makes it challenging to maintain an overview of current construction status. Systematic inspections identify missing components and work steps at an early stage, thereby reducing costly rework and delays. At the same time, however, they require a significant amount of manual effort for documentation and comparison with the target model and deadlines.
The aim of this article is therefore to support data collection with machine learning models and to automate comparison with the production plan to significantly increase efficiency in CPM. The methodological approach follows the design science research model as described in [4]. In the first step, the limitations of conventional methods are analyzed, and research needs are identified. Building on this, the requirements for objective CPM are formulated. In the third step, a system architecture comprising sensor-based data collection, machine-learning-based data analysis, and automated plan comparison is developed. Prototype testing of the architecture is conducted in a shipbuilding context. Finally, the results are evaluated, and further research needs are identified.
Limitations of conventional construction progress monitoring
Conventional construction progress monitoring relies primarily on manual visual inspections with photographic documentation. Particularly at shipyards, inspections take place under varying spatial and organizational conditions. Consequently, condition assessments are not standardized but depend largely on the experiential knowledge of the specialists involved [5]. To provide an objective digital database for assessment, artificial intelligence methods are increasingly being used alongside optical measurement techniques. Techniques such as 3D laser scanning or digital image processing can be used to map construction progress [1].
At the same time, augmented reality (AR) approaches are becoming more common. They project planning information into the user’s field of view and allow for a visual comparison of target versus actual on-site conditions. AR approaches have proven effective for simple inspection tasks but reach their limits in complex construction situations, as the assessment is performed manually and therefore remains subjective [6]. The operation of 3D laser scanners also requires qualified specialists, which brings about higher costs [7]. The approach presented here stands out due to its automated, learning-based detection and algorithmic comparison with design data, which enables evaluation independent of human intervention.
The integration of 3D scanning into planning models—such as BIM—for schedule coordination is already taking place on a case-by-case basis. In one-off production, however, this process remains largely manual. Reference models often lack the necessary level of detail or flexibility [8, 9]. The integration of data from enterprise resource planning (ERP) and product lifecycle management (PLM) systems is not sufficiently guaranteed.
The large number of components—and thus inspection characteristics—also complicates the automation of the process. These include, for example, geometric characteristics such as the position, orientation, or completeness of pipes, brackets, or fittings. In particular, components whose installation location and timing vary (“soft features”) require flexibility in data capture, unlike precisely quantifiable variables.
Likewise, the dynamic environment of construction sites poses a hurdle for digital CPM documentation. Custom construction is characterized by continuous changes in the construction process. Existing methods are not sufficiently robust when applied in such real-world conditions [10]. Organizational and economic challenges, such as training needs and high investment costs, complicate implementation further. All these shortcomings clearly point to a need for more research.
Objective of objective condition assessment
The goal is to develop an efficient approach to objective and reproducible condition assessment in shipbuilding. An objective digital database is to be made available for the evaluation process. The following requirements can be derived from the research needs outlined above:
- The system must be able to simultaneously and reproducibly record and evaluate various test characteristics.
- Regular data collection must not result in any additional effort compared to existing procedures.
- The evaluation must function reliably under real-world construction site conditions and be possible without an external network connection.
- The recorded actual data must be linkable to existing design information to enable a comparison of target versus actual values over time.
- The results must be presented in a way that is traceable and interpretable.
The following section presents a system architecture that addresses these requirements.
AI-supported construction progress tracking
The system architecture is based on three core components:
- a mobile multisensor system
- a local computing unit for data analysis
- a verification logic for the algorithmic comparison of detected objects with the digital planning model
Only the interaction of these three components enables the transition from pure data collection to an evaluative CPM. A continuous data flow accompanies the process, spanning all construction phases from planning to evaluation. Figure 1 shows the methodological workflow. In the initial planning phase, building components and their relevant inspection characteristics (primarily presence and position) are first defined based on a 3D target model. In addition, inspection routes and schedules are established to ensure structured and reproducible data collection.
Measurements are taken regularly throughout the construction process. To this end, visual and geometric sensor data reflecting the current state of construction are recorded along the inspection routes. The local computing unit processes and segments the sensor data and detects building components using a residual network (ResNet) for image data [11] and the deep learning network (PointNet) for point clouds [12]. For building components with soft features, 2D and 3D information is merged to enable context-based alignment with the 3D target model. 2D information includes features from images—color, texture, contour—while 3D information consists of geometric properties from point clouds, such as shape and position.
A 3D planning model serves as a foundation. It is based on established BIM methods and links the 3D target model with the schedule. Due to sometimes limited detail, the comparison is performed at the type level within spatial contexts. In this process, component types—such as a pipe in a defined area—are considered rather than individual component instances. This goes beyond existing approaches by ensuring that it remains applicable even for models with limited detail [13]. Work packages and schedule deadlines are integrated via an ERP interface.
The key added value of this approach lies in the fact that it is only by linking actual status with the schedule—based on a static status assessment—that progress can be reliably monitored. To this end, the components identified during data collection are assigned to corresponding elements in the 3D planning model. For the target-actual comparison, the identified object types—and their quantities—are assigned to the components specified in the model. Components not identified before the target deadline are marked as overdue, thereby systematically highlighting deviations from the construction schedule. Missing components whose target deadlines have been exceeded are also visually highlighted in the digital model.
In addition, status reports are generated that compare target and actual dates. This provides a sound basis for decision-making by the construction management team regarding subsequent process steps.
Practical implementation in a shipbuilding context
A robust, mobile handheld system was developed for use at a shipyard: a compact action camera and a wireless 3D handheld laser scanner enable continuous data capture along the inspection route. Image and video data, as well as point clouds for the CPM, are captured. This data is processed on a standalone computer as part of an edge computing approach. Using appropriate machine learning models, the data is segmented and objects are detected. By comparing the data to the 3D planning model and the object IDs, a precise comparison between target and actual states can be performed (Fig. 2).
To this end, a software prototype with an object database was developed that merges the target data locally and evaluates and visualizes the actual data. Interfaces to ERP and PLM systems are implemented to integrate the required information. The central foundation is the 3D model with component structure from the PLM system. The ERP system provides data on work packages and schedule deadlines. The assignment of assembly work packages to components is based on the respective component and the associated space.
The object IDs detected during measurement are algorithmically assigned to the target model. The defining factor here is the location where the data was captured. The type-level comparison described earlier is implemented here. In this way, a reliable assessment of current status is also made possible in cases where component identification is ambiguous. Through this linkage, the installation status and the capture date for each component can be updated automatically. Based on the capture time, the actual due date for identified components is set simultaneously.
A 3D-model-based user interface visually represents the target-actual comparison using color coding. A traffic-light system evaluates schedule deviations based on the degree of completion and delay (time of capture relative to the target date). “Green” indicates installed components, “Yellow” indicates uninstalled components with an upcoming deadline, and “Red” indicates a missed deadline. This allows for a quick assessment of the construction status and provides more time for evaluation and the identification of appropriate measures between data entry cycles. Figure 2 shows the schematic process.
Increased efficiency in one-off production
The system developed here offers clear advantages for the complex and dynamic shipbuilding and plant construction industries, as it captures components with variable installation locations. Its ease of use allows for an increase in inspection frequency, enabling deviations to be detected earlier. A mobile handheld scanner also enables intuitive operation independent of specialized personnel, allowing measurements to be performed in less time and with a 50% reduction in rework. At the same time, objective evaluation ensures a more consistent CPM. The characteristics formalized during the initial planning phase are stored and thus remain permanently available.
Another advantage is the combination of 2D and 3D data. This improves the robustness of object recognition and alignment with the 3D planning model. The decentralized system architecture also offers the advantage of low dependence on external network infrastructures. Since all processing steps take place locally, the system remains operational even in environments with limited connectivity. At the same time, this ensures a high level of data security.
A current limitation of the system is its initial integration. Although employee training effort is minimal, training the model itself for inspection involves a certain amount of organizational effort. Nevertheless, significant efficiency gains can be expected.
To date, the system has only been trialed in test environments. In these tests, all components were successfully detected. To reliably demonstrate the system’s performance, quantifiable tests in an industrial setting will be required in the future.
For now, the research presented here serves as a proof of concept for linking current state and schedule via a time-contextualized assessment. The traffic-light system provides an objective basis for evaluation. Future studies will also take dependencies and cascade effects into account to enable the automatic suggestion of countermeasures. With this transition from pure status monitoring to a decision support system, further efficiency gains can be expected.
Bibliography
[1] Perkonig, C. E.: Gebäudeerkundungsmethoden zur ganzheitlichen Gebäudeerfassung, Diploma thesis, Vienna University of Technology 2025.[2] Borrmann, A.; König, M.; Koch, C.; Beetz, J. (eds): Building Information Modeling—Technology Foundations and Industry Practice. Cham 2018.
[3] Gruß, R.: Schlanke Unikatfertigung: Zweistufiges Taktphasenmodell zur Steigerung der Prozesseffizienz in der Unikatfertigung auf Basis der Lean Production, 1st edition. Beiträge zur Produktionswirtschaft. Wiesbaden 2010.
[4] Peffers, K.: The Design Science Research Process: A Model for Producing and Presenting Information Systems Research. In DESRIST International Conference on Design Science Research in Information Systems and Technology, Claremont, CA, USA, February 24–25, 2006, pp. 83–106.
[5] Sender, J.; Geist, M.; Fischer, A.; Flügge, W.: Digitales Bauzustandsmonitoring im Schiffbau. In: ZWF Zeitschrift für wirtschaftlichen Fabrikbetrieb 116 (2021).
[6] Asmar, P. G.: Contextualizing benefits and limitations reported for augmented reality in construction research, In: Journal of Information Technology in Construction (2021).
[7] Kim, J.-Y.: Measurement of Work Progress Using a 3D Laser Scanner in a Structural Framework for Sustainable Construction Management. In: Sustainability 16 (2024), 1215.
[8] Bosché, F.: Computer vision-based interior construction progress monitoring: A literature review and future research directions. In: Advanced Engineering Informatics 24 (2010), pp. 107–118.
[9] Jagusch, K., Sender, J., Jericho, D., Flügge, W.: Digital thread in shipbuilding as a prerequisite for the digital twin. In: Procedia CIRP 104 (2021), pp. 318–323.
[10] Ekanayake, B.: Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction. In: Automation in Construction 127 (2021), 103705.
[11] He, K.: Deep Residual Learning for Image Recognition (2015).
[12] Qi, C.: PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation (2016).
[13] Wandt, R.: Modellgestützte Fertigungssteuerung in der Unikatfertigung am Beispiel des Schiffbaus. Dissertation. Hamburg-Harburg University of Technology, 2014.
You might also be interested in
