Immersive Human Digital Twins for Industry 4.0

Industry 4.0 has significantly improved manufacturing and industrial operations through automation, artificial intelligence and smart sensing. The virtual replication of physical entities to create so-called digital twins (DTs) [1] has become a vital component of this transformation. However, the use cases of DTs have been limited, with a focus on machines, tools and processes. 

Most industrial systems remain fundamentally reliant on humans for supervision, decision-making and manual adaptation in reaction to unforeseen events. Since existing DTs rarely capture human dynamics, cognitive states and variability, automation and simulation operate under the assumption of flawless execution by a human worker. Unfortunately, this is not always the case, as the human worker can suffer from cognitive load, stress or fatigue. To mitigate those issues, it is essential to extend the DT platform to the human. 

Toward that goal, human digital twins can map the human paradigm into the digital world to achieve human-centric automation [2-3]. This paper introduces an immersive framework for human digital twins that can be adapted across industries. We map both the physical and cognitive characteristics of human dynamics in an immersive platform capable of greater automation and simulation tasks. 

Limitations of current human modeling in industry

Despite the growing recognition of human-centric tasks in Industry 4.0, most human modeling is designed for offline analysis and post-task evaluation. As a result, it fails to capture the real-time continuous interaction between human operators, machines and the environment. With limited support for decision-making, personalization and predictive risk assessment, these models are highly inefficient [4]. The following limitations motivate the development of the integrated, adaptive and predictive representation of the human operator presented in this paper. 

Static and generic human models

Traditional ergonomic assessments rely on static anthropometric tables, rule-based posture analysis (e.g., RULA, REBA) or simplified biomechanical models. These approaches work on the basis of an imagined “average worker” and thus fail to capture interindividual variability, adaptation over time or transient physiological states.

Reactive rather than predictive safety

Most safety systems respond only after a certain threshold is crossed (e.g., excessive force, unsafe posture). Without predictive human models, early warning of fatigue, cognitive overload or injury risk remains limited.

Fragmented view of human state

Industrial monitoring often treats physical, cognitive and emotional factors separately. Wearable sensors, if used at all, are rarely integrated into a coherent model that links physiology, motion and task context. 

Immersive human digital twin 

Although debate continues regarding the minimum scope of data that each human digital twin should contain, researchers have agreed on a blueprint that defines the present and future of human digital twins. A five-stage roadmap is proposed that indicates the level of the human digital twin in operation [5]. 

Figure 1 illustrates the adoption of this roadmap in compliance with our study. The initial physical entity layer portrays the human and the data collection medium. Physiological data is gathered in this layer, which serves as the input for a level 1 human digital twin. At this stage of human digital twin, providing in-depth data analytics is possible based on past data. Since this stage focuses on finding out which indicators trigger specific outcomes, a human digital twin system of this level is called a cross-sectional human digital twin. 

With the output from the level 1 human digital twin and integrating past data, at level 2, it is possible to train a predictive model capable of forecasting future data points. Thus, it is called a deductive human digital twin. A level 3 human digital twin is then referred to as an editable human digital twin since the model capability at this stage reaches beyond simple simulation. In this phase, the human digital twin model can be run on modified data, allowing further human intervention and simulation of scenarios with unseen data. 

This study only extends to the level 3 human digital twin. A jump above this level requires accounting for human lifestyle and greater environmental variables, such as dietary habits or living conditions. Given that evolutionary data is also taken into consideration for this, the model at this stage is called an evolutionary human digital twin. 

Lastly, at level 5, the objective is to reach a white box or an explainable human digital twin. Unfortunately, the current technological infrastructure constrains the realization of an explainable human digital twin model. However, future technological advancements and more sophisticated input data are expected to enable the greater computational capacity required to reach level 5.

Multimodal data acquisition

One of the major challenges that distinguishes human digital twins from other digital twins is the integration of multimodal data originating from different sensor modalities [6-7]. Given the complexity of human dynamics, it is not possible to adequately map human-centric data with a single type of data source. Rather, fully capturing the complex relation of human data and contextual environmental factors requires a data stream from different modalities (kinematics, biometrics, etc.).

Figure 2 visualizes the different data modalities examined in this paper and the possible data fusion levels. The level of fusion depends on the integration technique [8]. This integration establishes the mechanism for synchronizing and fusing the heterogeneous data inputs into a single representation. 

The different modalities of data in a human digital twin system typically start with metadata. Metadata generally represents historical data (disease, known conditions, etc.) about the human entity and basic characteristics (age, height, weight, etc.). Depending on the twin, human digital twins can include physiological, kinematic, environmental and behavioral data. Although the number of data modalities varies and depends on the twin’s purpose, all human digital twin systems should follow a proper data integration medium to make the system interconnected. 

There are multiple computational approaches to achieving this, as explained in Figure 2, starting from a low-level fusion technique where raw data streams are merged immediately after pre-processing at an input level. At the intermediate fusion level, features from individual data modalities are extracted before fusing the data. This approach allows for the extraction of standalone features of individual modalities. Lastly, at a high level, different data from each modality are processed and trained separately. Utilizing deep learning architectures such as neural networks, transformer-based models [9-10], and ensemble learning methods has all demonstrated the capacity to capture the temporal dependency [11] and nonlinear relationship between heterogeneous data modalities. 

By combining the data into a unified framework, human digital twins can theoretically overcome the limitations of isolated sensing systems. Once multimodal data integration is achieved, it becomes possible to implement the digital counterparts that reflect the dynamic, interconnected, and multidimensional nature of human beings. For our study, all the participants have given their informed consent as required, and an IRB approval for human-centric data collection is granted with approval No. 24-01-01 by the IRB of Saarland University on January 23, 2024.

System architecture of the human digital twin

The system setup for the human digital twin involves data streams collected through multiple devices of different modalities (Fig. 3). The data collection pipeline consists of four primary categories of contact-based and contactless devices:

1. Wearable capacitive-based sensors, embedded within the Hexoskin Smart Vest for capturing physiological signals of the human body, including Electrocardiogram (ECG) and respiratory dynamics.
2. A pair of StretchSense gloves [12] to collect fine-grained biomechanical movements of human hands, including flexion, angle, and rotation of the fingers.
3. ANR MuscleSense EMG sensors for capturing musculoskeletal dynamics with muscle onset, offset, and intensity of activation.
4. A ZED 2i stereo camera as a vision-based MoCAP system for capturing full-body kinematics and environmental interactions.

These different categories of data are preprocessed and then integrated into a common platform for human digital twins. The framework developed for the human digital twin model follows a layered architecture consisting of multiple machine learning and deep learning algorithms. Our framework design architecture integrates multiple algorithms into a unified structure for predictive modeling, simulation, and real-time interaction. 

The paper follows a modular approach, in which different algorithms within the framework address different aspects of the system: an ensemble learning method [13] for human biometric modeling, PINN for motion simulation [14-15], and a BiLSTM network [16-17] for fine motor control and cognitive state detection. The algorithms for the integration of the cognitive and physical state of humans are explained in previously published papers [13, 18]. 

Validation of the human digital twin

The human digital twin framework is assessed both at the individual algorithm level using multiple evaluation metrics for different sensor data modalities, and as a complete system as well, through qualitative evaluation with real users in an application-oriented study, as shown in our previous work [18]. A standard metric of Root Mean Square Error (RMSE) [19] is employed as a quantitative evaluation metric for all continuous prediction or regression tasks across all sensor modalities and algorithms. In addition to RMSE, each algorithm is also validated with specific validation metrics tailored to the respective sensor modality. 

Figure 4 visualizes the individual RMSE score for different tasks containing data from different sensor modalities. However, the framework consists of tasks that cannot be accurately evaluated through RMSE and require additional metrics for better understanding. For biometric data evaluation, mean average error, mean square error, normalized mean square error, and scattered index (SI) [20] are implemented to better assess the model’s performance while dealing with time series data as presented in our previous study [13]. 

The human digital twin can be adopted for numerous application scenarios with use cases in both industry and the clinical domain. One use case is for cognitive state monitoring for human workers during industrial tasks to assist decision-making, which is extensively reported in our previous study [18]. The study highlights how the use of human digital twins can potentially enable informed decision-making, reduce cognitive overload and improve overall ergonomics for human workers. Despite its potential, the human digital twin framework is highly dependent on the integration of sensor data. The model’s performance can vary significantly depending on the modality used for data collection and the manner of data fusion. 

Human digital twins as a paradigm shift

This study proposes a unified human digital twin framework solution that pushes current technological boundaries in the domain of human digital twins by integrating cutting-edge actuation, sensing, simulation, and bidirectional feedback. The approach includes integrating multimodal sensing, AI, and biomechanical simulation into one compact system. The system then provides real-time cognitive state monitoring and physical movement replication to address challenges of worker safety, task optimization, and human-machine interaction.

Thus, the human digital twin represents a paradigm shift from machine-centered to human-centered industrial digitalization. By capturing the dynamic, individual, and predictive aspects of human workers, human digital twins enable safer, more productive, and more sustainable industrial systems.

You might also be interested in

Developing Virtual Reality in Learning Contexts Navigating efficiency, content relevance and scalability Stella Kanatouri , Oliver Sosna , Alexander Kulik, Sina C. Truckenbrodt , Friederike Klan , Christian Erfurth

While virtual reality can facilitate hands-on learning, its development faces barriers, including high costs and time demands and scalability challenges. This article presents two case studies that illustrate strategies for overcoming such barriers when training the next generation of skilled workers in environmental technologies. By examining approaches for streamlining development and increasing content relevance and scalability, we highlight lessons learned for future practice. We conclude by envisioning a future in which educational institutions can flexibly and cost-effectively prototype virtual reality in learning contexts, ensuring alignment with curricular goals and learners’ needs.

Industry 4.0 Science | Volume 42 | Edition 3 | Pages 26-34 | DOI 10.30844/I4SE.26.3.3

Serious Games as a Training Tool Game mechanics design to promote resilience Annika Lange , Thomas Knothe

Unforeseen events are increasingly challenging manufacturing companies. Being resilient during crises is becoming a key competence. Serious games (SG) can help make resilience-building processes more transparent. This article derives specific requirements for SG from different phases of resilience and shows how these can be implemented in game mechanics in order to effectively support the training of resilience.

Industry 4.0 Science | Volume 42 | 2026 | Edition 2 | Pages 98-104

Experiencing Digital Twins in Production and Logistics The fischertechnik® Learning Factory 4.0 as a development platform for possible expansion stages Deike Gliem , Sigrid Wenzel , Jan Schickram, Tareq Albeesh

The fischertechnik® Learning Factory 4.0 has proven to be a suitable experimental environment for testing digital twins. Depending on the targeted maturity stage, the functions of a digital twin range from status monitoring and forecasting to the operational control of production and logistics systems. To systematically classify these functions, this article presents a maturity model that serves as a framework for the development of a digital twin. Building on this, selected use cases are implemented in a test and development environment based on a system architecture with multi-layered logic structure. These initial implementations serve to highlight application purposes, relevant methods, and typical challenges and potentials in the transfer to real factory environments.

Industry 4.0 Science | Volume 42 | Edition 2 | Pages 30-37 | DOI 10.30844/I4SE.26.2.30

From Brownfield to Industry 4.0 Learning factories as training and testing environment for digital transformation Jakob Weber, Sven Völker

To succeed in their digital transformation, manufacturing companies need engineers with in-depth knowledge of key technologies and concepts, and a profound understanding of the transition from Industry 3.0 to Industry 4.0. This article describes the concept of a learning factory that is continuously subjected to a digital transformation, thereby creating an environment for the development of transformation competencies. The concept of digital transformation is based on digital worker assistance systems and multi-agent systems for production control. These enable the incremental integration of existing resources into the digitalized factory. The learning factory is not presented to students as a completed solution. Instead, it is continuously developed further as part of student projects. This way, it contributes directly to the qualification of personnel for the implementation of Industry 4.0.

Industry 4.0 Science | Volume 42 | 2026 | Edition 2 | Pages 88-96

AI Colleagues? Competence requirements and training for AI use in industry Swetlana Franken

Artificial intelligence is fundamentally changing tasks, roles, and skills in (industrial) companies. Increasingly, it acts as a colleague, preparing decisions, supporting processes, and interacting with people. This article highlights key competence requirements for AI use in industry, presents an integrated competence model, and outlines practical strategies for the transfer of skills. The aim is to prepare companies and employees for humane, competence-oriented AI implementation that combines technological efficiency with human creativity and judgment.

Industry 4.0 Science | Volume 42 | 2026 | Edition 2 | Pages 78-86