In this article, the concept of a digital assistance system based on an empathic digital twin of the human as well as Artificial Intelligence for data analytics is introduced. The field of application is manual assembly, chosen because it displays a high potential for the use of technical empathy to support people and because a high proportion of the manufacturing costs of a product are incurred at this stage in the process.<\/p>\n\n\n\n
Manual assembly is often still necessary when manufacturing complex, possibly customized goods in small quantities, a primary domain of German industrial companies. By means of an assistance system, quality, cost-effectiveness and flexibility of manual work could potentially be increased via suitable monitoring, feedback and information provision, as well as supportive functions, for example for analysis. <\/p>\n\n\n\n
The article is structured as follows: First, the state of the art in assembly assistance systems, including digital twins, is presented. Then, an approach for an empathic assembly system is conceptualized, its realization described and a proof-of-concept provided. The paper ends with a discussion and an outlook on future work.<\/p>\n\n\n\n
Conventional technologies in assembly assistance <\/h2>\n\n\n\n
Assistance systems are deployed in situations where there is a mismatch between the performance demands of a work task and the skills or capacities of the employees [3, 4]. Typically, these systems acquire the necessary data from their surroundings, process it internally, and then present the information to the worker through an interface. The worker, in turn, receives and cognitively interprets this information, providing feedback via the corresponding input devices [4]. <\/p>\n\n\n\n
Assistance systems can be used in all areas of a company, from production and assembly to maintenance and logistics [5]. A primary objective of assistance systems is to empower users to make prompt, accurate decisions, thereby boosting task efficiency [6]. In manual assembly, cognitive assistance systems are chiefly utilized to deliver work instructions, parts catalogues, technical drawings, and other critical information. Additionally, these systems can document work processes, monitor the accuracy of task execution, and alert workers when deviations occur [7].<\/p>\n\n\n\n
By linking assistance systems to physical entities, they may assist and react based on the physical states of these entities, thereby leveraging digital twins of the related entities. Here, digital twins are seen as information technology representations of things in the real world, which enable the information exchange between the real thing and its representation in the \u201cdigital\u201d world [8]. Thereby, the thing becomes available in the digital world, which enables its planning, monitoring, and control by means of software. For this, a digital twin typically consists of one or more software elements which make models and algorithms accessible and executable. <\/p>\n\n\n\n
Classical digital models of the human, like the ones stemming from digital factory approaches and tools are anthropometric and especially serve for ergonomics. They are too limited for reasoning and must be extended. Lin et al. present an application-independent framework for human digital twins [9]. For this, they differentiate between human external data, human physiological data, human behavior data, human-human social interaction data, and human environmental data. They introduce two essential models in a human digital twin: the human body\/organs model and the human behavior model. Whereas the state of the human is addressed by Lin et al., effect chains and intentions are presented only minorly. <\/p>\n\n\n\n
Cognitive digital twins are seen as extended digital twins [10]. These extensions refer to cognitive capabilities like perception to form an (internal) presentation, attention to enable focus, memory concerning working memory and remembrance, reasoning to draw conclusions, problem-solving to derive solutions and achieve goals, and learning. A combination of knowledge graphs and artificial intelligence can be used to realize a self-learning digital twin for the field of production systems [11].<\/p>\n\n\n\n
The identified systems leverage information and findings about the state of the actor concerned, i.e. the human worker, but they do not take its intentions into account. Thus, they waste the potential for even more cooperative assistance functions, which, based on the premises of the EMOTION project, could result in the increased resilience of the resulting production system. Potentially helpful features of assistance systems for workers include but are not limited to quality monitoring with related feedback and analytics, suitable information provision, and situation-specific manufacturing order scheduling.<\/p>\n\n\n\n
The question that guides this article is as follows: \u201cHow can an empathic assembly assistance system be conceptualized and, on this basis, how can it be realized?\u201d To answer this, an empathic digital twin of a human is proposed, with a bi-directional flow of information between them, which can reason about the state and intentions of the actor and decide on its re-\/action accordingly.<\/p>\n\n\n\n
First steps towards empathic assembly assistance<\/h2>\n\n\n\n
The developed assembly assistance system takes into account both the worker and the work process. To address the worker, a digital twin is applied. Such a twin needs information about the worker and their state so that it can reason and decide on actions, act or trigger activities, and exchange information with other information systems. Therefore, it consists of elements for sensing, realizing cognitive capabilities, acting, and networking and can be seen as modular software system, which may reason on the state of the worker and their intentions. In this way, it lays the foundation for an empathic human digital twin. <\/p>\n\n\n\n
To support cognitive capabilities like reasoning, an ontology in the sense of a semantic data model is required. Essential parts of this model are described in the next paragraphs. Sensing enables the perception of the actor as well as their environment, which may be influenced by the action taken. First examples of a generic sensing chain and potential actions are given later in the text. <\/p>\n\n\n\n
![\"Architecture](\"https:\/\/industry-science.com\/wp-content\/uploads\/2025\/09\/Fig1-1024x518.webp\")
To address the work process, in this case a screwing process, data from a networked screwdriver is transferred via Message Queuing Telemetry Transport (MQTT) to the software system. The data is analyzed by an AI-module and stored in a time series database. Results of the analysis are visualized for the worker by means of a dashboard.<\/p>\n<\/div>\n<\/div>\n\n\n\n
Via an adapter between the time series database and the digital twin, data about the screwing process, its classification, related states of screw joint and the worker can be transferred between the software sub-systems. This enables the further processing and interpretation of the related information by the digital twin, on the one hand, and the display of the results in the dashboard, on the other. The resulting architecture is visualized in Figure 1<\/strong>.<\/p>\n\n\n\n To support the linkage of the software modules, an overarching semantic data model is used. This semantic data model is a modular ontology, which builds upon well-known base ontologies and consists of parts to model humans and the assembly domain, especially with regard to the screwing process and related errors. It integrates these sub-ontologies into a data model in the sense of an internal lingua franca for the overall assistance system. Furthermore, by using a semantic data model, the software can benefit from semantic possibilities like relations and reasoning.<\/p>\n\n\n\n To enable empathic human digital twins, an ontology module to model humans, covering physiological aspects as well as behavioral, social and psychological perspectives, is required. One starting point for such a model can be derived from the classical stress-strain model, taken from work sciences. The classical model and therefore the semantic implementation are based on the premise that work-related stresses result from the work task, work environment, work organization and social climate, which cause stress to a person regarding their resources. The consequences of these stresses have a feedback effect on the individual\u2019s resources. Resources include qualifications as well as physiological and psychological characteristics, and a work-related state, which includes conditions and behavioral tendencies such as fatigue. <\/p>\n\n\n\n Since the employee-oriented ontology contains characteristics that are not clearly measurable, an EMOTION characteristic was defined that, in addition to scalars and quantities based on the Quantities, Units, Dimensions, and Types (QUDT) ontology [12] of the National Aeronautics and Space Administration (NASA), can also represent linguistic variables in the sense of the theory of fuzzy sets.<\/p>\n\n\n\n For sensing, first generic tool chains concerning the worker\u2019s pose were established. Pose recognition of the worker can be applied to analyze ergonomics on the fly and to technically reason about postures that indicate fatigue, so that conclusions can be drawn about the work-related state of the worker. To support this, one tool chain is implemented based on open-source software and standard hardware.<\/p>\n\n\n\n Lightweight OpenPose is an open-source software library based on the well-known OpenPose framework [13], which enables fast and accurate human pose recognition in PyTorch. This results in a system which is executable on a laptop or even single-board computer equipped with a webcam. Consequently, there is no need for expensive AI-devices or specific depth-measuring devices like depth cameras. The detected poses are transferred into the semantic data model and can be analyzed further, based on the RULA method, for example [14].<\/p>\n\n\n\n The conceptual work was implemented by means of the demonstrator, shown in Figure 2<\/strong>, as proof-of-concept. The workstation is a standing desk with a monitor positioned at ergonomic eye level to display the screw-data curves as needed, while results and recommendations can also be shown in the dashboard or on additional interfaces such as tablets.<\/p>\n\n\n\n The developed assistance system uses a networked cordless screwdriver, which captures process data as torque and rotational angle. The data is transmitted to a pre-trained clustering AI-model, which subsequently classifies it into predetermined categories. Based on the resulting measurement curves, the system determines the type of screw used, the number of tightening operations, and whether washers were applied. It then identifies deviations such as faulty screw insertion, the use of substandard screws, or missing\/improper installation of washers using pre-established classification criteria and provides specific corrective instructions accordingly. <\/p>\n\n\n\n Exemplarily, Figure 3<\/strong> shows screwing curves for loosening a screw joint (a) on the left with a nearly fixed screwdriver and (b) on the right with a yielding screwdriver due to insufficient holding force, which may be interpreted as an indicator for fatigue. In the diagrams, the X-axis represents the screwing angle, the Y-axis the screwing torque.<\/p>\n\n\n\n In the case of a faulty screwing process, the system delivers visual feedback through the display. Moreover, inherent characteristics in the process data allow for inferences about possible fault reasons, or error types. Error types are linked to specific work-related states, such as fatigue, and are also based on data from the digital twin model like the captured body pose. Additionally, a signal lamp employing a broad spectrum of colors indicates the worker\u2019s current state and triggers corresponding suggestions, such as break recommendations. To establish a consistent data model encompassing human, machine, product, and factory, the initial focus is set on error types and work-related states in manual assembly. <\/p>\n\n\n\n Empathic technical systems expand cognitive technical systems by considering the intentions of actors. This opens up possibilities concerning the cooperation between technical systems, between technical systems and people, and therefore also for assistance systems. By combining technologies like a networked cordless screwdriver, AI-based analytics of the related screwing process data, human pose recognition, and an enhanced digital twin with semantic reasoning capabilities, a new kind of assembly assistance system can be realized.<\/p>\n\n\n\n The resulting system is implemented with a proof-of-concept via a demonstrator, showing the basic functionality of the system and its components. Parts of the systems, concerning the worker for example, are generic, whereas parts are specific to screwing activities. <\/p>\n\n\n\n The contribution that an empathic manual assembly assistance system makes to helping an enterprise cope with its challenges is admittedly limited. However, it could lead to improvements in quality and productivity as such a system may respond better to the individual worker and their situation. By means of the demonstrator, a study will be conducted to analyze quantitative results of the demonstrator on screwing tasks and its acceptance by workers.<\/p>\n\n\n\n Based on this, the assistance functions as well as the empathic actions and reactions of the system will be advanced and extended. For future work to encompass emotion recognition, which could result in an increase in the empathic capabilities of software systems, ethical and regulatory aspects like the AI Act must be considered.<\/p>\n\n\n\nDemonstrator of the assembly assistance system<\/h2>\n\n\n\n
![\"Demonstrator](\"https:\/\/industry-science.com\/wp-content\/uploads\/2025\/09\/Bild2_Lentes_final-1-1024x513.webp\")
![\"Exemplary](\"https:\/\/industry-science.com\/wp-content\/uploads\/2025\/09\/Bild3_Lentes_final-1-1024x473.webp\")
Initial findings<\/h2>\n\n\n\n