Empathic Assembly Assistance

The situation of industrial companies in Germany can be characterized by challenges of demographic change, stagnating labor productivity, and highly dynamic sales and procurement markets. To support companies in coping with the resulting requirements, the Fraunhofer flagship project EMOTION was initiated. EMOTION aims at advancing the resilience of industrial companies by bringing empathic technical systems for manufacturing to life [1]. 

The basic assumption of EMOTION is that empathy promotes cooperation and thus leads to greater flexibility and resilience. Empathic technical systems are seen as an extension of cognitive technical systems. Thereby, cognitive technical systems can perceive their environment as well as their state and decide as well as act.

In addition, empathic technical systems detect the state and intentions of other actors, may they be human or technical, and use this knowledge to react in an empathic way. Empathic reactions may be altruistic, selfish, or somewhere in between. It is important to note that empathic technical systems consider state and intentions of actors but not emotions of humans—this follows both ethical reasoning and the regulations provided in the Artificial Intelligence (AI) Act of the European Parliament and of the Council of the European Union [2]. 

In this article, the concept of a digital assistance system based on an empathic digital twin of the human as well as Artificial Intelligence for data analytics is introduced. The field of application is manual assembly, chosen because it displays a high potential for the use of technical empathy to support people and because a high proportion of the manufacturing costs of a product are incurred at this stage in the process.

Manual assembly is often still necessary when manufacturing complex, possibly customized goods in small quantities, a primary domain of German industrial companies. By means of an assistance system, quality, cost-effectiveness and flexibility of manual work could potentially be increased via suitable monitoring, feedback and information provision, as well as supportive functions, for example for analysis. 

The article is structured as follows: First, the state of the art in assembly assistance systems, including digital twins, is presented. Then, an approach for an empathic assembly system is conceptualized, its realization described and a proof-of-concept provided. The paper ends with a discussion and an outlook on future work.

Conventional technologies in assembly assistance 

Assistance systems are deployed in situations where there is a mismatch between the performance demands of a work task and the skills or capacities of the employees [3, 4]. Typically, these systems acquire the necessary data from their surroundings, process it internally, and then present the information to the worker through an interface. The worker, in turn, receives and cognitively interprets this information, providing feedback via the corresponding input devices [4]. 

Assistance systems can be used in all areas of a company, from production and assembly to maintenance and logistics [5]. A primary objective of assistance systems is to empower users to make prompt, accurate decisions, thereby boosting task efficiency [6]. In manual assembly, cognitive assistance systems are chiefly utilized to deliver work instructions, parts catalogues, technical drawings, and other critical information. Additionally, these systems can document work processes, monitor the accuracy of task execution, and alert workers when deviations occur [7].

By linking assistance systems to physical entities, they may assist and react based on the physical states of these entities, thereby leveraging digital twins of the related entities. Here, digital twins are seen as information technology representations of things in the real world, which enable the information exchange between the real thing and its representation in the “digital” world [8]. Thereby, the thing becomes available in the digital world, which enables its planning, monitoring, and control by means of software. For this, a digital twin typically consists of one or more software elements which make models and algorithms accessible and executable. 

Classical digital models of the human, like the ones stemming from digital factory approaches and tools are anthropometric and especially serve for ergonomics. They are too limited for reasoning and must be extended. Lin et al. present an application-independent framework for human digital twins [9]. For this, they differentiate between human external data, human physiological data, human behavior data, human-human social interaction data, and human environmental data. They introduce two essential models in a human digital twin: the human body/organs model and the human behavior model. Whereas the state of the human is addressed by Lin et al., effect chains and intentions are presented only minorly. 

Cognitive digital twins are seen as extended digital twins [10]. These extensions refer to cognitive capabilities like perception to form an (internal) presentation, attention to enable focus, memory concerning working memory and remembrance, reasoning to draw conclusions, problem-solving to derive solutions and achieve goals, and learning. A combination of knowledge graphs and artificial intelligence can be used to realize a self-learning digital twin for the field of production systems [11].

The identified systems leverage information and findings about the state of the actor concerned, i.e. the human worker, but they do not take its intentions into account. Thus, they waste the potential for even more cooperative assistance functions, which, based on the premises of the EMOTION project, could result in the increased resilience of the resulting production system. Potentially helpful features of assistance systems for workers include but are not limited to quality monitoring with related feedback and analytics, suitable information provision, and situation-specific manufacturing order scheduling.

The question that guides this article is as follows: “How can an empathic assembly assistance system be conceptualized and, on this basis, how can it be realized?” To answer this, an empathic digital twin of a human is proposed, with a bi-directional flow of information between them, which can reason about the state and intentions of the actor and decide on its re-/action accordingly.

First steps towards empathic assembly assistance

The developed assembly assistance system takes into account both the worker and the work process. To address the worker, a digital twin is applied. Such a twin needs information about the worker and their state so that it can reason and decide on actions, act or trigger activities, and exchange information with other information systems. Therefore, it consists of elements for sensing, realizing cognitive capabilities, acting, and networking and can be seen as modular software system, which may reason on the state of the worker and their intentions. In this way, it lays the foundation for an empathic human digital twin. 

To support cognitive capabilities like reasoning, an ontology in the sense of a semantic data model is required. Essential parts of this model are described in the next paragraphs. Sensing enables the perception of the actor as well as their environment, which may be influenced by the action taken. First examples of a generic sensing chain and potential actions are given later in the text. 

To address the work process, in this case a screwing process, data from a networked screwdriver is transferred via Message Queuing Telemetry Transport (MQTT) to the software system. The data is analyzed by an AI-module and stored in a time series database. Results of the analysis are visualized for the worker by means of a dashboard.

Via an adapter between the time series database and the digital twin, data about the screwing process, its classification, related states of screw joint and the worker can be transferred between the software sub-systems. This enables the further processing and interpretation of the related information by the digital twin, on the one hand, and the display of the results in the dashboard, on the other. The resulting architecture is visualized in Figure 1.

To support the linkage of the software modules, an overarching semantic data model is used. This semantic data model is a modular ontology, which builds upon well-known base ontologies and consists of parts to model humans and the assembly domain, especially with regard to the screwing process and related errors. It integrates these sub-ontologies into a data model in the sense of an internal lingua franca for the overall assistance system. Furthermore, by using a semantic data model, the software can benefit from semantic possibilities like relations and reasoning.

To enable empathic human digital twins, an ontology module to model humans, covering physiological aspects as well as behavioral, social and psychological perspectives, is required. One starting point for such a model can be derived from the classical stress-strain model, taken from work sciences. The classical model and therefore the semantic implementation are based on the premise that work-related stresses result from the work task, work environment, work organization and social climate, which cause stress to a person regarding their resources. The consequences of these stresses have a feedback effect on the individual’s resources. Resources include qualifications as well as physiological and psychological characteristics, and a work-related state, which includes conditions and behavioral tendencies such as fatigue. 

Since the employee-oriented ontology contains characteristics that are not clearly measurable, an EMOTION characteristic was defined that, in addition to scalars and quantities based on the Quantities, Units, Dimensions, and Types (QUDT) ontology [12] of the National Aeronautics and Space Administration (NASA), can also represent linguistic variables in the sense of the theory of fuzzy sets.

For sensing, first generic tool chains concerning the worker’s pose were established. Pose recognition of the worker can be applied to analyze ergonomics on the fly and to technically reason about postures that indicate fatigue, so that conclusions can be drawn about the work-related state of the worker. To support this, one tool chain is implemented based on open-source software and standard hardware.

Lightweight OpenPose is an open-source software library based on the well-known OpenPose framework [13], which enables fast and accurate human pose recognition in PyTorch. This results in a system which is executable on a laptop or even single-board computer equipped with a webcam. Consequently, there is no need for expensive AI-devices or specific depth-measuring devices like depth cameras. The detected poses are transferred into the semantic data model and can be analyzed further, based on the RULA method, for example [14].

Demonstrator of the assembly assistance system

The conceptual work was implemented by means of the demonstrator, shown in Figure 2, as proof-of-concept. The workstation is a standing desk with a monitor positioned at ergonomic eye level to display the screw-data curves as needed, while results and recommendations can also be shown in the dashboard or on additional interfaces such as tablets.

The developed assistance system uses a networked cordless screwdriver, which captures process data as torque and rotational angle. The data is transmitted to a pre-trained clustering AI-model, which subsequently classifies it into predetermined categories. Based on the resulting measurement curves, the system determines the type of screw used, the number of tightening operations, and whether washers were applied. It then identifies deviations such as faulty screw insertion, the use of substandard screws, or missing/improper installation of washers using pre-established classification criteria and provides specific corrective instructions accordingly. 

Exemplarily, Figure 3 shows screwing curves for loosening a screw joint (a) on the left with a nearly fixed screwdriver and (b) on the right with a yielding screwdriver due to insufficient holding force, which may be interpreted as an indicator for fatigue. In the diagrams, the X-axis represents the screwing angle, the Y-axis the screwing torque.

In the case of a faulty screwing process, the system delivers visual feedback through the display. Moreover, inherent characteristics in the process data allow for inferences about possible fault reasons, or error types. Error types are linked to specific work-related states, such as fatigue, and are also based on data from the digital twin model like the captured body pose. Additionally, a signal lamp employing a broad spectrum of colors indicates the worker’s current state and triggers corresponding suggestions, such as break recommendations. To establish a consistent data model encompassing human, machine, product, and factory, the initial focus is set on error types and work-related states in manual assembly. 

Initial findings

Empathic technical systems expand cognitive technical systems by considering the intentions of actors. This opens up possibilities concerning the cooperation between technical systems, between technical systems and people, and therefore also for assistance systems. By combining technologies like a networked cordless screwdriver, AI-based analytics of the related screwing process data, human pose recognition, and an enhanced digital twin with semantic reasoning capabilities, a new kind of assembly assistance system can be realized.

The resulting system is implemented with a proof-of-concept via a demonstrator, showing the basic functionality of the system and its components. Parts of the systems, concerning the worker for example, are generic, whereas parts are specific to screwing activities. 

The contribution that an empathic manual assembly assistance system makes to helping an enterprise cope with its challenges is admittedly limited. However, it could lead to improvements in quality and productivity as such a system may respond better to the individual worker and their situation. By means of the demonstrator, a study will be conducted to analyze quantitative results of the demonstrator on screwing tasks and its acceptance by workers.

Based on this, the assistance functions as well as the empathic actions and reactions of the system will be advanced and extended. For future work to encompass emotion recognition, which could result in an increase in the empathic capabilities of software systems, ethical and regulatory aspects like the AI Act must be considered.

This article was created as part of the Fraunhofer flagship project “Empathic technical systems for resilient production – EMOTION” and thus supported with internal funds.

This is an original article. The German translation can be accessed via DOI: 10.30844/I4SD.25.5.6

You might also be interested in

Serious Games as a Training Tool Game mechanics design to promote resilience Annika Lange , Thomas Knothe

Unforeseen events are increasingly challenging manufacturing companies. Being resilient during crises is becoming a key competence. Serious games (SG) can help make resilience-building processes more transparent. This article derives specific requirements for SG from different phases of resilience and shows how these can be implemented in game mechanics in order to effectively support the training of resilience.

Industry 4.0 Science | Volume 42 | 2026 | Edition 2 | Pages 98-104

Serious Gaming and the Energy Transition Collaborative knowledge generation and interactive understanding of complex interrelationships Janine Gondolf , Gert Mehlmann, Jörn Hartung, Bernd Schweinshaut, Anne Bauer

Conveying the complexity and multifaceted nature of the energy transition to a broad audience is a challenge. This article demonstrates how interactive serious games on a multitouch table can help make connections tangible and comprehensible. The games and the table were used in various conversational contexts. These are presented here in three case vignettes based on participant observation of the different applications, as well as situated and shared reflection. The vignettes demonstrate how interaction can trigger epistemic processes, enable shifts in perspective, and foster collective thinking, all of which are necessary for shaping the future of society as a whole.

Industry 4.0 Science | Volume 42 | 2026 | Edition 2 | Pages 62-69

Experiencing Digital Twins in Production and Logistics The fischertechnik® Learning Factory 4.0 as a development platform for possible expansion stages Deike Gliem , Sigrid Wenzel , Jan Schickram, Tareq Albeesh

The fischertechnik® Learning Factory 4.0 has proven to be a suitable experimental environment for testing digital twins. Depending on the targeted maturity stage, the functions of a digital twin range from status monitoring and forecasting to the operational control of production and logistics systems. To systematically classify these functions, this article presents a maturity model that serves as a framework for the development of a digital twin. Building on this, selected use cases are implemented in a test and development environment based on a system architecture with multi-layered logic structure. These initial implementations serve to highlight application purposes, relevant methods, and typical challenges and potentials in the transfer to real factory environments.

Industry 4.0 Science | Volume 42 | Edition 2 | Pages 30-37 | DOI 10.30844/I4SE.26.2.30

From Brownfield to Industry 4.0 Learning factories as training and testing environment for digital transformation Jakob Weber, Sven Völker

To succeed in their digital transformation, manufacturing companies need engineers with in-depth knowledge of key technologies and concepts, and a profound understanding of the transition from Industry 3.0 to Industry 4.0. This article describes the concept of a learning factory that is continuously subjected to a digital transformation, thereby creating an environment for the development of transformation competencies. The concept of digital transformation is based on digital worker assistance systems and multi-agent systems for production control. These enable the incremental integration of existing resources into the digitalized factory. The learning factory is not presented to students as a completed solution. Instead, it is continuously developed further as part of student projects. This way, it contributes directly to the qualification of personnel for the implementation of Industry 4.0.

Industry 4.0 Science | Volume 42 | 2026 | Edition 2 | Pages 88-96

AI Colleagues? Competence requirements and training for AI use in industry Swetlana Franken

Artificial intelligence is fundamentally changing tasks, roles, and skills in (industrial) companies. Increasingly, it acts as a colleague, preparing decisions, supporting processes, and interacting with people. This article highlights key competence requirements for AI use in industry, presents an integrated competence model, and outlines practical strategies for the transfer of skills. The aim is to prepare companies and employees for humane, competence-oriented AI implementation that combines technological efficiency with human creativity and judgment.

Industry 4.0 Science | Volume 42 | 2026 | Edition 2 | Pages 78-86

Building the Future Workforce Today Trendiation as a strategic framework for employee qualification and training Jürgen Fritz, Sebastian Busse, Ingo Dieckmann, Torsten Laub

As Industry 4.0 and artificial intelligence reshape organizational capabilities, traditional training systems struggle to keep pace with evolving skill requirements. This paper introduces Trendiation—a structured methodology for translating emerging trends into actionable strategies—as a systematic approach to this challenge. Through a workshop-based application examining Edutainment, Human-Centered Design, and Workforce Transformation, we demonstrate how organizations can move from abstract trend identification to concrete qualification requirements and prioritized training initiatives. The method produces a traceable artifact chain spanning trend framing, capability-gap assessment, and implementation roadmaps. Participant evaluations indicate high perceived clarity and practical utility. By bridging foresight analysis with participatory design, Trendiation enables organizations to proactively cultivate adaptive capabilities and build learning cultures aligned with future work ...

Industry 4.0 Science | Volume 42 | 2026 | Edition 2 | Pages 22-29 | DOI 10.30844/I4SE.26.2.22