Experiential Knowledge Powered by AI |
Practical insights from industry
| Journal | Industry 4.0 Science |
| Issue | Volume 42, 2026, Edition 5, Pages 128-135 |
| Open Access | https://doi.org/10.30844/I4SE.26.5.15 |
| Bibliography | Share | Cite | Download |
Abstract
Keywords
Article
Preserving and utilizing experiential knowledge continues to pose significant challenges for companies [7]. This article focuses specifically on experiential knowledge that is relevant for making decisions not guided by formalized policy, reducing task complexity, or optimizing time and resource consumption [1, 4]. Currently, there is a risk of losing experiential knowledge due to the retirement of employees, workforce reductions, or changes in job responsibilities resulting from automation and digitization [7]. As our research over the past 15 years shows, companies must respond by implementing solutions for identifying, preserving, and making experiential knowledge available [5, 8]. At the same time, however, effective strategies are needed to continuously identify and integrate newly emerging knowledge into the corporate knowledge base [3].
With the development of generative AI, particularly retrieval-augmented generation (RAG), high expectations are placed on technical support for capturing experiential knowledge [9]. RAG systems combine language models with intelligent retrieval mechanisms for local corporate knowledge bases, enabling context-sensitive access [6].
Although RAG systems hold great potential for capturing experiential knowledge, their practical implementation—especially in small and medium-sized enterprises—poses significant technical, organizational, and personnel challenges. In addition to building suitable knowledge bases, issues related to data preparation, system integration, and operational implementation must be addressed. The PAL project (PerspektiveArbeit Lausitz, funded by the German Federal Ministry of Research, Technology and Space 2021–2027) developed a RAG tool addressing these requirements. The tool has been tested in various business contexts since 2022 [9].
Research objective: Identify framework conditions for the deployment of RAG systems
This research aims to analyze the framework conditions for the successful use of RAG to capture experiential knowledge. It also derives recommendations for the successful implementation of AI-powered solutions.
The study uses an action research approach [2], combining scientific knowledge acquisition with the collaborative development and testing of practical solutions with corporate partners. It encompasses four corporate case studies. Project teams were formed within the companies and a co-creation process was established. Regular project status meetings served to document ideas, decisions, and identified deviations. The data set consists of this project documentation, as well as workshop results and guided feedback sessions with employees.
To address each business case, a pilot solution was implemented within the company and RAG systems were tested in-house.
- Case A: To streamline assembly processes, the existing, very extensive (> 150 pages) assembly instructions can be searched by employees via the RAG system in order to obtain answers to assembly problems. In parallel, interviews are conducted with experienced employees to document tips for work performance—supplementing the instructions—and to integrate them into the knowledge base.
- Case B: An RAG system supports the searching of existing document collections that are over 10 years old. The technical challenge lies in processing different file formats.
- Case C: In an assembly area, the RAG system is used to determine how employees can be supported with assembly tasks that have not been performed in a long time. As a supplement, a self-produced video is integrated to provide hands-on insight into specific processes.
- Case D: The RAG system supports field employees with maintenance and commissioning tasks, relieving back-office staff of recurring technical inquiries. All relevant user manuals and work instructions are provided as PDF files. In addition, one-on-one discussions are held between departing employees with extensive experience and less experienced colleagues regarding specific work tasks. This knowledge source, alongside a video created by another employee, is integrated into the RAG system.
Technical challenges and solutions
The literature describes a number of technical challenges associated with the use of RAG systems [6]:
- The selection of appropriate strategies and parameters for processing digital information (chunking, embedding) is crucial for providing relevant and accurate information on demand. This is particularly important given the small datasets in an RAG system.
- Despite the retrieval component, RAG systems can generate erroneous or fabricated content (so-called hallucinations).
- Processing large, heterogeneous corporate datasets requires sufficient computing power.
- Corporate data is often available in various formats. Integration requires specialized interfaces or preprocessing.
- The limited context length of large language models (LLMs) makes it difficult to process complex queries that aggregate information from multiple documents.
- Outdated, incomplete, or inconsistent data leads to errors in retrieval and increases the risk of hallucinations.
These aspects are reflected in the corporate use cases. The technical framework and design decisions identified in the case studies are presented below.
Both systems run on a high-performance PC equipped with a GeForce RTX graphics card. The latter is a prerequisite for an acceptably fast search process; however, given the currently low number of concurrent users, no wait times occurred.
In all the cases examined, the focus is on information retrieval, not on AI-assisted text generation. Copilot Pro, for example, formulates short answers to questions by evaluating several relevant, ranked documents or passages and combining them into a concise response. To reduce the risk of hallucinations and promote acceptance, the focus—in consultation with users—is placed on displaying relevant documents rather than formulating full answers.
Organizational and personnel requirements
The case studies represent different use cases for the deployment of RAG systems. Scientific monitoring of the trials shows that there is often a gap between expectations for these systems and the prerequisites for their successful deployment. Key challenges lie in the availability of suitable digital knowledge sources and the willingness to allocate resources to ongoing development and maintenance. Several concrete findings were derived:
In Case B, it became apparent that RAG systems could be usefully deployed, particularly with large document collections that have grown over an extended period. The ability to find information independently of exact search terms is especially beneficial for new employees. The list view of references to the sought-after information provides a quick and comprehensive overview of past specifications. While there is a specific need for information in Case D, the clearly structured and well-known work instructions available in Case C eliminate the need for additional information-based support.
Dealing with different file formats is technically challenging but solvable. WINI was equipped with additional functionality to automatically convert text documents into PDFs. However, graphic- and number-based formats, such as Excel, cannot be processed by language-model-based systems unless they contain analyzable text as metadata. For the employee videos documenting work processes (Case C, Case D), transcripts of the commentary were therefore created and integrated into the system. This makes the videos accessible for information retrieval based on the “image description.”
In Cases A, B, and C, the system could not be integrated into the company’s IT infrastructure for cybersecurity reasons, which is why a separate computer was required in each case. This both negatively affects acceptance and hinders use in the work process. In Case D, the browser-based WINI system was made available to field service employees on tablets for order processing. It is evident that involving the relevant IT department to establish individual requirements is of immense importance.
Overall, employee acceptance plays a central role in the provision and use of experiential knowledge. The employees in Cases B through D tested WINI and provided positive feedback on its functionality and usability. In Case A, the use of RAG failed, in part, due to a lack of willingness on the part of employees to make their experiential knowledge explicit. Without discernible benefits for work performance or targeted incentives, the willingness to make tacit knowledge explicit is low. After all, the continuous collection, review, and maintenance of information from various sources involves significant effort. In Case D, high appreciation from management led to an increased number of video files created independently in the knowledge base.
Another key finding concerns the time it takes for the RAG system to be put to productive use. Cases C and D show that employees who have previously worked without digital support do not immediately or automatically develop a need for these new forms of information retrieval. In Case D, it took seven months for employees to begin systematic testing. It is therefore sensible to select one initial use case, such as a new product or process for which there is still little experiential knowledge and thus a high need for knowledge exchange. A sufficiently long trial period, including an initial rollout and repeated encouragement by management, is necessary to assess usage across the workforce and, consequently, the solution’s effectiveness.
The knowledge base is the priority
However, as the case studies show, the priority for AI-supported knowledge provision clearly lies with the knowledge base and is therefore the responsibility of the user companies. The well-known large language models reach their limits when dealing with domain-specific and knowledge-intensive queries [6]. RAG systems therefore utilize company-specific know-how, which must be sufficiently complex and comprehensive to generate added value.
To investigate the influence of the knowledge base on the performance of RAG systems, these were evaluated comparatively in Case D.
To this end, identical queries were posed to the respective RAG systems. The evaluation focused on the number of correctly identified documents or passages, as well as the usability of the generated output for answering the queries. In addition, a qualitative assessment was conducted regarding the manner in which answers were provided and their suitability for practical use (Fig. 4).
Questions:
- How should the office chair be adjusted?
- How should the monitor be positioned? What angles should be observed?
- Which distances should be maintained at the desk and in the surrounding area?
- What should be considered when using LED light sources?
- Who is responsible for designing computer workstations?
The tests reveal a correlation between system functionality, knowledge base, query type, and user expectations regarding the output of the RAG system. For example, while Copilot’s fully formulated responses initially provide a high level of convenience, their practical usefulness depends on users’ willingness to read and verify them. Company-specific alignment of system functionality, knowledge base, and user expectations is therefore central to a successful implementation strategy.
The definition and continuous development of the knowledge base should be closely linked to corporate strategy, e.g., through regular workshops with employees, as carried out in Case D. New key topics can be derived from these workshops, for which experiential knowledge should be jointly documented either retrospectively or as part of the ongoing process. In addition, the knowledge base’s timeliness and quality must be maintained, for example by analyzing usage frequency, user feedback, and frequently requested knowledge content.
Conclusion and further recommendations
The case studies confirm that the successful deployment of RAG systems depends on the quality of the knowledge base and appropriate organizational and personnel conditions [6]. The long-term benefits of such systems stem less from the choice of a specific technical solution than from the continuous development, maintenance, and use of the knowledge base.
To integrate a user perspective, testing and evaluation tools must be developed to assess the functionality, acceptance, and benefits of RAG systems under real-world conditions. Further research and development regarding employees’ ability to make knowledge explicit is also required [8]. This involves fostering individual reflection and articulation skills, as well as implementing documentation principles that support automated processing.
The focus in implementation should be on process design and knowledge base development, independent of the specific technical solution, as technologies are evolving rapidly [6, 9]. Acceptance can be supported by transparent communication of mutual benefits for all affected employee groups.
The PAL – PerspektiveArbeit Lausitz project is funded by the Federal Ministry of Research, Technology, and Space under grant numbers 02L19C300 – 02L19C333.
Bibliography
[1] Baumhauer, M.; Meyer, R.: Berufliche Handlungsfähigkeit und Erfahrungswissen: Stellenwert für die Facharbeit in der digitalen Transformation. Eine empirische Analyse am Beispiel der Chemieindustrie. In: Arbeit 30 (2021), pp. 263–282. DOI: https://doi.org/10.1515/arbeit-2021-0019.[2] Franke-Jordan, S.; Hausmann, S.; Hunger, K.; Krause-Jüttler, G.; Ott, G.; Schmauder, M.: Wissen teilen – Technologietransfer aus Sicht der Arbeitsforschung. In: Borowski, E.; Cernavin, O.; Hees, F.; Joerißen, T. (Eds.): Erfolgreicher Transfer in der Arbeitsgestaltung. Wie Dienstleistungen zur präventiven Arbeitsgestaltung und Ergebnisse der Arbeitsforschung die Akteure in Unternehmen wirkungsvoll erreichen. Münster/New York 2023. DOI: https://doi.org/10.31244/9783830998082.
[3] Bretschneider, J.; Zösch, A.; Ott, G.; Hausmann, S.; Gonsior, J. et al.: Digitalisierung in der Werkstoffprüfung. In: Nitsch, V.; Brandl, C.; Häußling, R.; Roth, P.; Gries, T.; Schmenk, B. (Eds.): Digitalisierung der Arbeitswelt im Mittelstand 3. Ergebnisse und Best Practice des BMBF-Forschungsschwerpunkts „Zukunft der Arbeit: Mittelstand – innovativ und sozial“. Berlin/Heidelberg 2023, pp. 369–402.
[4] Büssing, A.; Herbig, B.: Implizites Wissen und Wissensmanagement – Schwierigkeiten und Chancen im Umgang mit einer wichtigen menschlichen Ressource. In: Zeitschrift für Personalpsychologie 2 (2003) 2, pp. 51–65. DOI: https://doi.org/10.1026//1617-6391.2.2.51.
[5] CIMTT Center for Production Engineering and Organization: StratEWiss – Entwicklung innovativer Strategien zur Ermittlung und Systematisierung betrieblicher Wissensressourcen, insb. Erfahrungswissen, und neuer Methoden für deren Transfer unter Berücksichtigung der demografischen Entwicklung. TU Dresden. URL: https://tu-dresden.de/ing/maschinenwesen/cimtt/projekte/abgeschlossene-forschungsprojekte/stratewiss-development-of-innovative-strategies-for-identifying-and-systematizing-organizational-knowledge-resources-particularly-experiential-knowledge-and-new-methods-for-their-transfer-taking-demographic-trends-into-account, accessed 11.06.2026.
[6] Gao, Y.; Xiong, Y.; Gao, X.; Jia, K.; Pan, J. et al.: Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv 2024. DOI: https://doi.org/10.48550/arXiv.2312.10997.
[7] Maier, E.: Erfahrung – der unsichtbare Erfolgsfaktor in Wirtschaftsunternehmen. Dokumentation der Ergebnisse einer Befragung von Führungskräften in der Schweiz, Österreich und Deutschland. Europa-Institut Erfahrung & Management – METIS. 2016.
[8] Ott, G.; Hausmann, S.; Schmauder, M.: Werkzeuge zum Umgang mit Erfahrungswissen in der Werkstoffprüfung. In: Vortragsband. DGM – Deutsche Gesellschaft für Materialkunde e.V. 2022.
[9] Schönwälder, E.; Hahmann, M.; Ott, G.: Using Compact Retrieval-Augmented Generation for Knowledge Preservation in SMBs. In: Human Interaction and Emerging Technologies (IHIET-AI 2025): Artificial Intelligence and Future Applications. 2025. DOI: https://doi.org/10.54941/ahfe1005891.
Potentials: Management Strategy Training
You might also be interested in
