From Pixels to Presence |
Transforming remote interactions with telepresence robots
| Journal | Industry 4.0 Science |
| Issue | Volume 40, Edition 5, Pages 18-25 |
| Open Access | https://doi.org/10.30844/I4SD.24.5.18 |
| Bibliography | Share | Cite | Download |
Abstract
Article
Today’s world presents several contemporary global challenges that weave through society and encompass professional and social life. Most prominently the current climate change as well as the rippling effects of the COVID-19 pandemic ask for solutions to remote interaction and collaboration, reducing travel while still maintaining social interaction.
With the advancement in electronics, robotics, as well as information and communication technology in the second half of the 20th century, increasingly sophisticated teleoperative systems became possible. Today, those systems are used particularly in areas where people would reach or exceed, e.g. personal, safety or spatial, limits when facing those challenges.
TPRs, by enabling remote interaction and collaboration, significantly reduce the need for travel, thereby contributing to the reduction of carbon emissions associated with transportation. Moreover, these robots facilitate the seamless integration of remote and hybrid work models, aligning with the digital transformation that reshapes modern workplaces by enhancing productivity and communication. Additionally, in the post-COVID era, TPRs offer innovative ways to maintain social connections and ensure inclusivity, bridging the gap created by physical distancing and supporting mental well-being.
Given the maturity of the available technological components for creating TPRs tailored to various specific tasks and needs, the most important question is how to establish an appropriate work/interaction environment where the potential present in TPRs can best be used for supporting the climate, digital and social transformation seen today.
As Figure 1 illustrates, TPRs currently combine a mobile video conferencing system that can be controlled remotely and be equipped with additional tools to manipulate the remote environment. The characteristic feature of a TPR is that a remote user can use a computer or mobile device to control the TPR in a different location.
The TPR allows the remote user to move autonomously and interact with the remote environment, i.e., remote infrastructure/machines and other users. For the local user, the perception of the remote user (driver) is also changed—they appear more socially present (telepresent, first described by Minsky [1]) and real than, for example, on a screen.
Telepresence application domains
Telepresence systems are described in a plethora of contexts that each make use of different aspects of telepresence and/or robotics to solve challenges specific to the context. Youssef et al. [3] give a concise overview of different application areas described in recent scientific literature—see Figure 2 for an overview. They found most research interest to be in the domains of care and assistance, medicine and healthcare, education, and industrial applications. However, telepresence solutions covering more social aspects, like virtual tours, conference attendance or entertainment were also found.
Due to the advancement of medicine, electronics, and communication technology, combined with declining birth rates, especially in industrialized countries, there is a growing need for care and assistance for the elderly [4]. It is estimated that robot-assisted surgeries, wearable devices, and sensors can save up to three million lives each year [5]. TPRs are also used for virtual consultations in the medical and care context to facilitate communication between doctors and isolated patients [6]. In teaching, TPRs can bring teachers and learners from different locations together virtually [7].
Also, students with chronic illnesses or reduced mobility can be brought into an educational context with less effort [8]. While remote work has been on research agendas for decades, the COVID-19 pandemic raised the number of functional remote working and learning solutions.
Industry representatives have thus been found to support TPR usage as it saves time, resources, and CO2 emissions [9]. When deployed on the shop floor, several use cases have been explored to date: internal production audits, product development, vocational training, quality control, equipment setup, maintenance, and logistics [10, 11].
In an international context, i.e., meetings of multinational working in diverse locations as well as international conference attendance, participants have been shown to profit from TPR-enabled virtual participation. With TPRs, attendance can easily be extended to remote lab tours or site visits, thus going beyond the possibilities of video conferencing [12‒14].
During the pandemic, TPRs were newly deployed in cultural institutions like museums [15], allowing for joint visits by on-site visitors and remote persons. At the same time, virtual tours, which enable a more comprehensive visit to a museum, were tested [16]. In the aftermath of COVID-19, TPRs still allow for remote participation with family and friends over (long) distances. These shared activities have been shown to increase well-being as well as reduce the feeling of isolation [17]—while saving travel costs and CO2 emissions as a positive side effect.
Given this broad spectrum of potential use cases, the question arises whether the currently available TPRs are meeting the requirements of their users and, if not, whether additional requirements can be generalized over different use case. The purpose of this paper is to outline the basic requirements for the human-centered development of TPRs, leveraging advances in robotics, sensing, and communication technologies.
Case 1: Integrating TPRs in industrial workflows.
Having a TPR present in a manufacturing environment, especially on the shopfloor, poses a unique set of demands to integrate TPRs into existing workflows, stemming from the environmental conditions of modern manufacturing, maintenance, and inspection tasks. The case study in the field with multiple partner companies follows an iterative, methodical approach divided into two phases: in the first phase, which is presented subsequently, requirements from the application domain are elicited, forming the basis for the second phase. Here, a TPR is designed and implemented to fulfill those requirements in specified workflows; evaluation of the quality of implementation closes phase 2.
To gather insights from different perspectives, requirements elicitation was conducted with industry experts. The participants have backgrounds in the field of robotics and software development or come with industry expertise in manufacturing load carriers (for intralogistics). All participants had prior experience providing services for industrial B2B customers and are required to travel extensively on business.
At the start of the initial workshop, all participants (n=12) were familiarized with the available TPRs to have a common understanding of the robots’ capabilities. To start the process, three state of the art TPRs—Temi by temi Robots as a Service, Double 3 by Double Robotics and OhmniPro by OhmniLabs—were presented in terms of (dis-)advantages, technical requirements for use, and existing application areas. After this presentation, participants gained initial experience in the TPRs’ operation and driving characteristics. These experiences were discussed, and participants came up with five scenarios of potential TPR deployment in a moderated discussion:
- Customer meetings and instructions on a system or a /machine
- Support from experts for maintenance tasks
- Training on a system or a machine
- Product acceptance of load carriers by customers
- Maintenance of load carriers at customers’ premises
Within the identified scenarios and according testing necessities, TPRs were assigned to the participants for individually negotiated testing periods based on the participants preferences and availability (total of 16 weeks; average testing time per scenario: 30,5 min; 268 tests). Evaluation of field tests were done using an approach for systematic evaluation of TPR usage, developed in [18], consisting of structured protocols, including elements on operation, technology, interaction, and user mood.
Aggregated testing results were used for the second workshop, where a joint discussion of results among the participants (n=10) took place to derive requirements to make TPRs more human-centered (Fig. 3). Subsequently, each requirement was assigned one of five levels of difficulty (Likert scale: (very) low, medium, (very) high) for its realization, based on available technical documentation for the TPRs, expert input from fellow researchers working in robotics/computer science as well as feedback from developers. One main aspect was the operating domain of the TPRs, e.g. the integration into existing, proprietary Smart Building/Shopfloor systems.
Case 2: Using TPRs for virtual tours and social integration
Social integration via technological tools poses a unique set of demands stemming from human-machine-interaction as well as social routines, which have additionally been shown to be age-dependent [19, 20]. To address the requirements and expectations towards a TPR within existing social routines, e.g., a joint museum tour by a family group, a strong focus is on the TPRs’ potential to represent remote humans in the local space.
To inquire the socio-technical requirements for virtual tours, a laboratory study with two workshops (approx. 2h) was used, mirroring the requirements elicitation of the industrial workflows. Differently and to better represent future users, participants were recruited to have little or no previous experience with a TPR to represent the social, family-related context of the use case (17 participants; 12 with no previous experience with a TPR).
In the initial workshop, participants jointly developed scenarios of the intended use of the TPR, resulting in four scenarios of potential TPR deployment:
- a guided tour of a school class with a minor participating via TPR
- a guided tour with an adult participating via TPR
- a guided tour with a guide participating via TPR
- an assessment of an art competition with a jury member participating via TPR
For the second workshop, these scenarios were set up in an access-restricted meeting area to represent a museum equipped with paintings, photographs, and installations (Fig. 4). Participants were divided in two groups, with one group enacting the scenario (TPR was controlled remotely by an assistant from a separate room) and the other group observing and documenting the problems and requirements.
At the beginning of the scenario set-up, two state of the art TPRs—Temi and Double 3—were
presented to all participants, allowing them to gain initial experience in the TPRs’ operation and driving characteristics as well as form an initial opinion on their capabilities for social interaction
At the end of the workshop, the participants jointly discussed the problems and identified requirements in a group discussion. The requirements derived were then ranked by the group according to priority from highest (1) to lowest (10) (Fig. 5). The participants reviewed the necessity of the proposed requirements and based on how often a requirement was rated as important and feasible by the participants, this ranking was formed.
As with the study in an industrial setting, one of five levels of difficulty were assigned (Likert scale: (very) low, medium, (very) high) for its realization to each requirement. The ranking is based on available technical documentation for the TPRs as well as expert input from fellow researchers working in robotics/computer science. Again, the application domain, i.e. the museum setting, comes with a unique set of considerations for this assessment. Most prominently the need for safety measures such as virtual barriers to avoid collisions with exhibits, along with emergency control by someone on-site.
Commonality of the requirements of industrial workflows and virtual tours
From analysis of the results, it’s evident that users’ needs in various scenarios, whether in production or social engagement within a cultural setting, largely overlap. Looking at the top three ranked requirements, stable connectivity, audio support for communication, and visual clarity are prominent in both contexts. The necessity of a stable internet connection, or an alternative means of connectivity, emerges as a primary requirement across both domains. This isn’t surprising, as interrupted connectivity undermines the utility of TPR usage. Thus, ensuring connectivity is crucial and should be prioritized.
Audio support, typically in the form of headsets, is highly valued by both user groups, albeit for different reasons. In industrial settings, where noise levels can be high, headsets facilitate smooth and private communication between local and remote TPR users. In cultural settings, where calmness and quietness are expected, headsets are requested to enable communication without disturbing other museum visitors. Similarly, a strong emphasis is placed on visual elements in both domains, albeit for different purposes. Industrial users require a pointer to help remote TPR users reference onsite objects more easily, while museum visitors desire a clear, high-resolution camera image to view installations remotely.
These three categories of requirements are fundamental and likely applicable across various domains. However, the more detailed and diverse requirements specific to each domain should be explored further within those contexts and possibly in additional domains as well.
This research presented in this article has been funded in parts by the Federal Ministry of Education and Research projects PRaeRI (Grant No. 02L21B001) and TeleVeRbundenheit (Grant No. 16SV9138). The authors would like to thank Lena Franzkowiak for her input writing this article.
Bibliography
[1] Minsky, M.: Telepresence. OMNI, (1980), pp. 44–52.[2] Hernandez, F.; Löffler, T.; Schleicher, T.; Bullinger, A. C.: Eignung eines Telepräsenz-Roboters für die Remote-Instandhaltung. In: Instandhaltung als Erfolgsfaktor: Strategie, Lebenszyklusorientierung und Digitalisierung. 2021.
[3] Youssef, K.; Said, S.; Al Kork, S.; Beyrouthy, T.: Telepresence in the Recent Literature with a Focus on Robotic Platforms, Applications and Challenges: Robotics. (2023) URL: https://doi.org/10.3390/robotics12040111
[4] Niemelä, M.; Van Aerschot, L.; Tammela, A.; Aaltonen, I.; Lammi, H.: Towards Ethical Guidelines of Using Telepresence Robots in Residential Care. In: International Journal of Social Robotics (2021) 13, pp. 431–439.
[5] Singh, R. P.; Javaid, M.; Haleem, A.; Vaishya, R.; Ali, S.: Internet of Medical Things (IoMT) for Orthopaedic in COVID-19 Pandemic: Roles, Challenges, and Applications. In: Journal of Clinical Orthopaedics and Trauma 11 (2020) 4, pp. 713-717.
[6] Rojas, A.; Nørskov, S.: Interactions Afforded by Mobile Telepresence Robots in Health Care Settings. In: International Conference on Human-Computer Interaction. (ed): Springer Nature Switzerland. (2023), pp. 138-145.
[7] Ahumada-Newhart, V.; Olson, J. S.: Going to School on a Robot: Robot and User Interface Design Features That Matter. In: ACM Transactions on Computer-Human Interaction (TOCHI) 26 (2019) 4, pp. 1-28.
[8] Page, A.; Charteris, J.; Berman, J.: Telepresence Robot Use for Children with Chronic Illness in Australian Schools: A Scoping Review and Thematic Analysis. In: International Journal of Social Robotics (2021) 13, pp. 1–13.
[9] Appinio: Future of Work Report 2022. URL: https://de.statista.com/page/der-future-of-work-report-2022, Accessed 19.08.2024.
[10] Darvish, K.; Penco, L.; Ramos, J.; Cisneros, R.; Pratt, J.; Yoshida, E.; Ivaldi, S.; Pucci, D.: Teleoperation of Humanoid Robots: A Survey. In: IEEE Transactions on Robotics (2023) 39, pp. 1706-1727.
[11] Hernandez, F.; Waechter, M.; Bullinger, A. C.: A First Approach for Implementing a Telepresence Robot in an Industrial Environment. (2021) URL: https://doi.org/10.1007/978-3-030-79816-1_18
[12] Lee, W.; Park, J.; Park, C. H.: Acceptability of Tele-Assistive Robotic Nurse for Human-Robot Collaboration in Medical Environment. In: Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (2018), pp. 171-172.
[13] Herring, S. C.: Telepresence Robots for Academics. In: Proceedings of the American Society for Information Science and Technology 50 (2013) 1, pp. 1-4. URL: https://doi.org/10.1002/meet.14505001156
[14] Hernandez, F.; Birke, J.; Bullinger, A. C.: The Tribrid-Meeting-Setup: Improving Hybrid Meetings Using a Telepresence Robot. (2023) URL: https://doi.org/10.1007/978-3-031-34609-5_26
[15] Ahumada-Newhart, V.; De La Pena Jr, A.; Riek, L. D.: Promoting Access in a Mexican-American Museum during the Pandemic: Online Community Events and Robots. In: The International Journal of the Inclusive Museum 16 (2023) 2, p. 67.
[16] Kachach, R.; Perez, P.; Villegas, A.; Gonzalez-Sosa, E.: Virtual Tour: An Immersive Low Cost Telepresence System. In: Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA 2020. pp. 504–506.
[17] Tsui, K.; Dalphond, J.; Brooks, D.; Medvedev, M.; McCann, E.; Allspaw, J.; Kontak, D.; Yanco, H.: Accessible Human-Robot Interaction for Telepresence Robots: A Case Study. In: Paladyn Journal of Behavioral Robots (2015) 6, pp. 1–21.
[18] Birke, J.; Hernandez, F.; Schwarzkopf, M.; Zeiner-Fink, S.; Bullinger, A. C.: Entwicklung eines Erprobungskonzeptes für den Einsatz von Telepräsenzrobotern. In: Nachhaltig Arbeiten und Lernen. 69. Kongress der Gesellschaft für Arbeitswissenschaft e. V. 2023.
[19] Börner, K.; Löffler, T.; Bullinger-Hoffmann, A. C.: CheckAge: Screening-Verfahren für die Bewertung alter(n)sgerechter Arbeitsplätze. In: Arbeitswissenschaft und Innovationsmanagement 2 (2017).
[20] Bullinger-Hoffmann, A. C.; Löffler, T.: Die Vernetzte Organisation und der demografische Wandel. In: Vernetzte Organisation. Munich 2014, pp. 96-101.
Your downloads
Potentials: Innovation
You might also be interested in
