Socially Aware Robot Navigation
In 2022 NAVER Corporation deployed a fleet of 100 robots at its new headquarters building, ‘1784’, in Korea. These service delivery robots are designed to move and operate autonomously in indoor environments. The NAVER high rise building is robot-friendly and a testbed for future technology where the goal is to deploy these kinds of robots in any environment. This means the robot’s navigation policies should be aware of humans, the social context and adaptable to new and dynamic scenarios. Our interdisciplinary research team studies human behavior in context and designs robot behaviors for seamless integration into shared spaces as well as implementing them in the robots through different strategies, such as training ML models.
Our research on socially acceptable robot navigation is illustrated in the scenario of robots entering an elevator shared with humans. The NAVER robots currently have their own elevator called “Roboport” but, to move away from being dependent upon such specific robot infrastructures, we need to develop policies that enable our robots to share regular elevators with people. This elevator scenario may appear to be simple but it involves subtle and nuanced negotiations, navigation within constrained spaces and precise timing. Also, although not yet entirely solved, navigating vertically is a critical ability for autonomous robots operating in high rise buildings. This step is the first towards developing a set of core navigation behaviors that can be generalized to other scenarios and social contexts.
Exploring different approaches to robot behavior design
We’ve investigated the potential advantages of an approach that deliberately differentiates robot behavior from human-like behaviors. While there is a wealth of research on designing robots that mimic human social norms [1], studies on the interactions between humans and robots (HRI) reveal that people’s responses to robots differ significantly from their responses to other humans [2]. This suggests that social expectations for robots might actually diverge from those for humans. Our research has focused on understanding the extent to which robots should emulate human-like navigation behaviors versus adopting machine-like behaviors, which leverage unique robotic capabilities [3], can limit the amount of social understanding required, as well as the illusion of social competence [4]. Through our research we’ve assessed the impact of such a distinction based on their social acceptance and successful integration into human-occupied spaces.
We first conducted ethnographic observations to understand, in practice, how people behave in what we might call ‘the practice [5] of taking the elevator’. While taking an elevator is in many ways straightforward, it is also constituted of practices that are methodical and accountable, with normative components (i.e. first come first served) and nuanced, often including the non-verbal use of space and resources.
Before performing in-person experiments we conducted a preliminary online survey to evaluate the hypotheses that i] people desire different robot behaviors when queuing for elevators ii] prioritize human access over robot access regardless of arrival order, and iii] have different positional preferences for robots and humans within an elevator. Our analysis of the survey results has revealed that, while human-like behaviors in a robot are perceived as familiar and easy to understand, some machine-like behaviors may be preferred or closer to a priori expectations of how a robot should behave. Some might consider a behavior natural if it is similar to what a human would do but, for others it might mean that robots are not expected to behave like humans, so another set of behaviors is more appropriate.
Building on the findings of the online study, we carried out an in-person study to investigate what the right balance is when designing policies that integrate human-like and machine-like behaviors, to leverage the best of each. In this Wizard-of-Oz experiment, participants experienced a variety of elevator waiting situations with the robot. They were video recorded and provided feedback via questionnaires and follow-up interviews. The findings suggested a preference for a blend of behaviors, where machine-like fixed and marked waiting positions were deemed more visible, predictable and efficient. The study also highlight the flexibility required in robot behaviors to adapt to complex, real-world situations, pointing to the potential limitations of predefined machine-like positions in dynamic human environments. In general, the robot’s visibility and timely communication of its intentions i.e. when the elevator doors opened, were crucial for participants to understand the robot’s actions, especially when it sought to prioritize its entry.
Beyond proxemics towards embedding context specific social norms in robot navigation
While a fixed position proposed in the machine-like approach has advantages in terms of predictability, consistency of behavior, and ease of implementation, it may not always be available in practice or may not satisfy other technical dependencies of the service carried out by the robot so we tackle the problem of finding socially acceptable waiting positions autonomously. We consider two scenarios, i.e., (1) when the robot gives way to allow people to enter the elevator before it, and (2) when the robot claims priority due to an urgent task and intends to board before people.
We train a machine learning model that, given the robot’s sensory inputs, would segment areas as socially acceptable for waiting. As for other practical HRI problems, annotated data is scarce and labels costly to gather. We therefore provide scaffolding to the training by operationalizing our prior knowledge on context specific social norms, in a set of feature maps applicable to any configuration of the space shared by one or multiple bystanders, the robot and the elevator. In addition to allowing the training on our low data regime, the feature maps are (a) reusable for similar social navigation scenarios, and (b) provide interpretable insights on which social norms both the annotators and the models consider more relevant. You can see the overview of this procedure in Figure 3. We’re integrating the elevator waiting algorithm into the navigation pipeline, initially through simulation to refine and validate its effectiveness before implementing it in the real robots.
Communicating robot intent
To minimize disruption in contexts where people are likely to encounter a robot on several occasions throughout the day, and to avoid using speech-based communication to limit people’s expectations of what the robot can understand, we’ve also worked on designing non-verbal behaviors for the robot so that it can effectively communicate it’s intent when navigating. The focus of our research is on two modalities: motion and prosody-based sounds. In motion-based communication we’re been inspired by human non-verbal cues and previous research and have designed two different gestures to communicate the robot’s intent when boarding an elevator: a slow backward motion [6] with blue pulsing lights to yield priority/give way, and a slow forward motion (also referred to as anticipatory motion [7]) with an alert sound and directional blue light to claim priority. Both gestures were understood by most of the participants in our user experiment. They perceived the yielding motion as polite behavior, similar to what people do in this situation, and the step forward as making the interaction faster, but also more aggressive, which is only justified in urgent cases. In sound, we’re currently conducting experiments with prosody-based sounds and planning the integration of the different communication modalities for evaluation in user experiments.
Video depicting the ‘Yielding’ and ‘Stepping Forward’ communication behaviors of a test robot in NAVER LABS Europe.
Deploying navigation behaviors in real robots and real-world environments is a critical step to their integration and adoption. It involves taking navigation algorithms from controlled simulations and applying them to complex, unpredictable scenarios, where their effectiveness can be evaluated and we continue to work towards this goal. At the same time, we’re interested in adapting our elevator policies to new scenarios by tailoring the feature maps as needed to different requirements. Feature maps provide a layer of controllability to the models, allowing us to adjust them for scenarios where social norms may differ (i.e. across cultures) or where new social norms might need to be engineered.
Recent publications
- Exploring machine-like behaviors for socially acceptable robot navigation in elevators. Danilo Gallo, Shreepriya Gonzalez-Jimenez, Maria Antonietta Grasso, Cecile Boulard, Tommaso Colombino. 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI). Best paper award.
- Investigating the Integration of Human-Like and Machine-Like Robot Behaviors in a Shared Elevator Scenario. Danilo Gallo, Prescillia Bioche, Jutta Willamowski, Tommaso Colombino, Shreepriya Gonzalez-Jimenez, Herve Poirier, Cecile Boulard. Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI). Finalist for Best Paper Award.