Table des matières

  • 1. Introduction
  • 2. SoftBank Robotics
    • 2.1. Proceedings de workshops et de conférences
    • 2.2. Journaux
  • 3. Armines
    • 3.1. Proceedings de workshops et de conférences
  • 4. INRIA
    • 4.1. Proceedings de workshops et de conférences
    • 4.2. Journaux
  • 5. LIMSI
    • 5.1. Proceedings de workshops et de conférences
    • 5.2. Journaux
    • 5.3. Chapitres de livre
  • 6. LAAS
    • 6.1. Proceedings de workshops et de conférences
    • 6.2. Journaux
  • 7. LIRMM
    • 7.1. Proceedings de workshops et de conférences
  • 8. ISIR
    • 8.1. Proceedings de workshops et de conférences
    • 8.2. Journaux
  • 9. Participation à des workshops et des tables rondes
    • 9.1. Workshops
    • 9.2. Invitations, présentations, keynotes
    • 9.3. Contributions à des universités d’été ou d’hiver
  • 10. Thèses défendues
    • 10.1. INRIA
    • 10.2. LAAS
    • 10.3 LIMSI-CNRS

1. Introduction

Ce livrable a pour fonction de référencer l’intégralité des publications scientifiques issues du projet Romeo2 sur la troisième période clé du projet, c’est à dire du 01/07/2015 au 31/10/2017. 33 publications scientifiques ont été réalisée par les partenaires : 33 dans des conférences internationales, 1 dans une conférence française et 8 dans des revues internationales.

Conférences :
– IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man) 2015, 2017
– IEEE Conference on Human Robotics Interaction (HRI) 2016, 2017
– International Conference on Social Robotics (ICSR) 2015, 2016, 2017
– IEEE International Conference on Humanoid Robots (Humanoids) 2015, 2016, 2017
– IEEE International Conference on Intelligent Robots and Systems (IROS) 2015, 2016, 2017
– International Workshop on Spoken Dialog System technology (IWSDS) 2016, 2017
– Workshop on Designing Humour in Human-Computer Interaction (HUMIC) 2017
– International Conference on Human-Computer Interaction (HCII) 2016
– IEEE International Conference on Robotics and Automation (ICRA) 2017
– IEEE International Conference on Control, Automation, Robotics ans Vision (ICARCV) 2016
– Congrès Reconnaissance des Formes et Intelligence Artificielle (RFIA) 2016
– International Conference on Distributed, Ambient, and Pervasive Interactions 2016
– ETHics In Corpus Collection, Annotation & Application Workshop (ETHI-CA) 2016
– ACM International Conference on Multimodal Interaction 2015

Journaux :
– International Journal of Social Robotics (IJSR)
– IEEE Automation & Letters
– IEEE Robotics & Automation magazine
– IEEE Transactions on Affective Computing
– IEEE/ACM transactions on audio, speech and language processing
– Lecture Notes in Computer Science, Springer
– Frontiers in Psychology

2. SoftBank Robotics

2.1. Proceedings de workshops et de conférences

  • Giulio Cerruti, Damien Chablat, David Gouaillier, Sophie Sakka, “ALPHA: A hybrid self-adaptable hand for a social humanoid robot”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2016, Deajeon, South Korea.

This paper presents a novel design of a compact and light-weight robotic hand for a social humanoid robot. The proposed system is able to perform common hand gestures and self-adaptable grasps by mixing under-actuated and self-adaptable hand kinematics in a unique design. The hand answers the need for precise finger postures and sensor-less force feedback during gestures and for finger adaptation and autonomous force distribution during grasps. These are provided by a dual actuation system embodied within the palm and the fingers. Coexistence is ensured by compliant transmissions based on elastomer bars rather than classical tension springs, thanks to their high elastic coefficient at reduced sizes and strains. The proposed solution significantly reduces the weight and the size of the hand by using a reduced number of small actuators for gesturing and a single motor for grasping. The hand prototype (ALPHA) is realized to confirm the design feasibility and functional capabilities. It is controlled to provide safe human-robot interaction and preserve mechanical integrity in order to be embodied on a humanoid robot.

  • Semantic-based interaction for teaching robot behavior compositions, Paléologue, V. and Martin, J. and Pandey, A. K. and Coninx, A. and Chetouani, M., in 26th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2017

Allowing humans to teach robot behaviors will facilitate acceptability as well as long-term interactions. Humans would mainly use speech to transfer knowledge or to teach high level behaviors. In this paper, we propose a proof-of- concept application allowing a Pepper robot to learn behaviors from their natural-language- based description, provided by naive human users. In our model, natural language input is provided by grammar-free speech recognition, and is then processed to produce semantic knowledge, grounded in language and primitive behaviors. The same semantic knowledge is used to represent any kind of perceived input as well as actions the robot can perform. The experiment shows that the system can work independently from the domain of application, but also that it has limitations. Progress in semantic extraction, behavior planning and interaction scenario could stretch these limits.

  • Gaze and Filled Pause Detection for Smooth Human-Robot Conversations, Miriam Bilac, Marine Chamoux, and Angelica Lim, in Humanoids 2017

Let the human speak! Interactive robots and voice interfaces such as Pepper, Amazon Alexa, and OK Google are becoming more and more popular, allowing for more natural interaction compared to screens or keyboards. One issue with voice interfaces is that they tend to require a « robotic » flow of human speech. Humans must be careful to not produce disfluencies, such as hesitations or extended pauses between words. If they do, the agent may assume that the human has finished their speech turn, and interrupts them mid-thought. Interactive robots often rely on the same limited dialogue technology built for speech interfaces. Yet humanoid robots have the potential to also use their vision systems to determine when the human has finished their speaking turn. In this paper, we introduce HOMAGE (Human-rObot Multimodal Audio and Gaze End-of- turn), a multimodal turntaking system for conversational humanoid robots. We created a dataset of humans spontaneously hesitating when responding to a robot’s open-ended questions such as, « What was your favorite moment this year? ». Our analyses found that users produced both auditory filled pauses such as « uhhh », as well as gaze away from the robot to keep their speaking turn. We then trained a machine learning system to detect the auditory filled pauses and integrated it along with gaze into the Pepper humanoid robot’s real-time dialog system.
Experiments with 28 naive users revealed that adding auditory filled pause detection and gaze tracking significantly reduced robot interruptions. Furthermore, user turns were 2.1 times longer (without repetitions), suggesting that this strategy allows humans to express themselves more, toward less time pressure and better robot listeners.

  • When a Social Robot might Learn to Support Potentially Immoral Behaviors on the name of Privacy: The Dilemma of Privacy vs. Ethics for a Socially Intelligent Robot, Amit Kumar Pandey, Rodolphe Gelin, Martina Ruocco, Marco Monforte, and Bruno Siciliano, in Privacy-Sensitive Robotics 2017, HRI 2017

Robots are becoming commonplace. They are also becoming capable of learning. Combination of these, from one perspective, might also be problematic. What if someone teaches a robot some ‘bad’ things? As a precautionary measure robot could be pre-programmed to not learn a list of ‘bad’ things. But on the other side robots will have to be programmed for supporting the privacy of the people. What if someone uses the ‘privacy’ channel to teach ‘bad’ things, and as bad as making the robot to be part of supporting even potentially unethical and immoral behaviors? This paper illustrates such possibilities through a simple human-robot interaction based robot learning system. The aim is to proactively fetch the attention of the community towards such possible future threats and how to address those scientifically. The presented system is part of an ongoing study about how people expect a social robot to behave if there is a dilemma of Privacy vs. Moral, Social and Ethical accountability.

  • A novel concept of Human-Robot competition for evaluating a robot’s reasoning capabilities in HRI, Amit Kumar Pandey, Lavindra De Silva, and Rachid Alami, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (LBR), Christchurch, 2016, pp. 491-492.

For intelligent Human-Robot Interaction (HRI), a robot should be equipped with some core reasoning capabilities such as perspective taking, effort analysis, and affordance analysis. This paper starts to explore how a robot equipped with such reasoning abilities could be evaluated. To this end, inspired by the Turing test, we design a game involving a human-robot competition scenario. Interestingly, the participants’ subjective feedback, which tended to compare the robot’s abilities with their own, points toward potential criteria for developing benchmark scenarios and evaluation matrices.

  • A Human-Robot Competition: Towards Evaluating Robots’ Reasoning Abilities for HRI, Amit Kumar Pandey, Lavindra De Silva, and Rachid Alami, The Eight International Conference on Social Robotic (ICSR) 2016

For effective Human-Robot Interaction (HRI), a robot should be human and human-environment aware. Perspective taking, effort analysis and affordance analysis are some of the core components in such human-centered reasoning. This paper is concerned with the need for benchmarking scenarios to assess the resultant intelligence, when such reasoning blocks function together. Despite the various competitions involving robots, there is a lack of approaches considering the human in their scenarios and in the reasoning processes, especially those targeting HRI. We present a game that is centered upon a human-robot competition, and motivate how our scenario, and the idea of a robot and a human competing, can serve as a benchmark test for both human-aware reasoning as well as inter-robot social intelligence. Based on subjective feedback from participants, we also provide some pointers and ingredients for evaluation matrices.

  • Evaluating Human Robot Dialog based Affordance Learning and the Challenges, Amit Kumar Pandey, Coline Le Dantec, and Rodolphe Gelin, 7th International Workshop on Spoken Dialogue Systems (IWSDS 2016)

Perception of Affordances shapes how we interact with environment. A novel way to learn affordances at symbolic level is through human-robot natural dialog. The challenge is how to evaluate such learned knowledge. As a first step this paper studies how do humans associate affordances and objects in a given scenario and can it be used as ground truth to evaluate the content learned by the robot.

2.2. Journaux

  • Amit Kumar Pandey, Rachid Alami and Kazuhiko Kawamura, “Developmental Social Robotics: an Applied Perspective”, International Journal of Social Robotics, Editorial of Special Issue. Pages 417-420, Volume 7(4), Aug. 2015.

For robots to coexist with us in harmony and be our companion, they should be able to explicitly reason about humans, their presence, the social and human-centered environment, and the social-cultural norms, to behave in socially expected and accepted manner. To develop such capabilities, from psychology, child developmental and human behavioral research, we can identify some of the key ingredients, such as the abilities to distinguish between self and others, and to reason about affordance, perspective taking, shared spaces, social signals, emotions, theory of mind, social situation, etc., and the capability to develop social intelligence through the process of social learning. Researchers across the world are working to equip robots with some of these aspects from diverse perspectives and developing various interesting and innovative applications. This special issue is intended to reflect some of those high-quality research works, results and potential applications.

3. Armines

3.1. Proceedings de workshops et de conférences

  • Cruz, A., Agrigoroaie, R. and Tapus, A. (2017) « Improving User’s Performance by Motivation: Matching Robot Interaction Strategy with User’s Regulatory State » In Proceedings of International Conference on Social Robotics (ICSR 2017), Tsukuba, Japan

The presence of a robot in our everyday life can generate both positive and negative effects on us. While performing a difficult task, the presence of a robot can generate a negative effect on the performance and it can also increase the stress and anxiety levels. In order to minimize these undesired effects, we propose the use of user’s motivation, based on the Regulatory Focus Theory. We analyze the effects of using Regulatory oriented strategies in a robot speech, when giving a person the instructions of how to perform a Stroop Test. We found evidence that matching the Chronic Regulatory state of the participants with the Regulatory oriented strategy of the robot improves the user’s performance, and a mismatch leads to an increase of cognitive load and stress in the participants.

Ferland, F., and Tapus, A. (2017) « Crowd Sourcing Approach Behavior Control » In Proceedings of RO-MAN 2017, Lisbon, Portugal

For service robots to be well received in our daily lives, it is desirable that they appear as friendly as possible rather than some unfriendly characters. While a robot’s physical appearance influences this perception, its behavior also has an impact. In order to be sure that a specific robot behavior will be correctly perceived, we propose to its potential users to shape the robot’s behavior. In this paper, a specific behavior, “approaching a person”, is evaluated with a Robosoft Kompaï robot. To avoid logistics issues associated with having large groups of novice users performing demonstrations on a physical robot, a web-based approach built around a simulation of the actual robot is proposed. The relationship between the robot and the person is described by the two dimensions of the interpersonal circumplex: communion (hostile or friendly) and agency (submissive or dominant). The users can adjust three parameters of the approach behavior (i.e., distance, trajectory curvature, and deceleration) in a manner that corresponds the best to the described relationship. An analysis of the data from 69 users is presented, along with a verification experiment done with 10 participants and the real robot. Results suggest that users associate hostile robots with straight trajectories, and submissive robots with smoother deceleration.

  • Cruz, A and Tapus, A. (2017) « Learning Users’ and Personality-Gender Preferences in Close Human-Robot Interaction » In Proceedings of RO-MAN 2017, Lisbon, Portugal

Robots are expected to interact with persons in their everyday activities and should learn the preferences of their users in order to deliver a more natural interaction. Having a memory system that remembers past events and using them to generate an adapted robot’s behavior is a useful feature that robots should have. Nevertheless, robots will have to face unknown situations and behave appropriately. We propose the usage of user’s personality (introversion/extroversion) to create a model to predict user’s preferences so as to be used when there are no past interactions for a certain robot’s task. For this, we propose a framework that combines an Emotion System based on the OCC Model with an Episodic-Like Memory System. We did an experiment where a group of participants customized robot’s behavior with respect to their preferences (personal distance, gesture amplitude, gesture speed). We tested the obtained model against preset behaviors based on the literature about extroversion preferences on interaction. For this, a different group of participants was recruited. Results shows that our proposed model generated a behavior that was more preferred by the participants than the preset behaviors. Only the group of introvert-female participants did not present any significant difference between the different behaviors.

  • Ferland, F., Cruz, A., Tapus, A., (2015) « Adapting an hybrid behavior-based architecture with episodic memory to different humanoid robots » In Proceedings of RO-MAN 2015, Japan

A common goal of robot control architecture designers is to create systems that are sufficiently generic to be adapted to different robot hardware. Beyond code re-use from a software engineering standpoint, having a common architecture could lead to long-term experiments spanning multiplerobots and research groups. This paper presents a first step toward this goal with HBBA, a Hybrid Behavior-Based Architecture first developed on the IRL-1 humanoid robot and integrating an Adaptive Resonance Theory-based episodic memory (EMART). This paper presents the first step of the adaptation of this architecture to two different robots, a Meka M-1 and a NAO from Aldebaran, with a simple scenario involving learning and sharing objects’ information between both robots. The experiment shows that episodes recorded as sequences of people and objects presented to one robot can be recalled in the future on either robot, enabling event anticipation and sharing of past experiences.

  • Cruz, A., Ferland, F., and Tapus, A. (2015) « Social Facilitation in a Game-Like Human-Robot Interaction using Synthesized Emotions and Episodic Memory » In Proceedings of International Conference on Social Robotics (ICSR 2015), Paris, France

Companion robots will be more and more part of our daily lives in the next years, and having long-term interactions can have both positive and negative effects on their users. This paper presents an experiment that is focusing on social facilitation. Our scenario is a memory game with the Nao robot and is combining an emotional system based on the OCC Model, and an episodic memory mechanism. Our first preliminary results show evidence that support the theory and present a first step towards an adaptive lifelong learning system.


4.1. Proceedings de workshops et de conférences

  • Nestor Bohorquez, Pierre-Brice Wieber, “Adaptive Step Duration in Biped Walking: A
    Robust Approach to Nonlinear Constraints”, Humanoids 2017

When a biped robot is walking in a crowd, being able to adapt the duration of steps is a key element to avoid collisions. Model Predictive Control (MPC) schemes for biped walking usually assume a fixed step duration since adapting it leads to a nonlinear problem, in general. Nonlinear solvers do not guarantee the satisfaction of nonlinear constraints at every iterate and this can be problematic for the real-time operation of robots. We propose a method to make sure that all iterates satisfy the nonlinear constraints by borrowing concepts from robust control: we make the problem robust to nonlinearities within some bounds. These bounds are linear with respect to the variables of the problem and can be adapted online.

  • Matteo Ciocca, Thierry Fraichard, Pierre-Brice Wieber, “Strong Recursive Feasibility in Model Predictive Control of Biped Walking”, Humanoids 2017

Realizing a stable walking motion requires satisfying a set of constraints. Model Predictive Control (MPC) is one of few suitable methods to handle such constraints. The capacity to satisfy constraints, which is usually called feasibility, is classically guaranteed recursively. In our applications, an important aspect is that the MPC scheme has to adapt continuously to the dynamic environment of the robot (e.g. collision avoidance, physical interaction). We aim therefore at guaranteeing recursive feasibility for all possible scenarios, which is called strong recursive feasibility. Recursive feasibility is classically obtained by introducing a terminal constraint at the end of the prediction horizon. Between two standard approaches for legged robot, in our applications we favor a capturable terminal constraint. When the robot is not really planning to stop and considers actually making a new step, recursive feasibility is not guaranteed anymore. We demonstrate numerically that recursive feasibility is actually guaranteed, even when a new step is added in the prediction horizon.

  • Nahuel A. Villa, Pierre-Brice Wieber, “Model Predictive Control of Biped Walking with Bounded Uncertainties”, Humanoids 2017

A biped walking controller for humanoid robots has to handle together hard constraints, dynamic environments, and uncertainties. Model Predictive Control (MPC) is a suitable and widely used control method to handle the first two issues. Uncertainties on the robot imply a non-zero tracking error when trying to follow a reference motion. A standard solution for this issue is to use tighter constraints by introducing some hand tuned safety margins, for the reference motion generation to ensure that the actual robot motion will satisfy all constraints even in presence of the tracking error. In this article, we find bounds for the tracking error and we show how such safety margins can be precisely computed from the tracking error bounds. Also, a tracking control gain is proposed to reduce the restrictiveness introduced with the safety margins. MPC with these considerations ensure the correct operation of the biped robot under a given degree of uncertainties when it is implemented in open-loop. Nevertheless, the straightforward way to implement an MPC closed- loop scheme fails. We discuss the reasons for this failure and propose a robust closed-loop MPC scheme.

  • Stanislas Brossette, Pierre-Brice Wieber, “Collision Avoidance Based on Separating Planes for Feet Trajectory Generation”, Humanoids 2017

In this paper, we present a formulation of the collision avoidance constraints that relies on the use of separating planes instead of a distance function. This formulation has the advantage of being defined and continuously differentiable in every situation. Because it introduces additional variables to the optimization problems, making it bigger and potentially slowing down its resolution, we propose a different resolution method that takes advantage of the independence of the variables, to form two subproblems that can be solved efficiently in an alternate problem fashion. We present some preliminary results using this approach in order to highlight its potential and promises in terms of convergence speed and robustness.

  • G. Claudio, F. Spindler, F. Chaumette, “Vision-based manipulation with the humanoid robot Romeo”. IEEE-RAS International Conference on Humanoid Robotics, Humanoids 2016, Cancun, Mexico, November 2016.

The aim of this paper is to show how visual servoing can help a humanoid robot to realize manipulation tasks with one and two hands, in conjunction with a gaze control. In addition, the use of vision in a closed-loop control scheme allows to accomplish these tasks with high repeatability without an accurate calibration of the robot kinematic model, the intrinsic and extrinsic parameters of the camera. The first application shown is grasping: one arm is controlled in order to reach the desired pose for a successful grasp along with a gaze control that keeps hand and object in the camera field of view. This approach is extended to the grasping of cylindrical objects, ignoring in the control the orientation along the revolution axis, that could cause an unnecessary motion of the arm. Moreover, we show how to control both arms with a master/slave approach for a two-handed manipulation without any force control. The robot is then able to solve a ball-in- maze game in augmented reality manipulating a tray with two hands, just using vision. A secondary task to avoid the joint limits using a large projection operator is added to improve the reliability of the system. Finally, the framework is implemented and evaluated on the humanoid robot Romeo using the ViSP library. The source code of the libraries, demos and examples are completely accessible on GitHub, allowing an easy adaptation of the visual servoing framework to other robots.

  • S. Raj Bista, P. Robuffo Giordano, F. Chaumette. “Appearance-based Indoor Navigation by IBVS using Mutual Information” IEEE Int. Conf. on Control, Automation, Robotics and Vision, ICARCV 2016, Phuket, Thailand, November 2016.

This paper presents a method for image-based navigation from an image memory using line segments as landmarks. The entire navigation process is based on 2D image information without using any 3D information at all. The environment is represented by a set of reference images with overlapping landmarks, which are acquired during a prior learning phase. These reference images define the path to follow during the navigation. The switching of reference images is done exploiting the line segment matching between the current acquired image and nearby reference images. Three view matching result is used to compute the rotational velocity of a mobile robot during its navigation by visual servoing. Real-time navigation has been validated inside a corridor and inside a room with a Pioneer 3DX equipped with an on-board camera. The obtained results confirm the viability of our approach, and verify that accurate mapping and localization are not necessary for a useful indoor navigation as well as that line segments are better features in the structured indoor environment.

  • S. Raj Bista, P. Robuffo Giordano, F. Chaumette. “Combining Line Segments and Points for Appearance-based Indoor Navigation by Image Based Visual Servoing” IROS 2017

This paper presents a complete framework for image-based navigation from an image memory that exploits mutual information and does not need any feature extraction, matching or any 3D information. The navigation path is represented by a set of automatically selected key images obtained during a prior learning phase. The shared information (entropy) between the current acquired image and nearby key images is exploited to switch key images during navigation. Based on the key images and the current image, a control law is used to compute the rotational velocity of a mobile robot during its qualitative visual navigation. Using our approach, realtime navigation has been performed inside a corridor and inside a room with a Pioneer 3-DX equipped with an on-board perspective camera without the need of accurate mapping and localization.

4.2. Journaux

  • D.J. Agravante, G. Claudio, F. Spindler, F. Chaumette. “Visual servoing in an optimization framework for the whole-body control of humanoid robots”. IEEE Robotics and Automation Letters, (Also presented at ICRA’39;2017), 2(2):608-615, April 2017.

In this paper, we show that visual servoing can be formulated as an acceleration-resolved, quadratic optimization problem. This allows us to handle visual constraints, such as field of view and occlusion avoidance, as inequalities. Furthermore, it allows us to easily integrate visual servoing tasks into existing whole-body control frameworks for humanoid robots, which simplifies prioritization and requires only a posture task as a regularization term. Finally, we show this method working on simulations with HRP-4 and real tests on Romeo.

  • S.R. Bista, P. Robuffo Giordano, F. Chaumette. “Appearance-based Indoor Navigation by IBVS using Line Segments” IEEE Robotics and Automation Letters, (also presented at ICRA’39;16), 1(1):423-430, January 2016.

See abstract above.


5.1. Proceedings de workshops et de conférences

  • Mélanie Garcia, Lucile Bechade, Guillaume Dubuisson-Duplessis, Gabrielle Pittaro, Laurence Devillers. “Towards metrics of Evaluation of Pepper robot as a Social Companion for Elderly People” International Workshop on Spoken Dialog System Technology 20187 Pittsburg, Mai 2017

For the design of socially acceptable robots, field studies in Human-Robot Interaction are
necessary. Constructing dialogue benchmarks can have a meaning only if researchers take into account the evaluation of robot, human, and their interaction. This paper describes a study aiming at finding an objective evaluation procedure of the dialogue with a social robot. The goal is to build an empathic robot (JOKER project) and it focuses on elderly people, the end-users expected by ROMEO2 project. The authors carried out three experimental sessions. The first time, the robot was NAO, and it was with a Wizard of Oz (emotions were entered manually by experimenters as inputs to the program). The other times, the robot was Pepper, and it was totally autonomous (automatic detection of emotions and decision according to). Each interaction involved various scenarios dealing with emotion recognition, humor, negotiation and cultural quiz. The paper details the system functioning, the scenarios and the evaluation of the experiments.

  • Lucile Bechade and Laurence Devillers. “Detection of Humor Reactions from Emotional and Paralinguistic Clues in Social Human-Robot Interaction” Workshop on Designing Humour in Human-Computer Interaction (HUMIC 2017). Mumbai, Inde, Septembre 2017

This paper addresses the issue of automatically interpreting the participant reactions to the robot’s humorous utterances. Through the examination of a corpus, participants reactions to a joking robot have been explored by using emotional and prodic cues which consist in durations and type of basic emotions, duration of speech and reaction time in the participants speech. We assume that the participant emotional behavior and speech behavior can be classified to extract automatically rules defining appreciation or not. This study relies on 45 human-robot interactional dialogues and 246 human humor reactions to the joking robot. The human humor reactions, elicited through puns, riddles and canned jokes, were annotated. An Learning Classifier System (LCS) system is used to
extract the most accurate behaviors from data. A new experiment have been carried out to test the results from the LCS classifier. The system recognize well negative appreciation (86%) whereas positive appreciation is more difficult to identify (54% of good recognition).

  • Lucile Bechade, Guillaume Dubuisson Duplessis, and Laurence Devillers. “Humor Support in Social Human-Robot Interaction” HCII 2016 : International Conference on Human-Computer Interaction. Toronto, Canada, Juillet 2016

Humor support is defined as the conversational strategies used in reaction to humor utterances. This paper aims at exploring the phenomenon of responses to humor interventions from the robot through the examination of a corpus. We assume that using humor in human-robot interaction sets up a positive atmosphere in which participants are willing to contribute. This study relies on 49 human-robot interaction dialogues and 381 adjacency pairs of humorous acts made by the robot and the following human responses. The human humor responses, elicited through canned jokes and conversational humor, were annotated. Three main categories of human responses were found (1) providing no support, (2) recognizing the attempt of humor and (3) contributing with more humor. The findings indicate that, as in human-human interaction, strategies of humor support are strongly dependent of the humorous event’s context.

  • Lucile Béchade, Agnes Delaborde, Guillaume Dubuisson Duplessis, Laurence Devillers, Ethical Considerations and Feedback from Social Human-Robot Interaction with Elderly People, ETHI-CA2 2016: ETHics In Corpus Collection, Annotation & Application Workshop Programme, 2016

Abstract Field studies in Human-Robot Interaction are essential for the design of socially acceptable robots. This paper describes a data collection carried out with twelve elderly people interacting with the Nao robot via a Wizard-of- Oz system. This interaction involves two scenarios implementable in a social robot as an affective companion in everyday life. One of the scenarios involves humour strategies while the other one involves negotiation strategies. In this paper, the authors detail the designs of the system, the scenarios and the data collection. The authors take a closer look at the opinions of the elderly collected through self-reports and through a private verbal exchange with one experimenter. These opinions include recommendations about the features of the robot (such as speech rate, volume and voice) as well as points of view about the ethical usage of affective robots.

  • Björn Schuller, Jean-Gabriel Ganascia, Laurence Devillers, Multimodal sentiment analysis in the wild: Ethical considerations on data collection, annotation, and exploitation, ETHI- CA2 2016: ETHics In Corpus Collection, Annotation & Application Workshop Programme, 2016.

Some ethical issues that arise for data collection and annotation of audio-visual and general multimodal sentiment, affect, and emotion data “in the wild” are of types that have been well explored, and there are good reasons to believe that they can be handled in routine ways. They mainly involve two areas, namely research with human participants, and protection of personal data. Some other ethical issues coming with such data such as its exploitation in real-life recognition engines and evaluation in long-term usages are, however, less explored. Here, we aim to discuss both–the more “routine” aspects as well as the white spots in the literature of the field.

  • Lucile Bechade, Guillaume Dubuisson Duplessis, Mohamed Sehili, Laurence Devillers. Behavioral and emotional spoken cues related to mental states in human-robot social interaction, Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, 347-350, ACM, Septembre 2015.

Understanding human behavioral and emotional cues occurring in interaction has become a major research interest due to the emergence of numerous applications such as in social robotics. While there is agreement across different theories that some behavioral signals are involved in communicating information, there is a lack of consensus regarding their specificity, their universality, and whether they convey emotions, affective, cognitive, mental states or all of those. Our goal in this study is to explore the relationship between behavioral and emotional cues extracted from speech (eg, laughter, speech duration, negative emotions) with different communicative information about the human participant. This study is based on a corpus of audio/video data of humorous interactions between the nao robot and 37 human participants.

  • Marie Tahon, Mohamed A Sehili, Laurence Devillers, Cross-corpus experiments on laughter and emotion detection in HRI with elderly people, International Conference on Social Robotics Social Robotics pp 633-642, 2015.

Social Signal Processing such as laughter or emotion detection is a very important issue, particularly in the field of human-robot interaction (HRI). At the moment, very few studies exist on elderly-people’s voices and social markers in real-life HRI situations. This paper presents a cross-corpus study with two realistic corpora featuring elderly people (ROMEO2 and ARMEN) and two corpora collected in laboratory conditions with young adults (JEMO and OFFICE). The goal of this experiment is to assess how good data from one given corpus can be used as a training set for another corpus, with a specific focus on elderly people voices. First, clear differences between elderly people real-life data and young adults laboratory data are shown on acoustic feature distributions (such as F_0 standard deviation or local jitter).

  • Fan Yang, Mohamed A Sehili, Claude Barras, Laurence Devillers. Smile and laughter detection for elderly people-robot interaction, International Conference on Social Robotics, 694-703, 2015.

Affect bursts play an important role in non-verbal social interaction. Laughter and smile are some of the most important social markers in human-robot social interaction. Not only do they contain affective information, they also may reveal the user’s communication strategy. In the context of human robot interaction, an automatic laughter and smile detection system may thus help the robot to adapt its behavior to a given user’s profile by adopting a more relevant communication scheme. While many interesting works on laughter and smile detection have been done, only few of them focused on elderly people. Elderly people data are relatively rare and often carry a significant challenge to a laughter and smile detection system due to face wrinkles and an often lower voice quality. In this paper, we address laughter and smile detection in the ROMEO2 corpus, a multimodal (audio and video).

  • Guillaume Dubuisson Duplessis, Laurence Devillers. Towards the Consideration of Dialogue Activities in Engagement Measures for Human-Robot Social Interaction, International Conference on Intelligent Robots and Systems, 19-24, 2015.

This paper addresses the problem of evaluating Human-Robot spoken interactions in a social context by considering the engagement of the human participant. We present an activity model for Human-Robot social dialogue, and show that it offers a convenient local interpretation context to assess the human participation in the activity. We describe an engagement score based on this model, and discuss results obtained by its application on a corpus of entertaining interactions involving the Nao robot in a cafeteria. We show how this approach makes it possible to discern three groups of participants in terms of engagement.

5.2. Journaux

  • Marie Tahon, Laurence Devillers. Towards a small set of robust acoustic features for emotion recognition: challenges, IEEE/ACM transactions on audio, speech, and language processing, vol 24, 1, 16-28, 2016.

The search of a small acoustic feature set for emotion recognition faces three main challenges. Such a feature set must be robust to large diversity of contexts in real-life applications; model parameters must also be optimized for reduced subsets; finally, the result of feature selection must be evaluated in cross-corpus condition. The goal of the present study is to select a consensual set of acoustic features for valence recognition using classification and non-classification based feature ranking and cross-corpus experiments, and to optimize emotional models simultaneously. Five realistic corpora are used in this study: three of them were collected in the framework of the French project on robotics ROMEO, one is a game corpus (JEMO) and one is the well-known AIBO corpus.

  • Florian Eyben, Klaus R Scherer, Björn W Schuller, Johan Sundberg, Elisabeth André, Carlos Busso, Laurence Y Devillers, Julien Epps, Petri Laukka, Shrikanth S Narayanan, Khiet P Truong, The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing, in IEEE Transactions on Affective Computing, 2016

Work on voice sciences over recent decades has led to a proliferation of acoustic parameters that are used quite selectively and are not always extracted in a similar fashion. With many independent teams working in different research areas, shared standards become an essential safeguard to ensure compliance with state-of- the-art methods allowing appropriate comparison of results across studies and potential integration and combination of extraction and recognition systems. In this paper we propose a basic standard acoustic parameter set for various areas of automatic voice analysis, such as paralinguistic or clinical speech analysis. In contrast to a large brute-force parameter set, we present a minimalistic set of voice parameters here. These were selected based on a) their potential to index affective physiological changes in voice production, b) their proven value in former.

  • Alexei Grinbaum, Raja Chatila, Laurence Devillers, Jean-Gabriel Ganascia, Catherine Tessier, M Dauchet, Ethics in Robotics Research: CERNA Mission and Context, IEEE Robotics & Automation Magazine 24 (3), 139-145, IEEE Robotics & Automation Magazine, 24, 3, 139-145, 2017

This article summarizes the recommendations concerning robotics as issued by the Commission for the Ethics of Research in Information Sciences and Technologies (CERNA), the French advisory commission for the ethics of information and communication technology (ICT) research. Robotics has numerous applications in which its role can be overwhelming and may lead to unexpected consequences. In this rapidly evolving technological environment, CERNA does not set novel ethical standards but seeks to make ethical deliberation inseparable from scientific activity. Additionally, it provides tools and guidance for researchers and research institutions.

5.3. Chapitres de livre

  • Laurence Devillers, Guillaume Dubuisson Duplessis. (2017) Toward a Context-Based Approach to Assess Engagement in Human-Robot Social Interaction. In: Jokinen K., Wilcock G. (eds) Dialogues with Social Robots. Lecture Notes in Electrical Engineering, vol 427. Springer, Singapore.

This article addresses the issue of evaluating Human-Robot spoken interactions in a social context by considering the engagement of the human participant. Our work regards the Communication Accommodation Theory (CAT) as a promising paradigm to consider for engagement, by its study of macro- and micro-contexts influencing the behaviour of dialogue participants (DPs), and by its effort in depicting the accommodation process underlying the behaviour of DPs. We draw links between the accommodation process described in this theory and human engagement that could be fruitfully used to evaluate Human-Robot social interactions. We present our goal which is to take into account a model of dialogue activities providing a convenient local interpretation context to assess human contributions (involving verbal and nonverbal channels) along with CAT to assess Human-Robot social interaction.

  • Lucile Bechade, Guillaume Dubuisson-Duplessis, Gabrielle Pittaro, Mélanie Garcia, Laurence Devillers. “Evaluation of Pepper robot as a Social Companion for Elderly People : experiments at the French Broca hospital”, “Advanced Social Interaction with Agents”, 8th International Workshop on Spoken Dialog Systems, « Lecture Notes in Electrical Engineering » (LNEE), springer 2018 (long paper, in press)

For the design of socially acceptable robots, field studies in Human-Robot Interaction are necessary. Constructing dialogue benchmarks can have a meaning only if researchers take into account the evaluation of robot, human, and their interaction. This paper describes a study aiming at finding an objective evaluation procedure of the dialogue with a social robot. The authors carried out three experimental sessions. The first time, the robot was NAO, and it was with a Wizard of Oz (emotions were entered manually by experimenters as inputs to the program). The other times, the robot was Pepper, and it was totally autonomous (automatic detection of emotions and decision according to). Each interaction involved various scenarios dealing with emotion recognition, humor, negotiation and cultural quiz. The long paper details the system functioning, the scenarios and the evaluation of the experiments.


6.1. Proceedings de workshops et de conférences

  • Joseph Mirabel, Florent Lamiraux. Manipulation planning: addressing the crossed foliation issue. IEEE International Conference on Robotics and Automation 2017, May 2017, Singapour, Singapore. 2017

This paper deals with manipulation planning. First, we propose a new tool called the constraint graph to describe the various motion constraints relative to a manipulation planning problem. Then, we describe a problem arising for some manipulation planning problems called the crossed foliation issue. We propose a extension of RRT algorithm that explores the leaves of the foliations generated by motion constraints and that solves the crossed foliation problem. Finally, we show a wide variety of problem that our approach can solve.

  • Marion Guilhem, Mekonnen Alhayat Ali, et Lerasle Frédéric. Pertinence des combinaisons traqueur-détecteur pour le suivi par détection. Congrès francophone Reconnaissance des Formes et Intelligence Artificielle (RFIA’16), Clermont Ferrand, 2016.

Ces travaux étudient la pertinence des associations filtre- détecteur dans toute stratégie visuelle de suivi par détection. Ils privilégient notamment les méthodes de Monte Carlo et des traqueurs basés sur des banques de représentations couplés à des détecteurs visuels. Les évaluations quantifient l’influence des associations filtre, détecteur, mono versus multi représentations sur les performances globales. Nous observons, par exemple, que celles-ci sont très liées au choix du détecteur ; une différence de 1% en rappel entre détecteurs peut induire une baisse de 10% sur la précision du traqueur global.

  • Mekonnen Alhayat Ali, et Lerasle Frédéric. « Comparative evaluations of tracking-by- detection approaches ». Submission in IEEE Trans. on Circuits and Systems for Video Technology, August 2017.

In this work, we present a comparative evaluation of various multi-person tracking-by- detection approaches on public datasets. The work investigates popular sequential Monte Carlo and template ensemble based trackers coupled with relevant visual people detectors with emphasis on exhibited performance variation depending on tracker-detector choices. Our experimental results show that tracking is sensitive to the detector choice and should be done after careful evaluation. Even a 1% difference in detector recall could lead to a 10% drop in tracking accuracy.

6.2. Journaux

  • Fitte-Duval, Laurent, Mekonnen Alhayat Ali, et Lerasle Frédéric. « Combination of RGB-D Features for Head and Upper Body Orientation Classification ». In Advanced Concepts for Intelligent Vision Systems, 591‑603. Lecture Notes in Computer Science. Springer, Cham, 2016.

In Human-Robot Interaction (HRI), the intention of a person to interact with another agent (robot or human) can be inferred from his/her head and upper body orientation. Furthermore, additional information on the person’s overall intention and motion direction can be determined with the knowledge of both orientations. This work presents an exhaustive evaluation of various combinations of RGB and depth image features with different classifiers. These evaluations intend to highlight the best feature representation for the body part orientation to classify, i.e, the person’s head or upper body. Our experiments demonstrate that high classification performances can be achieved by combining only three families of RGB and depth features and using a multiclass SVM classifier.!Ai5X8SJeZOVZo2fSFxt9SVHQ-W6D


7.1. Proceedings de workshops et de conférences

● F. Flacco, A. Paolillo, A. Kheddar. “Residual-based contacts estimation for humanoid robots” IEEE-RAS 17th International Conference on Humanoid Robots , 2016

The residual method for detecting contacts is a promising approach to allow physical interaction tasks with humanoid robots. Nevertheless, the classical formula-tion, as developed for fixed-base robots, cannot be directly applied to floating-base systems. This paper presents a novel formulation of the residual based on the floating-base dynamics modeling of humanoids. This leads to the definition of the internal and external residual. The first estimates the joints effort due to the external perturbation acting on the robot. The latter is an estimation of the external forces acting on the floating- base of the robot. The potential of the method is shown proposing a simple internal residual-based reaction strategy, and a procedure for estimating the contact point that combines both the internal and external residuals

  • A. Paolillo, F. Flacco, and A. Kheddar. “The residual method for humanoid robots” 9th International Workshop on Human-Friendly Robotics, 2016

In the framework of physical human-robot interaction, the detection of contacts has a crucial importance. To this purpose, the residual method proved to be very effective for fixed-based manipulators. The question we tackle is whether it can be scaled for humanoid robots? We noticed that the formulation developed for fixed-base robots cannot be directly applied to floating-base systems. In fact, the floating- base dynamics modeling of humanoids has to be taken into account. This leads to the definition of the internal and external residuals. The first estimates the joints effort due to the external perturbation acting on the robot. The latter is an estimation of the external forces acting on the floating-base of the robot. The potential of the method is shown proposing a simple internal residual-based reaction strategy, and a procedure for estimating the contact force that uses the external residual.

  • F. Flacco, A. Kheddar. “Contact detection and physical interaction for low-cost personal robots” IEEE International Symposium on Robot and Human Interactive Communication, 2017

We present a methodology for estimating joints torque due to external forces applied to a robot with large joints backlash and friction. This undesired non-linearity is common in personal robot, due to the use of low cost mechanical components and type of usage. Our method enables contact detection and human-robot physical interaction capabilities without using extra sensors. The effectiveness of our approach is shown with experiments on a Romeo robot arm from SoftBank Robotics.

  • A. Paolillo, A, Bolotnikova, K. Chappellet, A. Kheddar. “Visual estimation of articulated objects configuration during manipulation with a humanoid” IEEE/SICE International Symposium on System Integration, 2017

Robotic manipulation tasks require on-line knowledge of the operated objects’ configurations. Thus, we need to estimate online the state of the (articulated) objects that are not equipped with positioning sensors. This estimated state w.r.t the robot control frame, is required by our controller to update the model and close the loop. Indeed, in the controller we use models of the (articulated) objects as additional ‘robots’ so that it computes the overall ‘robots-objects’ augmented system’s motion and contact interaction forces that fulfill all the limitation constraints together with the physics. Because of the uncertainties due to the floating-base nature of humanoids, we address the problem of estimating the configuration of articulated objects using a virtual visual servoing-based approach with the robot’s camera. Experimental results carried out with the humanoid robot HRP-4, manipulating the paper drawer of a printer, show the effectiveness of the approach.

  • V. Samy and K. Bouyarmane and A. Kheddar. “Active shock absorber control in humanoid robot falls with nonlinear optimization on a reduced dynamics model” IEEE International Conference on Robotics and Automation, 2018

We propose a reduced-model- based analytic approach to active shock absorption by a falling humanoid robot at impact. We model the segment between the contact point at impact and the impacting limb extremity attached to the torso or waist of the humanoid robot as a one degree-of-freedom linear mass-spring- damper system. By mapping the joint angle limits and torque limits of the joints in the impact-ing limb to a corresponding position limit and force limit in the reduced model, we formulate a nonlinear optimization problem to find an admissible pair of parameters of the reduced model (stiffness, damping) that prevents the model from violating the constraints before reaching a steady state rest. The nonlinear constraints are analytically derived using symbolic computation tools and then numerically solved with off-the- shelf nonlinear optimization solver. The reduced model trajectories are then mapped back on the full body of the humanoid robot and illustrated on the HRP-4 robot in simulation.

  • V. Samy, S. Caron, K. Bouyarmane and A. Kheddar. “Post-Impact Adaptive Compliance for Humanoid Falls Using Predictive Control of a Reduced Model” IEEE-RAS 17th International Conference on Humanoid Robots , 2017

We consider control of a humanoid robot in active compliance just after the impact consecutive to a fall. The goal of this post-impact braking is to absorb undesired linear momentum accumulated during the fall, using a limited supply of time and actuation power. The gist of our method is an optimal distribution of undesired momentum between the robot’s hand and foot contact points, followed by the parallel resolution of Linear Model Predictive Control (LMPC) at each contact. This distribution is made possible thanks to torque-limited friction polytopes, an extension of friction cones that takes actuation limits into account. Individual LMPC results are finally combined back into a feasible CoM trajectory sent to the robot's whole-body controller. We validate
the solution in full-body dynamics simulation of an HRP-4 humanoid falling on a wall.

  • V. Samy and A. Kheddar. “Falls control using posture reshaping and active compliance” IEEE-RAS International Conference on Humanoids, 2015

We address the problem of humanoid falls when they are unavoidable. We propose a control strategy that combines two behaviors: i) closed-loop posture re-shaping — during the falling phase, which allows best impact absorption from a predefined taxonomy, coupled with ii) an active compliance through instant PD gains reduction, instead of shutting-off the actuators or instead of high-gains control with additional implements as previously proposed by other works. We perform several simulations to assess our strategy and made experimental trials on the HRP-4 humanoid robot.

  • V. Samy, K. Bouyarmane, and A. Kheddar. “QP-based adaptive-gains compliance control in humanoid falls” IEEE International Conference on Robotics and Automation, 2017

We address the problem of humanoid falling with a decoupled strategy consisting of a pre-impact and a post-impact stage. In the pre-impact stage, geometrical reasoning allows the robot to choose appropriate impact points in the surrounding environment and to adopt a posture to reach them while avoiding impact-singularities and preparing for the post-impact. The surrounding environment can be unstructured and may contain cluttered obstacles. The post-impact stage uses a quadratic program controller that adapts on-line the joint proportional-derivative (PD) gains to make the robot compliant — to absorb impact and post-impact dynamics, which lowers possible damage risks. This is done by a new approach incorporating the stiffness and damping gains directly as decision variables in the QP along with the usually-considered variables of joint accelerations and contact forces. Constraints of the QP prevent the motors from reaching their torque limits during the fall. Several experiments on the humanoid robot HRP-4 in a full-dynamics simulator are presented and discussed.


8.1. Proceedings de workshops et de conférences

  • Fournier P, Sigaud O., Chetouani M. : “Combining artificial curiosity and tutor guidance for environment exploration”. In the RO-MAN workshop BAILAR, Lisbon, Portugal, August 2017.

In a new environment, an artificial agent should explore autonomously and exploit tutoring signals from human caregivers. While these two mechanisms have mainly been studied in isolation, we show in this paper that a carefully designed combination of both performs better than each separately. To this end, we propose an autonomous agent whose actions result from a user-defined weighted combination of two drives: a tendency for gaze-following behaviors in presence of a tutor, and a novelty-based intrinsic curiosity. They are both incorporated in a model-based reinforcement learning framework through reward shaping. The agent is evaluated on a discretized pick-and- place task in order to explore the effects of various combinations of both drives. Results show how a properly tuned combination leads to a faster and more consistent discovery of the task than using each drive in isolation. Additionally, experiments in a reward-free version of the environment indicate that combining curiosity and gaze-following behaviors is a promising path for real-life exploration in artificial agents.

8.2. Journaux

  • Chaby, L. and Hupont, I. and Avril, M. and Luherne-du Boullay, V. and Chetouani, M. (2017). Gaze behavior consistency among older and younger adults when looking at emotional faces. Frontiers in Psychology. Vol 8 Pages 548.

The identification of non-verbal emotional signals, and especially of facial expressions, is essential for successful social communication among humans. Previous research has reported an age-related decline in facial emotion identification, and argued for socio-emotional or aging-brain model explanations. However, more perceptual differences in the gaze strategies that accompany facial emotional processing with advancing age have been under-explored yet. In this study, 22 young (22.2 years) and 22 older (70.4 years) adults were instructed to look at basic facial expressions while their gaze movements were recorded by an eye-tracker. Participants were then asked to identify each emotion, and the unbiased hit rate was applied as performance measure. Gaze data were first analyzed using traditional measures of fixations over two preferential regions of the face
(upper and lower areas) for each emotion. Then, to better capture core gaze changes with advancing age, spatio-temporal gaze behaviors were deeper examined using data-driven analysis (dimension reduction, clustering). Results first confirmed that older adults performed worse than younger adults at identifying facial expressions, except for “joy” and “disgust,” and this was accompanied by a gaze preference toward the lower-face. Interestingly, this phenomenon was maintained during the whole time course of stimulus presentation. More importantly, trials corresponding to older adults were more tightly clustered, suggesting that the gaze behavior patterns of older adults are more consistent than those of younger adults. This study demonstrates that, confronted to emotional faces, younger and older adults do not prioritize or ignore the same facial areas. Older adults mainly adopted a focused-gaze strategy, consisting in focusing only on the lower part of the face throughout the whole stimuli display time. This consistency may constitute a robust and distinctive “social signature” of emotional identification in aging. Younger adults, however, were more dispersed in terms of gaze behavior and used a more exploratory-gaze strategy, consisting in repeatedly visiting both facial areas.

9. Participation à des workshops et des tables rondes

9.1. Workshops

  • 07/2015, “Development of Socially Intelligent Robots and the need of Learning: an industrial perspective and use cases”, in the workshop of Bottom-up and top-down development of robot skills, International Conference on Advanced Robotics (ICAR), (Amit Kumar Pandey, Aldebaran)
  • 31/07/2016 : « Social acceptability, the importance of getting robots right » by Rodolphe Gelin, Workshop on « Affective Computing for Social Robots » Rodolphe Gelin, conférence ROMAN 2016, New York
  • 14/09/2016 : Intervention de Rodolphe Gelin lors du workshop « Designing for Interaction in social robotics » conférence Clawar 2016, Londres
  • 16/10/2016 : « Audio Visual Interaction with a Companion Robot » par Rodolphe Gelin, Workshop « Computer Vision for Audio-Visual Media »conférence ECCV 2016, Amsterdam.

9.2. Invitations, présentations, keynotes

  • 09/2015, Presentation at the International Symposium on Robotics Research (ISRR 2015) by Darwin Lau (ISIR)
  • 19/10/2015, keynote de Rodolphe Gelin « Need of intelligent sensing for domestic robots » à la conférence IEEE International Symposium on Robotics and Intelligent Sensors, IRIS 2015, Langkawi, Malaisie
  • 23/10/2015, keynote de Rodolphe Gelin « Ethics for domestic robots » à l’occasion de 1st International Conference on Ethics & Robotics (ICRE 2015), Lisbonne, Portugal.
  • 09/11/2015, présentation de Rodolphe Gelin « Le langage corporel des robots humanoïdes » à l’occasion de la journée Gestualité, Musée des Arts et Métiers, Paris, France
  • 20/11/2015, présentation de Rodolphe Gelin « Interacting with humanoid robots » à l’occasion du PhD Day organisé par l’Institut de Neurobiologie de la Méditerranée, Luminy, France.
  • 20/05/2016 : Keynote « Éthique de la robotique au quotidien » de Rodolphe Gelin à l’occasion du Séminaire La Robotique Interactive à Paris-Saclay- E3-Robotic : Expérimentation/Evaluation/Ethique
  • 18/10/2016 : « Assistive humanoid robots for elderly people » par Rodolphe Gelin, Forum Cognition and Longevity, Paris
  • 08/2017: Chetouani (ISIR); Keynote Speaker at RO-MAN 2017 Workshop: Towards Intelligent Social Robots: Social Cognitive Systems in Smart Environments.

9.3. Contributions à des universités d’été ou d’hiver

  • 08/2015, Lecture on “Socially Intelligent Human-Robot Interaction – Applications and Needs from a commercial perspective”, 2nd International Summer School on Social Human-Robot Interaction, Åland, Finland. (Amit Kumar Pandey, Aldebaran)

10. Thèses défendues

10.1. INRIA

  • Thèse de Nicolas Cazy, Commande prédictive pour la réalisation de tâches d’asservissement visuel successives, soutenue à Rennes le 29 novembre 2016

On rencontre aujourd’hui la vision par ordinateur employée pour la réalisation de nombreuses applications de la robotique moderne. L’un des axes de recherche actuel qui tend à améliorer ces systèmes est centré sur la commande. L’objectif est de proposer des schémas de commande originaux permettant de lier efficacement les informations mesurées par les capteurs de vision aux actions que l’on souhaite réaliser avec les robots. C’est dans cet aspect que s’inscrit ce document en apportant de nouvelles méthodes à la commande robotique classique faisant intervenir la vision, l’asservissement visuel. Le cas de pertes d’informations visuelles pendant la réalisation d’une tâche d’asservissement visuel est étudié. Dans ce sens, deux méthodes de prédiction qui permettent à la tâche d’être réalisée malgré ces pertes sont présentées. Puis une méthode inédite de correction est proposée. Celle-ci permet d’obtenir de meilleurs résultats de prédiction qu’une méthode classique, comme le démontrent des résultats obtenus en simulation et en condition réelle. Enfin, dans le contexte de la réalisation de plusieurs tâches d’asservissement visuel successives, une nouvelle méthode est présentée. Celle-ci exploite les caractéristiques d’un schéma de commande utilisé depuis quelques dizaines d’années dans l’industrie et la recherche, la commande prédictive basée modèle. Des résultats obtenus en simulation proposent de visualiser les effets de cette méthode sur le comportement d’un drone qui embarque une caméra.

  • Thèse de Suman Raj Bista, Indoor Navigation of Mobile Robots based on Visual Memory and Image-Based Visual Servoing, soutenue à Rennes le 20 décembre 2016

Cette thèse présente une méthode de navigation par asservissement visuel à l’aide d’une mémoire d’images. Le processus de navigation est issu d’informations d’images 2D sans utiliser aucune connaissance 3D. L’environnement est représenté par un ensemble d’images de référence avec chevauchements, qui sont automatiquement sélectionnés au cours d’une phase d’apprentissage préalable. Ces images de référence définissent le chemin à suivre au cours de la navigation. La commutation des images de référence au cours de la navigation est faite en comparant l’image acquise avec les images de référence à proximité. Basé sur les images actuelles et deux images de référence suivantes, la vitesse de rotation d’un robot mobile est calculée en vertu d’une loi du commandé par asservissement visuel basé image. Tout d’abord, nous avons utilisé l’image entière comme caractéristique, où l’information mutuelle entre les images de référence et la vue actuelle est exploitée. Ensuite, nous avons utilisé des segments de droite pour la navigation en intérieur, où nous avons montré que ces segments sont de meilleurs caractéristiques en environnement intérieur structuré. Enfin, nous avons combiné les segments de droite avec des points pour augmenter l’application de la méthode à une large gamme de scénarios d’intérieur pour des mouvements sans heurt. La navigation en temps réel avec un robot mobile équipé d’une caméra perspective embarquée a été réalisée. Les résultats obtenus confirment la viabilité de notre approche et vérifient qu’une cartographie et une localisation précise ne sont pas nécessaires pour une navigation intérieure utile.

10.2. LAAS

  • Thèse de Joseph Mirabel. Manipulation planning for documented objects. Robotics [cs.RO]. Institut National Polytechnique De Toulouse, soutenue le 21 février 2017

This thesis tackles the manipulation planning for documented objects. The difficulty of the problem is the coupling of a symbolic and a geometrical problem. Classical approaches combine task and motion planning. They are hard to implement and time consuming. This approach is different on three aspects. The first aspect is a theoretical framework to model admissible motions of the robot and objects. This model uses constraints to link symbolic task and motions achieving such task. A graph of constraint models the manipulation rules. A planning algorithm using this graph is proposed. The second aspect is the handling of constrained motion. In manipulation planning, an abstract definition of numerical constraint is necessary. A continuity criterion for Newton-Raphson methods is proposed to ensure the continuity of trajectories in sub-manifolds. The last aspect is object documentation. Some information, easy to define for human beings, greatly speeds up the search. This documentation, specific to each object and end-effector, is used to generate a graph of constraint, easing the problem specification and resolution.


  • Thèse de Fan Yang, Détection de marqueurs affectifs et attentionnels de personnes âgées en interaction avec un robot, école doctorale en Informatique, Paris-Saclay, 23 octobre 2015

Ces travaux de thèse portent sur la détection audio-visuelle de marqueurs affectifs (rire et sourire) et attentionnels de personnes âgées en interaction sociale avec un robot. Pour comprendre efficacement et modéliser le comportement des personnes très âgées en présence d’un robot, des données pertinentes sont nécessaires. J’ai participé à la collection d’un corpus de personnes âgées notamment pour l’enregistrement des données visuelles. Le système utilisé pour contrôler le robot est un magicien d’Oz, plusieurs scénarios de conversation au quotidien ont été utilisés pour encourager les gens à coopérer avec le robot. Ces scénarios ont été élaborés dans le cadre du projet ROMEO2 avec l’association Approche. Nous avons décrit tout d’abord le corpus recueilli qui contient 27 sujets de 85 ans en moyenne pour une durée totale de 9 heures, les annotations et nous avons discuté des résultats obtenus à partir de l’analyse des annotations et de deux questionnaires. Ma recherche se focalise ensuite sur la détection de l’attention et la détection de rire et de sourire. Les motivations pour la détection de l’attention consistent à détecter quand le sujet ne s’adresse pas au robot et à adapter le comportement du robot à la situation. Après avoir considéré les difficultés liées aux personnes âgées et les résultats d’analyse obtenus par l’étude des annotations du corpus, nous nous intéressons à la rotation de la tête au niveau de l’indice visuel et à l’énergie et la qualité de voix pour la détection du destinataire de la parole. La détection de rire et sourire peut être utilisée pour l’étude sur le profil du locuteur et de ses émotions. Mes intérêts se concentrent sur la détection de rire et sourire dans la modalité visuelle et la fusion des informations audio-visuelles afin d’améliorer la performance du système automatique. Les expressions sont différentes des expressions actées ou posés à la fois en apparence et en temps de réaction. La conception d’un système qui marche sur les données réalistes des personnes âgées est encore plus difficile à cause de plusieurs difficultés à envisager telles que le manque de données pour l’entrainement du modèle statistique, l’influence de la texture faciale et de la façon de sourire pour la détection visuelle, l’influence de la qualité vocale pour la détection auditive, la variété du temps de réaction, le niveau de compréhension auditive, la perte de la vue des personnes âgées, etc. Les systèmes de détection de la rotation de la tête, de la détection de l’attention et de la détection de rire et sourire sont évalués sur le corpus ROMEO2 et partiellement évalués (détections visuelles) sur les corpus standard Pointing04 et GENKI-4K pour comparer avec les scores des méthodes de l’état de l’art. Nous avons également trouvé une corrélation négative entre la performance de détection de rire et sourire et le nombre d’évènement de rire et sourire pour le système visuel et le système audio-visuel. Ce phénomène peut être expliqué par le fait que les personnes âgées qui sont plus intéressées par l’expérimentation rient plus souvent et sont plus à l’aise donc avec des poses variées. La variété des poses et le manque de données correspondantes amènent des difficultés pour la reconnaissance de rire et de sourire pour les systèmes statistiques.Les expérimentations montrent que la rotation de la tête peut être efficacement utilisée pour détecter la perte de l’attention du sujet dans l’interaction avec le robot. Au niveau de la détection de l’attention, le potentiel d’une méthode en cascade qui utilise les modalités d’une manière complémentaire est montré. Cette méthode donne de meilleurs résultats que le système auditif seul. Pour la détection de rire et sourire, en suivant le même protocole « Leave-one- out », la fusion des deux systèmes monomodaux améliore aussi significativement la performance par rapport à un système monomodal au niveau de l’évaluation segmentale.