PhD in Humanoid Robotics - Safe Renforcement Learning (M/F)

Descrizione

PhD in Humanoid Robotics - Safe Renforcement Learning (M/F) Sintesi in italiano (traduzione automatica): Il progetto di dottorato proposto si inserisce nel programma di ricerca HAMMER, gestito dall'Agence Nationale de la Recherche (ANR) in Francia. La ricerca si concentra sullo sviluppo di un'architettura ibrida per la locomozione dinamica dei robot, combinando apprendimento per rinforzo e controllo predittivo. Il candidato lavorerà con un team esperto del LAAS-CNRS, contribuendo a garantire la sicurezza e la coerenza dinamica dei movimenti dei robot. È richiesta una laurea in ingegneria informatica, ingegneria robotica o un campo correlato. Le mansioni principali includono la progettazione di algoritmi di controllo e l'implementazione di tecniche di apprendimento automatico per la locomozione dei robot in ambienti complessi. This thesis is proposed within HAMMER, Target Project 2 of the priority research and acceleration-equipment program in robotics funded by France 2030 and managed by the Agence Nationale de la Recherche (ANR). HAMMER is a large collaborative project led by CNRS–Université Côte d'Azur I3S, gathering CNRS-AIST JRL, LAAS-CNRS, CNRS-UBE ICB, ONERA DTIS, CNRS-INP GIPSA-lab, CNRS-Sorbonne Université ISIR, Inria-ENS Willow, Inria ACENTAURI, Inria Larsen, and Mines Paris PSL. The project aims to endow robots with advanced locomotion and mobility capabilities by developing a hybrid approach that combines the complementary strengths of model-based and data-driven methods. Within this context, the LAAS-CNRS Gepetto team brings a long-standing expertise in model-based whole-body control for humanoid robots (Pinocchio, Crocoddyl, Stack-of-Tasks, TALOS) and, more recently, in reinforcement-learning-based locomotion, notably through the Constraints as Terminations (CaT) framework for constrained legged locomotion RL. This thesis builds directly on that lineage and on recent work at CNRS-AIST JRL that filters RL policy outputs through a second-order Quadratic Program (QP) safety layer to guarantee constraint satisfaction at execution time (Cariou, Muraccioli et al., accepted at IROS 2026). The proposed subject extends this line of research from single-step QP safety filtering toward a fully hybrid, horizon-based architecture in which RL and Model Predictive Control (MPC) are tightly coupled. 2. Thesis Statement This thesis aims to develop a hybrid dynamic locomotion architecture in which a reinforcement-learning (RL) policy makes discrete, high-level decisions — such as contact mode, footstep or gait-pattern selection, and behavior transitions — while an online-formulated and online-parametrized Model Predictive Controller enforces the robot's physical constraints and produces the continuous, whole-body motion. The MPC guarantees feasibility, safety, and dynamic consistency, while the RL component provides adaptation, anticipation, and strategic decision-making in complex, uncertain, or previously unseen situations. 3. State of the Art Legged and humanoid locomotion over unstructured or dynamic terrain requires two qualitatively different kinds of reasoning. On one hand, choosing where and how to make or break contact (which foot, which surface, which timing, which gait) is fundamentally a combinatorial, discrete decision problem that is difficult to encode as a smooth cost or convex constraint. On the other hand, once a contact plan or behavior mode is chosen, generating a dynamically consistent, torque- and constraint-feasible trajectory is a continuous optimization problem for which model-based whole-body control and Model Predictive Control (MPC) are mature and well understood. This section reviews the relevant state of the art along five axes: model-based whole-body and predictive control, contact-implicit and hybrid MPC, safe reinforcement learning and execution-time safety filters, approaches that couple RL and MPC, and learned discrete mode selection for hybrid legged locomotion. 3.1 Model-based whole-body control and predictive control for legged robots Quadratic-Program-based whole-body control has become the standard formulation for prioritized task execution under joint, torque, and contact constraints on legged and humanoid robots [1]–[3], with Hierarchical QP providing strict task prioritization for fast online motion generation [4]. A comprehensive recent survey consolidates the numerical algorithms and modeling choices behind optimization-based control for dynamic legged robots, including centroidal, whole-body, and contact-implicit formulations [5]. These model-based approaches provide strong constraint-satisfaction guarantees and generalize well across tasks, but they depend on accurate dynamic models and require an externally supplied contact plan or sequence, which is precisely the combinatorial decision this thesis propose Annuncio in inglese. Fonte: Euraxess (Commissione europea).