Robot Foundation Models Based Behavior Improvement Using Formal Data and Dialogue 6-month internship @ CEA List Internship context Based in Saclay (Essonne), the LIST is one of the two institutes of CEA Tech, the Technological Research Division of the CEA. Dedicated to intelligent digital systems, its mission is to carry out technological developments of excellence on behalf of industrial partners, in order to create value. Within the LIST, the Laboratory of Textual and Visual Semantic Analysis (LASTI) conducts its research in the field of natural language processing and computer vision to extract, classify and generate information. The laboratory's research themes include learning with few data, trustworthiness and multimodality. Missions The proposed internship focuses on enhancing robot behaviour through the integration of formal data, aiming to improve scene comprehension and robotic planning. In an era where robots are increasingly becoming integral to various industries, the ability to understand complex scenes and plan actions effectively is paramount. Firstly, the internship will concentrate on the analysis and integration of formal data to enrich models during inference. Formal data, which includes structured information and predefined knowledge bases, can provide robots with a solid foundation of understanding. By injecting this data in their VLA, robots can better interpret sensory inputs and make more informed decisions. For instance, if a robot is tasked with navigating a warehouse, formal data about the layout and inventory can help it plan more efficient routes and avoid obstacles. This process will involve developing algorithms that can seamlessly integrate formal data into existing robotic models, enhancing their inferential capabilities. Secondly, the construction of improved representations through data and dialogue is essential. Robots often operate in dynamic environments where static knowledge is insufficient. By dialogue, they can acquire real-time information and adapt their representations. This interaction will use the CoVA Virtual Assistant already integrated in our LLM based robotic system. Generating action plans is another critical component of this internship. Once robots have a comprehensive understanding of their environment, they need to translate this knowledge into actionable plans. This involves developing planning algorithms that can consider multiple factors, such as resource availability and time constraints. This part will be made in collaboration with colleagues at CEA LIST working on planning with critics and those working on VLA models. To evaluate his or her results, the candidate will leverage benchmarks from the community.The development of generative AI models based on large generative language models (LLMs) has led to significant advances in zero or few-shot approaches for a range of natural language processing Qualifications - Students in their 4th or 5th year of studies (M1, M2 or gap year) - Knowledge in Natural Language Processing - Machine learning skills (deep learning, perception models, generative AI...) - Python and Linux proficiency - Basic knowledge of databases Job-related benefits Joining the CEA List and the LASTI as an intern means: - Working in one of the most innovative research organizations in the world, addressing societal challenges to build the world of tomorrow. - Discovering a rich ecosystem: privileged connections between the industrial and academic sectors. - Conducting research autonomously and creatively: encouragement to disseminate and showcase results (scientific articles, patents, open-source codes...). - Benefit from an internal computing infrastructure with more than 300 state-of-the-art GPUs - Receive a stipend between ¤1300 and ¤1400 per month - Have the opportunity to continue with a PhD or as a research engineer after the internship. - Have the possibility of remote work, receive a 75% reimbursement on public transportation costs, and benefit from the "mobili-jeune" aid to reduce rent costs... To apply, please send your CV, a cover letter, and the title of the internship to: lastirecrute@cea.fr If you are interested in more than one internship, please indicate your order of preference.