[Research Internship] - Crafting NPCs & Bots behaviors with LLM/VLM - (F/M/NB)

Company Description

Ubisoft is a global leader in gaming with teams across the world creating original and memorable gaming experiences, from Assassin’s Creed, RainbowSix to Just Dance and more. We believe diverse perspectives help both players and teams thrive. If you’re passionate about innovation and pushing entertainment boundaries, join our journey and help us create the unknown!

Ubisoft Bordeaux

Founded in 2017 by a handful of Ubisoft experts, our studio now counts over 400 talents from 20 different nationalities. Our teams contribute to some of Ubisoft’s biggest franchises: leading the production of Assassin’s Creed Mirage in 2023, working on Assassin’s Creed Shadows and the Claws of Awaji expansion, as well as on Beyond Good & Evil 2.

Beyond our AAA productions, we also drive technological innovation within the group through our expertise in the Anvil game engine and Ubisoft’s Online Services. We have also established a dedicated R&D division, La Forge, which brings together Ubisoft engineers and academic researchers to collaborate on prototypes that support game development, particularly in the field of AI.

In 2024, we inaugurated our new offices in the Bastide-Niel eco-district, designed to foster the development of Ubisoft’s best gaming experiences and technologies in an exceptional, accessible, and central environment.

Job Description

Recent advancements in Large Language Models (LLM) and Visual Language Models (VLM) are revolutionizing many fields, including video games. Multiple video-game companies are already studying how to leverage LLMs for game-changing innovations, such as enabling players to cooperate in natural language with NPCs (non-player characters). Within academia, multiple research labs are studying how to adapt LLMs/VLMs to new use cases, such as interacting in complex environments through reinforcement learning or evolutionary strategies. These breakthroughs open up vast opportunities for enhancing player immersion through more intelligent and responsive NPCs. 

This internship aims to explore the integration of LLMs and VLMs with reinforcement learning (RL) or evolutionary strategies to enhance the intelligence and behavior of NPCs and bots in video games. By combining the creative potential of LLMs with the interactive feedback loops of RL, we aim to build better NPCs and reduce iteration time for game developers. 

The internship will take place within Ubisoft’s La Forge, where cutting-edge tools and resources are available for research, including a set of realistic 3D games specifically designed for research purposes, datasets, and state-of-the-art learning algorithms deployed on GPU clusters. This internship will benefit from close collaboration with the Inria Flowers team, led by Pierre-Yves Oudeyer. 

Objectives: 

• Investigate the current uses of LLMs/VLMs in gaming, focusing on how they enhance NPC decision-making. 
• Study recent research on using reinforcement learning to improve the performance of LLM/VLMs, with a particular focus on grounding these models in interactive environments such as video games. 
• Throughout discussion with production teams, identify a potential real-world use case on which to calibrate prototyping and research. 
• Experiment with methods to optimize NPC learning in simulated or live gaming environments, providing a balance between realism and computational efficiency. 

Qualifications

• You are in the final year of your engineering degree or pursuing a research master’s in computer science or a related field. 
• You possess a solid foundation in computer science and mathematics, particularly in areas relevant to machine learning, deep learning, LLMs/VLMs, or reinforcement learning. 
• You have experience using Python and machine learning libraries to build and experiment with models. 
• Your English proficiency enables you to collaborate effectively in an international environment and communicate clearly with colleagues from diverse backgrounds. 
• An interest in game design and game development. 

Additional Information

• Contract : 6 months Internship
• Location : role based in Bordeaux – France
• Remote: We embrace a hybrid work model helping you stay connected with your team and aligned with business priorities, while giving you the opportunity to maintain your work-life balance. Note, that some roles are fully office-based and are not eligible for hybrid work.

Relevant literature: 

• https://sakana.ai/shinka-evolve/ 
• https://eureka-research.github.io/ 
• Generative Agents: Interactive Simulacra of Human Behavior 
• Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning 
• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning 
• https://voyager.minedojo.org/ 

Check out this guide to help you with your application, and learn about our actions to encourage more diversity and inclusion.