Research Engineer/Research Scientist, Pre-Training Evaluation and Validation, UK

Snapshot

Artificial Intelligence could be one of humanity’s most useful inventions. At Google DeepMind, we’re a team of scientists, engineers, machine learning experts and more, working together to advance the state of the art in artificial intelligence. We use our technologies for widespread public benefit and scientific discovery, and collaborate with others on critical challenges, ensuring safety and ethics are the highest priority.

The Role

We are seeking a Research Scientist/Engineer to join the Gemini Large Scale Pre-training Evaluations team in London. This is a high-impact role with direct influence on decisions made for designing our large scale Gemini models. You will join a team working on evaluating and building large models, exploring their capabilities and improving the infrastructure used by a large team of researchers and engineers working on Gemini models.

Key responsibilities

• Spearhead and own the development of large scale validation datasets (text, image, audio, video) for our state of the art large scale pretrained models.
• Collaborate closely with model developers to provide them with actionable insights and trustworthy metrics.
• Own and drive improvements to their technical infrastructure.
• Drive the development of novel techniques and analyses to provide insights into the relationship between data, scaling laws, leakage and memorisation.
• Continuously develop new methods and insights to push the frontier of our understanding of model abilities and capabilities.
• Perform deep assessments of data and eval coverage to help drive future model decisions.
• Improve methods for data leakage detection and leakage prevention.

About you

In order to set you up for success as a Research Scientist at Google DeepMind, we look for the following skills and experience:

• PhD in Computer Science or related field, or equivalent practical experience.
• Strong publication record in top machine learning conferences (e.g., NeurIPS, CVPR, ICML, ICLR, ICCV, ECCV).
• Experience in at least one modern deep learning framework (e.g. JAX, PyTorch). 

In addition, some of the following would be an advantage:

• Experience with large scale models evaluation and statistical analysis of model capabilities.
• Deep knowledge in scaling laws and the relationship between data, model abilities and emergent abilities.
• Experience with data leakage detection and prevention.
• Excited to collaborate across orgs and disciplines to leverage diverse perspectives and expertise and find creative solutions. 

Application Deadline: 15th August

At Google DeepMind, we value diversity of experience, knowledge, backgrounds and perspectives and harness these qualities to create extraordinary impact. We are committed to equal employment opportunity regardless of sex, race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, gender identity, pregnancy, or related condition (including breastfeeding) or any other basis as protected by applicable law. If you have a disability or additional need that requires accommodation, please do not hesitate to let us know.