Senior Annotation Operations Lead - Modeling

Full-time

Job Description

This job posting has expired and no longer accepting applications.

Who are we?

Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI.

We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers.

Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products.

Join us on our mission and shape the future!

Why this role?

The Senior Annotation Operations Lead is responsible for building and managing Cohere’s team of annotators and reviewers who develop and deliver high-quality training data critical to the development of Cohere's flagship models. You will join a small and effective team who support Cohere’s data collection efforts.

Your day-to-day will include helping engineering stakeholders translate their needs into actionable human data projects, monitoring and improving data quality, and creating scalable processes and tools to effectively collect data.

As a Senior Annotation Operations Lead, you will:

Partner with engineering stakeholders to develop human data projects. This can include writing instructions, training our annotators, piloting projects, and defining success criteria.

Oversee quality assurance and reviews of internal and externally produced data, providing feedback to annotators to ensure alignment with project success criteria and guidelines.

Manage the performance of our annotator teams with the goal of continuously improving our performance KPIs, plans, and processes.

Test and iterate on our internal annotation processes and tooling to improve the quality of data, annotator quality of life and throughput.

Run and develop onboarding and training programs for new annotators, as well as identify learning and development opportunities to upskill our annotators.

Partner with internal stakeholders to decide on workforce planning and hiring strategy to meet roadmap requirements, as well as work with external vendors to enrich our data pipeline as needed.

You might be a good fit if you:

Thrive in a fast-paced, dynamic environment. Our data operations run 24/7.

Are resourceful, super-organized, and a clear and concise communicator.

Are adaptable - work looks different here every day. You should be comfortable leaning into human data projects spanning both familiar and unfamiliar domains (eg: finance, medicine, multilingual).

Have run human operations at scale - this could be data labeling, but it also might be something else!

Have a track record of working well with cross-functional stakeholders and being able to be brought up to speed quickly on project goals.

Have an educator background or a demonstrated eagerness to teach.

Enjoy staring at data on a daily basis, and can quickly identify trends or pathologies in data quality. Familiarity with data visualization tools, Excel, SQL is a plus.

If some of the above doesn’t line up perfectly with your experience, we still encourage you to apply! If you consider yourself a thoughtful worker, a lifelong learner, and a kind and playful team member, Cohere is the place for you.

We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.

Our Perks:

🤝 An open and inclusive culture and work environment

🧑‍💻 Work closely with a team on the cutting edge of AI research

🍽 Weekly lunch stipend, in-office lunches & snacks

🦷 Full health and dental benefits, including a separate budget to take care of your mental health

🐣 100% Parental Leave top-up for 6 months for employees based in Canada, the US, and the UK

🎨 Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement

🏙 Remote-flexible, offices in Toronto, New York, San Francisco and London and co-working stipend

✈️ 6 weeks of vacation

Note: This post is co-authored by both Cohere humans and Cohere technology.

Please mention that you found this job on MoAIJobs, this helps us grow. Thank you!