AI Evaluation Analyst

About Ema: Ema is at the forefront of artificial intelligence research and application, dedicated to creating cutting-edge AI solutions that empower users globally. Join our dynamic team to help shape the future of AI technology.

Responsibilities:

1. Create Enterprise AI Model Evaluation Test Sets: Develop and maintain comprehensive test sets tailored for various enterprise use cases.

2. Evaluate AI Models: Assess different versions, personas, and prompts of Ema on multiple quality parameters. Ensure new features and updates meet established standards before launch.

3. Generate AI Evaluation Reports: Compile detailed evaluation reports that provide insights for improving persona setups and Ema's AI models.

4. Test UX Flows: Evaluate the efficacy and usability of auxiliary user experience flows to ensure seamless end-to-end GenAI usage for Ema’s users.

5. Automate Testing Flows: Develop and implement scripts and browser automation tools to streamline testing processes. Enhance testing velocity and early issue detection to improve overall product quality.

Requirements:

1. Educational Background: Bachelor’s degree in Engineering. Bachelor’s degree in Computer Science, Data Science, or a related field is preferred.

2. Analytical Skills: Exceptional attention to detail with the ability to identify rare and complex edge cases.

3. Communication Skills: Strong verbal and written English communication abilities..

4. Professional Experience: 1-3 years of prior professional experience. Background in analytics, prompt engineering, or QA testing is a plus.

Benefits:

• Stay at the cutting edge of GenAI research, guided and coached by top researchers in the field.

• Learn best practices for solving user problems with GenAI products.

• Collaborate across cross-functional teams to effectively achieve customer impact.

Ema Unlimited is an equal opportunity employer and is committed to providing equal employment opportunities to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, sexual orientation, gender identity, or genetics.