Quality Analyst, Prime Video Trust & Safety

Our overall mission is simple: To be Earth’s most customer-centric company. We want Amazon to be the place where our customers can find, discover and buy anything online. Whatever our customers want, we will find the means to deliver it. With your help, Amazon will deliver world class AI generated experiences to our customers.

Prime Video is looking for a Quality Analyst II, who will have responsibility to assess the output produced by content moderators and machine learning models, assessing the quality and providing vital insights to program, operations, and science teams to improve moderator and machine learning models quality.

The role will require you to be a leader who prioritizes supporting the organization and ensuring defects do not persist. Your role will be to help set and oversee the quality standards of the data the team produces. You will define metrics, identify external signals, and build processes to reduce risk across the programs you support. You will consolidate large data sets into simplified insights that drive business growth and reduce risk.

You will also be responsible for producing reports that show the quality assessments completed by the quality team as well as conducting data analysis to identify key drivers and insights that may be reducing the effectiveness of the models’ outputs. You will produce reports and evidence backed by data and present these findings to science, engineering and program teams.

Key job responsibilities
Leadership
- Knows and communicates Amazon's mission, vision and strategy
- Interfacing with diverse stakeholders such as Program leadership, Operations leads, Technology teams, Business
- Ability to confidently facilitate team discussions and communicate business messages
- Can adapt well to changing circumstances, direction, and strategy
- Demonstrates the ability to give overall direction, performance, coordination and evaluation
- Strong interpersonal and communication skills, good listener, and comfortable interacting with up to Director level
- Ability to organize, prioritize and schedule work assignments
- Experience managing core business KPI’s

Operational Delivery
- Understands operational principles of service delivery and uses data to support continuous improvement
- Gathering requirements and delivering complete data solutions to drive insights and flag risks to Quality, Program, and Operations
- Leading complex analytical deep dives to identify potential process gaps
- Building metrics to analyze key inputs for risk detection and defect reduction
- Recognizing and adopting best practices in reporting and analysis: data integrity, test design, analysis, validation, and documentation

Continuous Improvement
- Uses data to identify areas of ongoing improvement in how content compliance and risk is managed
- Feedback Loop: Continuously communicate with the model development team to provide insights and observations regarding content moderation challenges, and potential improvements for the ML model's performance
- Data Integrity: Ensure the accuracy and integrity of the audited data by performing regular quality checks, identifying and addressing inconsistencies, and maintaining a high standard of data cleanliness
- Adaptation: Keep up-to-date with the latest content policy trends, customer expectations, language requirements, to adapt to audit strategies and guidelines accordingly

A day in the life
This is an individual contributor (IC) role focused on quality assessments and data insights for the program, operations, and quality teams to action and drive improvement.

You will provide reports that include the outcome and quality assessment of the audit and present findings, trends, and insights to business and science teams to improve the models training.