First Monday
Jobs
Jobs open to youOnly jobs that can hire where you liveA CV for each jobWritten from your record, every line checkedMemoryIt remembers, so you don’t repeat yourselfInterview helpResearch, prep and hints only you seeWatch how it works0:28
FAQDocsUse in AI, Free
Your countryNorth AmericaEuropeAsia-PacificLatin AmericaNot on the list? Tell us where you liveRequest early accessEarly access
First Monday

Remote jobs that can actually hire you.

Beta formacOSWindows

Jobs

Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico

Product

Jobs open to youA CV for each jobMemoryInterview helpHow it works

Company

FAQContact

Legal

Privacy PolicyTerms & Conditions
Jobs
Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico
Product
Jobs open to youA CV for each jobMemoryInterview helpHow it works
Company
FAQContact
Legal
Privacy PolicyTerms & Conditions
Remote jobs by country
United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand

Remote jobs by country

United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand
© 2026 First MondayNo tracking cookies. Anonymous page statistics only.
First Monday
Jobs/Operations/Program Manager, Human Evaluations (Recommendations)

Program Manager, Human Evaluations (Recommendations)

Netflix

Remote · USOperationsPosted October 20, 2026
Open to people in the United StatesApply for this job Write my CV for this job Early access
Pay
$230,000–$360,000 a year
Required
human labeling platforms, film and TV content
Experience
4+ years
Equity
Stock options
Benefits
Health insurance, Wellness, Retirement plan, Family care
Posted
October 20, 2026
Company
Netflix

Who it can hire

We read the post for each of the 21 countries we cover.

  • Open to people in the US

    The employer lists this job as remote there: “Remote, United States”.

  • Not open to people in the other 20 countries we cover, as far as the post says.

The job post

At Netflix, our mission is to entertain the world. Together, we are writing the next episode - pushing the boundaries of storytelling, global fandom and making the unimaginable a reality. We are a dream team obsessed with the uncomfortable excitement of discovering what happens when you merge creativity, intuition and cutting-edge technology. Come be a part of what’s next.

Job Description

Great recommendations are how members find something worth their time. We are looking for an experienced Human Evaluations Program Manager to help us measure and improve the quality of recommendations across the Netflix homepage and discovery experience. You will work alongside Product Management and ML teams to build the human evaluation programs that tell us whether recommendations are working for members, and where they are not.

About the Role

Netflix is building toward more relevant, more varied, and more trustworthy recommendations, and thoughtful, high-quality evaluation is essential to making sure we are moving in the right direction. Join a team creating the frameworks, tools, and workflows that ensure human judgment is applied with consistency, clarity, and care, whether we are evaluating relevance, repetition, diversity, or whether a recommendation simply makes sense to the member.

You will not only shape how human evaluations of our recommendations are designed, but also own the day-to-day execution. From scoping and planning to vendor management, rater onboarding, and calibration, you will be accountable for driving delivery from start to finish. Just as critically, you will act as a thought partner to Product Management, helping translate recommendation strategy into evaluation designs and building a shared language around recommendation quality across teams.

The Opportunity

In this role, you will partner across product, research, data science, UX, and engineering to develop rubrics, frameworks, and workflows that enable rigorous, scalable human evaluation of recommendations. Beyond shaping the "what" and "how," you will also lead the "when" and "done." You will keep evaluation projects on track, ensuring consistent execution, timely delivery, efficient use of vendor capacity, and high rater alignment. If you are excited to bring structure to ambiguity and influence what members see when they open Netflix, while being accountable for tangible delivery, this is your chance to create meaningful impact from day one.

The Ideal Candidate

The ideal candidate brings a rare combination of structure and flexibility. You know how to create evaluation frameworks that are rigorous and scalable, and you are also a driver who gets them out the door. You are skilled at translating product strategy into evaluation workflows, defining milestones, managing vendors, and delivering consistent results in a dynamic environment. You can steer teams across functions, keep timelines on track, and ensure rater quality without micromanaging. You are curious about how personalization works and can build credibility with PMs, data scientists, and ML engineers. You have working fluency in film and TV content and can quickly scan a set of titles to tell whether they belong together. You thrive in spaces where there is no roadmap, and you take pride in making things real, not just possible.

Responsibilities:

  • Lead end-to-end execution of human evaluation and data operations initiatives, from intake and scoping to delivery
  • Partner with Product Management to translate recommendation strategy and roadmap priorities into evaluation designs, success metrics, and timelines
  • Manage human annotation vendors, including capacity planning, SLAs, quality performance, and escalations
  • Drive operational efficiency across evaluation programs, including cost per task, throughput, turnaround time, and speed to IRR
  • Develop clear, scalable guidelines and scoring rubrics to ensure consistent rater judgment, including for subjective, taste-based questions
  • Oversee rater onboarding, calibration, and QA workflows
  • Pilot and refine evaluation tasks to improve clarity, inter-rater reliability, and feedback quality
  • Anticipate and resolve bottlenecks and blockers
  • Act as the connective tissue across product, ML, data science, and vendor partners to ensure alignment and effective execution of evaluations at scale

Qualifications

  • 4+ years of experience working in human evaluations, data collection, labeling, or annotation operations in ML environments
  • Track record of implementing process improvements or quality control systems for data collection needs
  • Prior experience managing human annotation vendors, raters, or data labeling teams, including vendor performance and cost management
  • Experience improving operational efficiency of annotation programs (throughput, turnaround, cost)
  • Strong understanding of evaluation design, including guidelines, rubrics, and scoring protocols
  • Working fluency in film and TV content, including genres, formats, and talent
  • Proven ability in end-to-end management of complex, cross-functional programs, demonstrating strong Program Management skills and clear accountability for successful delivery
  • Experience with human labeling platforms
  • Excellent written and verbal communication skills
  • Ability to synthesize feedback into clear recommendations and process improvements
  • Familiarity with how rater subjectivity and bias affect taste-based judgments, and how to account for them in evaluation design
  • Strong organizational skills and executional focus; ability to track details while seeing the bigger picture

Nice To Have

  • Experience with personalization, recommender systems, search, or ranking evaluation
  • Familiarity with how offline human evaluation complements online experimentation (A/B tests) and engagement metrics
  • Experience evaluating at the list or page level, not only the individual item level
  • Experience partnering with Product Managers and Data Scientists on metric definition
Generally, our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top of market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is $230,000.00 - $360,000.00. This compensation range will vary based on location.

Netflix provides comprehensive benefits including Health Plans, Mental Health support, a 401(k) Retirement Plan with employer match, Stock Option Program, Disability Programs, Health Savings and Flexible Spending Accounts, Family-forming benefits, and Life and Serious Injury Benefits. We also offer paid leave of absence programs. Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off. Full-time salaried employees are immediately entitled to flexible time off. See more details about our Benefits here.

Netflix is a unique culture and environment. Learn more here.

Inclusion is a Netflix value and we strive to host a meaningful interview experience for all candidates. If you want an accommodation/adjustment for a disability or any other reason during the hiring process, please send a request to your recruiting partner.

We are an equal-opportunity employer and celebrate diversity, recognizing that diversity builds stronger teams. We approach diversity and inclusion seriously and thoughtfully. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.

Job is open for no less than 7 days and will be removed when the position is filled.

First Monday isn’t the employer or a recruiter and isn’t affiliated with Netflix. Details may have changed since; confirm with the employer before you apply.

Open to people in the United StatesApply for this job Write my CV for this job Early access
Pay
$230,000–$360,000 a year
Required
human labeling platforms, film and TV content
Experience
4+ years
Equity
Stock options
Benefits
Health insurance, Wellness, Retirement plan, Family care
Posted
October 20, 2026
Company
Netflix

Similar jobs open to people in the United States

Digital Program ManagerNerdio · Remote · USProgram Manager (Regulatory Reporting)Waymo · Mountain View, CA, US · USTechnical Program ManagerRobots and Pencils · United StatesProject Manager – Employee ExperienceNet at Work · Remote · US
First Monday

Get jobs like this every Monday.

One email a week with new remote jobs open to people in Canada. Free, and one click to unsubscribe. Privacy Policy

Or request early access to the app
Apply for this job