First Monday
Jobs
Jobs open to youOnly jobs that can hire where you liveA CV for each jobWritten from your record, every line checkedMemoryIt remembers, so you don’t repeat yourselfInterview helpResearch, prep and hints only you seeWatch how it works0:28
FAQDocsUse in AI, Free
Your countryNorth AmericaEuropeAsia-PacificLatin AmericaNot on the list? Tell us where you liveRequest early accessEarly access
First Monday

Remote jobs that can actually hire you.

One email to confirm your address, and another when your group is invited. Free during the beta. By requesting access you agree to the Terms & Conditions and the Privacy Policy.

Beta formacOSWindows

Jobs

Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico

Product

Jobs open to youA CV for each jobMemoryInterview helpHow it works

Company

FAQContact

Legal

Privacy PolicyTerms & Conditions
Jobs
Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico
Product
Jobs open to youA CV for each jobMemoryInterview helpHow it works
Company
FAQContact
Legal
Privacy PolicyTerms & Conditions
Remote jobs by country
United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand

Remote jobs by country

United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand
© 2026 First MondayNo tracking cookies. Anonymous page statistics only.
First Monday
Jobs/Data and AI/Member of Technical Staff, Evals

Member of Technical Staff, Evals

Magic

Remote · San Francisco, USData and AIPosted November 11, 2026
Probably open to people in the United StatesApply for this job Write my CV for this job Early access
Pay
$225,000–$550,000 a year
Required
post-training techniques
Visa
Sponsors visas
Equity
Equity
Benefits
Retirement plan, Health insurance, Dental and vision, Unlimited time off
Posted
November 11, 2026
Company
Magic

Who it can hire

We read the post for each of the 21 countries we cover.

  • Probably open to people in the US · Location only

    The listing’s location: “San Francisco · Remote”.

  • Not open to people in the other 20 countries we cover, as far as the post says.

The job post

Magic’s mission is to build safe AGI that accelerates humanity’s progress on the world’s most important problems. We believe the most promising path to safe AGI lies in automating research and code generation to improve models and solve alignment more reliably than humans can alone. Our approach combines frontier-scale pretraining, domain-specific RL, ultra-long context, and inference-time compute to achieve this goal.

About the role

The Evals team measures the performance and capabilities of internal and external models, helping teams across Magic make informed decisions. The team supports pretraining, RL, data, inference, and product, and sits on the critical path of many of the company's most important decisions.

As a Member of Technical Staff on Evals, you will develop new benchmarks and evaluation methodologies, run data ablations, assess dataset quality, and investigate whether our measurements accurately reflect model performance.

Sweating the details matters on this team. Many benchmarks, papers, and open-source evaluation frameworks contain subtle bugs or flawed assumptions that lead to misleading conclusions. We care deeply about correctness, reproducibility, and measurement quality.

What you'll work on

  • Design and validate new evaluations and benchmarks across model training and deployment

  • Investigate model behavior and identify gaps in existing evaluations

  • Measure dataset quality and identify opportunities to improve training data

  • Improve evaluation correctness, reproducibility, and reliability

  • Audit and improve upon public benchmarks, research papers, and open-source implementations

  • Define metrics that accurately reflect model quality

  • Build tools that make evaluation results accessible and useful across Magic

What we're looking for

  • Experience developing model evaluations, benchmarks, or ML experiments

  • Strong programming skills and experience working with machine learning systems

  • Track record of owning technical projects end-to-end

  • Strong understanding of post-training techniques and their impact on model performance

  • Skepticism toward results that cannot be reproduced, validated, or explained

  • Experience analyzing model performance and interpreting experimental results

  • Comfortable navigating ambiguity and determining whether a measurement actually captures the behavior it claims to measure

  • Commitment to raising the standard of rigor in AI evaluations

Our culture

  • Integrity. Words and actions should be aligned

  • Hands-on. At Magic, everyone is building

  • Teamwork. We move as one team, not N individuals

  • Focus. Safely deploy AGI. Everything else is noise

  • Quality. Magic should feel like magic

Magic strives to be the place where high-potential individuals can do their best work. We value quick learning and grit just as much as skill and experience.

Compensation, benefits, and perks (US)

  • Annual salary range between $225K - $550K depending on experience

  • Equity is a significant part of total compensation, in addition to salary

  • 401(k) plan with 6% salary matching

  • Generous health, dental, and vision insurance for you and your dependents

  • Unlimited paid time off

  • Visa sponsorship and relocation support for candidates moving to San Francisco

  • A small, fast-moving, highly collaborative team working on frontier AI systems

First Monday isn’t the employer or a recruiter and isn’t affiliated with Magic. Details may have changed since; confirm with the employer before you apply.

Probably open to people in the United StatesApply for this job Write my CV for this job Early access
Pay
$225,000–$550,000 a year
Required
post-training techniques
Visa
Sponsors visas
Equity
Equity
Benefits
Retirement plan, Health insurance, Dental and vision, Unlimited time off
Posted
November 11, 2026
Company
Magic

Similar jobs open to people in the United States

Member of Technical Staff, EvalsHandshake · Remote · USMember of Technical Staff, ML Platform (Evals)Runway · Remote · USMember of Technical Staff (Data Intelligence)Reka · Remote · US, UKAI Evals Engineer — Evaluation Datasets & Ground TruthProphetic Technologies · Remote · US
First Monday

Get jobs like this every Monday.

One email a week with new remote jobs open to people in Canada. Free, and one click to unsubscribe. Privacy Policy

Or request early access to the app
Apply for this job