First Monday
Jobs
Jobs open to youOnly jobs that can hire where you liveA CV for each jobWritten from your record, every line checkedMemoryIt remembers, so you don’t repeat yourselfInterview helpResearch, prep and hints only you seeWatch how it works0:28
FAQDocsUse in AI, Free
Your countryNorth AmericaEuropeAsia-PacificLatin AmericaNot on the list? Tell us where you liveRequest early accessEarly access
First Monday

Remote jobs that can actually hire you.

Beta formacOSWindows

Jobs

Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico

Product

Jobs open to youA CV for each jobMemoryInterview helpHow it works

Company

FAQContact

Legal

Privacy PolicyTerms & Conditions
Jobs
Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico
Product
Jobs open to youA CV for each jobMemoryInterview helpHow it works
Company
FAQContact
Legal
Privacy PolicyTerms & Conditions
Remote jobs by country
United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand

Remote jobs by country

United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand
© 2026 First MondayNo tracking cookies. Anonymous page statistics only.
First Monday
Jobs/Data and AI/AI Benchmark & Datasets Engineer / Researcher

AI Benchmark & Datasets Engineer / Researcher

Pathway

Palo Alto, California, United StatesData and AIPosted March 19, 2026
Probably open to people in the United StatesApply for this job Write my CV for this job Early access
Pay
Not stated
Required
ML/LLM evaluation, Benchmarks, GitHub, English (fluent)
In person
Office (offered)
Starts
Immediate
Apply by
Open until filled
Posted
March 19, 2026
Company
Pathway

Who it can hire

We read the post for each of the 21 countries we cover.

  • Probably open to people in the US · Location only

    The listing’s location: “Palo Alto, California, United States”.

  • Not open to people in the other 20 countries we cover, as far as the post says.

The job post

About Pathway

Pathway builds the first post-transformer frontier model that enables recurrent reasoning and recurrent memory, unbounded context, and real-time adaptation. We're not optimizing yesterday's technology; we're building what comes after Transformers.

Pathway is currently scaling its internal models from billions of parameters towards hundreds of billions of parameters, aiming at achieving differentiated frontier intelligence.

Pathway is led by co-founder & CEO Zuzanna Stamirowska, a complexity scientist who created a team consisting of AI pioneers, including CTO Jan Chorowski who was the first person to apply Attention to speech and worked with Nobel laureate Geoff Hinton at Google Brain, as well as CSO Adrian Kosowski, a leading computer scientist and quantum physicist who obtained his PhD at the age of 20.

The company is backed by leading investors, as well as advisors including Lukasz Kaiser, co-author of the Transformer (“the T” in ChatGPT) and a key researcher behind OpenAI’s reasoning models, and Jonathan Frankle, Chief AI Scientist at Databricks and Co-founder at Mosaic. Pathway is headquartered in Palo Alto, California.

The Opportunity

You will design and execute rigorous benchmarks and define dataset standards. Collaborating closely with our R&D team, you will build the evaluation infrastructure that guides the evolution of Pathway’s post‑transformer models.

You Will

  • Proactively identify, prioritize, and curate relevant public and client-driven benchmarks across our target use cases and markets.
  • Evaluate candidate benchmarks for clarity, data quality, evaluation methodology, and fit with our model roadmap.
  • Run benchmarks with baseline models to validate setup, uncover edge cases, and de‑risk R&D runs.
  • Hand off “benchmark-ready” packages to R&D (specs, data, evaluation scripts, expected metrics, constraints)
  • Maintain a shared vocabulary and documentation around benchmarks, datasets, and evaluation formats that GTM and R&D can both use.
  • Track and organize benchmark results, model leaderboards, and “what good looks like” for different customers and scenarios.
  • Contribute to demos and public‑facing proof points based on benchmark outcomes.

You will play a key role in defining and driving the benchmarking process for AI model evaluation. Your work will directly influence what we build, how we talk about it, and how customers and the market experience BDH.

Requirements

Cover letter

It's always a pleasure to say hi! If you could leave us 2-3 lines, we'd really appreciate that.

You are expected to meet at least one of the following criteria:

  1. You have published at least one paper at NeurIPS, ICLR, or ICML - where you were the lead author or made significant conceptual & code contributions.
  2. You have significantly contributed to an LLM training effort which became newsworthy (topped a Hugging Face benchmark, best in class model, etc.), preferably using multiple GPU's.
  3. You have spent at least 6 months working in a leading Machine Learning research center (e.g. at: Google Brain / Deepmind, Apple, Meta, Anthropic, Nvidia, MILA).
  4. You were an ICPC World Finalist, or an IOI, IMO, or IPhO medalist in High School.

You

  • Have experience with ML/LLM evaluation, data science, or technical product roles, ideally around benchmarks or experimentation.
  • Are comfortable reading papers, leaderboards, and Github repos, and turning them into clear, repeatable benchmark specs.
  • Can talk comfortably with both engineers and customers, and translate between technical detail and business value.
  • Care about high‑quality data, reproducible experiments, and crisp documentation
  • Are respectful of others
  • Are fluent in English

Bonus Points

  • Published or open‑sourced work on LLM evaluation, benchmarking or data quality.
  • Experience designing custom benchmarks or evaluation protocols for novel model capabilities.

Why You Should Apply

  • Join an intellectually stimulating work environment.
  • Be a pioneer: you get to work with a new type of "Live AI" challenges around long sequences and changing data.
  • Be part of one of an early-stage AI startup that believes in impactful research and foundational changes.

Benefits

  • Type of contract: Full-time, permanent
  • Preferable joining date: Immediate. The positions are open until filled – please apply immediately.
  • Compensation: based on profile and location.
  • Location: Remote work. Possibility to work or meet with other team members in one of our offices: Palo Alto, CA; Paris, France or Wroclaw, Poland. Candidates based anywhere in the EU, UK, United States, and Canada will be considered.

If you meet our broad requirements but are missing some experience, don’t hesitate to reach out to us.

First Monday isn’t the employer or a recruiter and isn’t affiliated with Pathway. Details may have changed since; confirm with the employer before you apply.

Probably open to people in the United StatesApply for this job Write my CV for this job Early access
Pay
Not stated
Required
ML/LLM evaluation, Benchmarks, GitHub, English (fluent)
In person
Office (offered)
Starts
Immediate
Apply by
Open until filled
Posted
March 19, 2026
Company
Pathway

Similar jobs open to people in the United States

Senior Machine Learning Engineer, AI Safety EvaluationsNVIDIA · Remote · USMember of Technical Staff, Coding AgentsHandshake · Remote · USPrincipal AI ResearcherRecursion · Remote · USAI Engineer, SeniorPathward, N.A. · Remote · US
First Monday

Get jobs like this every Monday.

One email a week with new remote jobs open to people in Canada. Free, and one click to unsubscribe. Privacy Policy

Or request early access to the app
Apply for this job