First Monday
Jobs
Jobs open to youOnly jobs that can hire where you liveA CV for each jobWritten from your record, every line checkedMemoryIt remembers, so you don’t repeat yourselfInterview helpResearch, prep and hints only you seeWatch how it works0:28
FAQDocsUse in AI, Free
Your countryNorth AmericaEuropeAsia-PacificLatin AmericaNot on the list? Tell us where you liveRequest early accessEarly access
First Monday

Remote jobs that can actually hire you.

One email to confirm your address, and another when your group is invited. Free during the beta. By requesting access you agree to the Terms & Conditions and the Privacy Policy.

Beta formacOSWindows

Jobs

Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico

Product

Jobs open to youA CV for each jobMemoryInterview helpHow it works

Company

FAQContact

Legal

Privacy PolicyTerms & Conditions
Jobs
Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico
Product
Jobs open to youA CV for each jobMemoryInterview helpHow it works
Company
FAQContact
Legal
Privacy PolicyTerms & Conditions
Remote jobs by country
United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand

Remote jobs by country

United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand
© 2026 First MondayNo tracking cookies. Anonymous page statistics only.
First Monday
Jobs/Data and AI/Subject Matter Experts: ML Model Evaluation (PhD)

Subject Matter Experts: ML Model Evaluation (PhD)

Weekday AI

United States · United Kingdom · Canada…Data and AIPosted November 13, 2026Recruitment agency
Open to people in 5 countriesApply for this job Write my CV for this job Early access
Pay
$150 / hr (contract, currency not stated)
Visa
No visa sponsorship
Starts
Within Friday EOD
Posted
November 13, 2026
Company
Weekday AI

Who it can hire

We read the post for each of the 21 countries we cover.

  • Open to people in the US, Canada, the UK, Switzerland and Norway

    The employer lists this job as remote there: “United States · United Kingdom · Canada · Norway · Switzerland · Iceland”.

  • Not open to people in the other 16 countries we cover, as far as the post says.

The job post

This role is for one of Weekday’s clients

Engagement Type: Independent Contractor
Work Arrangement: Fully Remote
Estimated Commitment: Approximately 5 Hours
Schedule: Flexible, Asynchronous
Payment Frequency: Weekly, based on services rendered

Eligibility:

  • Applicants must be from any of these geographies : US, UK, Canada, the EU, Norway, Switzerland and Iceland.
  • Should have a finished / running paper in any of these topics : NeurIPS, ICML, ICLR, COLM or RLC
  • We are unable to support H1-B or STEM OPT candidates at this time.

Note:

  • Pay : $150 / hr
  • This requirement is super-urgent & would be awesome if you can start on the project within Friday EOD.

Role Overview

We are seeking highly qualified Subject Matter Experts with PhD-level academic or professional expertise across specialised research and professional domains to participate in a paid Machine Learning (ML) Model Evaluation project.

The project focuses on assessing the quality, accuracy, and reliability of AI-generated responses to complex research prompts and domain-specific questions. Your evaluations will help identify where current machine learning models perform effectively and where their outputs require further refinement.

This opportunity is suited to experienced researchers, academics, data analysts, technical writers, and domain specialists who can critically assess complex information, identify factual inaccuracies, and provide structured, evidence-based feedback.

Requirements

Key Responsibilities

  1. Review machine learning model outputs generated in response to a range of research-focused and domain-specific prompts.
  2. Evaluate AI-generated responses for factual accuracy, relevance, completeness, and overall quality.
  3. Assess the technical validity and reliability of information presented in model-generated responses.
  4. Use a structured rating system to evaluate and score responses consistently.
  5. Provide concise, clear, and well-reasoned written justifications supporting evaluation scores.
  6. Identify factual errors, gaps in reasoning, irrelevant information, and areas requiring improvement.
  7. Complete approximately five hours of independent evaluation work asynchronously.
  8. Maintain consistency and attention to detail throughout the evaluation process.

Required Qualifications and Skills

  1. PhD or equivalent advanced subject matter expertise in a relevant academic, technical, or professional discipline.
  2. Strong academic or professional background in a specialised domain, with the ability to assess complex information critically.
  3. Excellent analytical, research, and critical-thinking skills.
  4. Ability to evaluate technical content, research findings, and domain-specific information for factual correctness and relevance.
  5. Strong written communication skills, particularly in providing clear and structured evaluation justifications.
  6. High attention to detail and the ability to identify subtle inaccuracies, inconsistencies, and gaps in AI-generated responses.
  7. Ability to work independently and complete approximately five hours of asynchronous evaluation tasks.

Suitable Candidate Profiles

Professionals from the following backgrounds are encouraged to apply:

  • Academic researchers and university faculty
  • PhD scholars and doctoral-level subject matter experts
  • Data analysts and research analysts
  • Technical writers and research professionals
  • Domain specialists with experience evaluating technical or research-focused content

Engagement and Payment Terms

  • Contract Type: Independent contractor engagement.
  • Work Arrangement: Fully remote, with flexible working hours.
  • Project Duration: Approximately five hours of evaluation work.
  • Payment: Payments are processed weekly based on services rendered.
  • Project Continuity: The engagement may be extended, shortened, or concluded early depending on project requirements and individual performance.
  • Confidentiality: Participation will not require access to confidential or proprietary information belonging to any current or former employer, client, or academic institution.
  • Work Authorisation: H-1B and STEM OPT candidates cannot be supported for this opportunity.

Interview and Evaluation Expectations

During the interview process, candidates may be assessed on their review practices, critical thinking, and ability to improve the quality of their evaluations. Responses should address the following three habits:

  1. Exchanging unfinished review notes with another reviewer.
  2. Maintaining a separate log of opinions or assessments that have changed during the review process.
  3. Using review notes and feedback to update personal reading lists.

Ideal Candidate Profile

The ideal candidate combines deep subject matter expertise with strong research judgement and analytical precision. They should be comfortable reviewing complex AI-generated content, distinguishing accurate information from flawed reasoning, and communicating their assessments through clear, structured feedback.

First Monday isn’t the employer or a recruiter and isn’t affiliated with Weekday AI. Details may have changed since; confirm with the employer before you apply.

Open to people in 5 countriesApply for this job Write my CV for this job Early access
Pay
$150 / hr (contract, currency not stated)
Visa
No visa sponsorship
Starts
Within Friday EOD
Posted
November 13, 2026
Company
Weekday AI

Similar jobs open to people in the United States

MML Engineer (MSc. or PhD)Micro1 · Remote · US, Canada, UK, Ireland, Germany, Netherlands, Spain, Portugal, Australia, Mexico, Brazil, India, Poland, Sweden, Denmark, Norway, Finland, Switzerland, Austria, Belgium, New ZealandAI Trainer (ML Engineering)Xperteez Technology · Remote · WorldwideMachine Learning Experts – PhD – AI TrainingProlific Academic · Remote · UKCMathematics Model Evaluators (AI Data Trainers)Careerflow.ai · Remote · Worldwide
First Monday

Get jobs like this every Monday.

One email a week with new remote jobs open to people in Canada. Free, and one click to unsubscribe. Privacy Policy

Or request early access to the app
Apply for this job