First Monday
Jobs
Jobs open to youOnly jobs that can hire where you liveA CV for each jobWritten from your record, every line checkedMemoryIt remembers, so you don’t repeat yourselfInterview helpResearch, prep and hints only you seeWatch how it works0:28
FAQDocsUse in AI, Free
Your countryNorth AmericaEuropeAsia-PacificLatin AmericaNot on the list? Tell us where you liveRequest early accessEarly access
First Monday

Remote jobs that can actually hire you.

Beta formacOSWindows

Jobs

Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico

Product

Jobs open to youA CV for each jobMemoryInterview helpHow it works

Company

FAQContact

Legal

Privacy PolicyTerms & Conditions
Jobs
Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico
Product
Jobs open to youA CV for each jobMemoryInterview helpHow it works
Company
FAQContact
Legal
Privacy PolicyTerms & Conditions
Remote jobs by country
United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand

Remote jobs by country

United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand
© 2026 First MondayNo tracking cookies. Anonymous page statistics only.
First Monday
Jobs/Data and AI/VBT Task Author
C

VBT Task Author

Careerflow.ai

Remote · British Columbia, CanadaData and AIPosted October 23, 2026
Probably open to people in CanadaApply for this job Write my CV for this job Early access
Pay
Not stated
Required
Open Source Intelligence (OSINT)
Experience
1+ years
Posted
October 23, 2026
Company
Careerflow.ai

Who it can hire

We read the post for each of the 21 countries we cover.

  • Probably open to people in Canada · Only some regions

    The listing’s location: “Remote (British Columbia, Canada)”.

  • Not open to people in the other 20 countries we cover, as far as the post says.

The job post

Careerflow Human Data Labs partners with AI companies to bring real-world professional expertise into their products.

Role Overview

We are seeking an analytical researcher and red-teaming specialist to build evidence-based web-research benchmark tasks that reliably expose weaknesses in state-of-the-art (SOTA) AI models (e.g., Claude Opus 5 Max). In this role, you will discover hard-to-surface facts, build precise research tasks around them, test them directly against leading LLMs, and diagnose the exact failure mechanisms in the model's search and reasoning paths.

Your work will directly shape AI evaluation frameworks, ensuring next-generation frontier models are tested against rigorous, non-trivial real-world research scenarios.

Job Requirements:

  • Core Experience: 1+ years in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.

  • Benchmark Familiarity: Proven experience authoring, testing, or reviewing tasks for AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity’s Last Exam (HLE).

  • Analytical & OSINT Skills: Intermediate-to-advanced web search and Open Source Intelligence (OSINT) skills; ability to navigate academic archives, local records, or obscure databases.

  • Model Trajectory Debugging: Strong analytical capability to inspect model reasoning chains, search logs, and source selection to diagnose root causes of failure.

  • Precision Writing: Ability to author clear, unambiguous benchmark prompts with bulletproof, primary-sourced answer keys.

Job Responsibilities:

  • Obscure Fact Discovery: Deeply research niche topics to identify factual ground truths that cannot be surfaced via simple, direct search queries.

  • Source Verification: Rigorously verify facts against primary or highly credible sources, cross-check for contradictory evidence, and document exact source URLs.

  • Task Construction: Draft clear, answerable benchmark tasks designed specifically to expose structural weaknesses in model research strategies.

  • Red-Teaming & Model Testing: Execute tasks against designated SOTA models to confirm they reliably trigger intended model failures.

  • Failure Diagnosis & Debugging: Trace model run logs, search trajectories, and intermediate outputs to pinpoint exactly where and why the model failed (distinguishing genuine model limitations from ambiguous prompts).

  • Pattern Scaling & Review: Apply validated "model-breaking" research patterns across new domains and, for strong performers, review tasks authored by peer trainers.

Preferred Qualifications

  • Bachelor or higher degree from relevant engineering domain is a must for this position.

  • Prior experience specifically designing prompt-injection or reasoning-trap tasks that break SOTA models.

  • Advanced OSINT capabilities or published investigative/academic research demonstrating deep source verification.

    Location

    Bangladesh, India, Indonesia, Egypt, Ghana, Nigeria, Turkey ,Brazil, Columbia

    Total number of positions: 180

    Estimated earning potential of approximately $60 per task

    Engagement Length: 4 weeks, Full-time (8 hours/day) 40 hours per week. Overlap 4 hours with PST

First Monday isn’t the employer or a recruiter and isn’t affiliated with Careerflow.ai. Details may have changed since; confirm with the employer before you apply.

Probably open to people in CanadaApply for this job Write my CV for this job Early access
Pay
Not stated
Required
Open Source Intelligence (OSINT)
Experience
1+ years
Posted
October 23, 2026
Company
Careerflow.ai

Similar jobs open to people in Canada

Machine Learning Engineer — Inference OptimizationFeatherless AI · Remote · Worldwide🌍 Subject Matter Expert – Data & Statistical Chart Analysis, Remote – ContractXperteez Technology · Remote · WorldwideSenior Data EngineerGrowth Minded · Anywhere in the WorldStaff Machine Learning / Operations Research EngineerBurq · United States · Canada
First Monday

Get jobs like this every Monday.

One email a week with new remote jobs open to people in Canada. Free, and one click to unsubscribe. Privacy Policy

Or request early access to the app
Apply for this job