First Monday
Jobs
Jobs open to youOnly jobs that can hire where you liveA CV for each jobWritten from your record, every line checkedMemoryIt remembers, so you don’t repeat yourselfInterview helpResearch, prep and hints only you seeWatch how it works0:28
FAQDocsUse in AI, Free
Your countryNorth AmericaEuropeAsia-PacificLatin AmericaNot on the list? Tell us where you liveRequest early accessEarly access
First Monday

Remote jobs that can actually hire you.

Beta formacOSWindows

Jobs

Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico

Product

Jobs open to youA CV for each jobMemoryInterview helpHow it works

Company

FAQContact

Legal

Privacy PolicyTerms & Conditions
Jobs
Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico
Product
Jobs open to youA CV for each jobMemoryInterview helpHow it works
Company
FAQContact
Legal
Privacy PolicyTerms & Conditions
Remote jobs by country
United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand

Remote jobs by country

United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand
© 2026 First MondayNo tracking cookies. Anonymous page statistics only.
First Monday
Jobs/Data and AI/Applied AI Engineer

Applied AI Engineer

On The Spot Development

PolandData and AIPosted November 13, 2026
Open to people in PolandApply for this job Write my CV for this job Early access
Pay
Not stated
Required
multi-step, tool-using agents, English (comfortably)
Benefits
benefits package
Posted
November 13, 2026
Company
On The Spot Development

Who it can hire

We read the post for each of the 21 countries we cover.

  • Open to people in Poland

    The employer lists this job as remote there: “Poland”.

  • Not open to people in the other 20 countries we cover, as far as the post says.

The job post

About On The Spot
We bring startups and disruptive tech companies to the local market, offering engineers the opportunity to work on innovative, high-impact products. Join us and become a member of their R&D team while we take care of employment, HR support, a benefits package, and an enjoyable work environment.

About the team
Handle (formerly 44pixels) is a London-based, VC-backed startup founded in 2024. We started as a venture studio building AI-first mobile apps, including Cue, Wordcast and Clara, and now focus on one product: Handle.

Handle is an AI agent for owners of local service businesses: groomers, roofers, bookkeepers, caterers and many more. Owners chat with it, and it handles routine work such as replying to customer messages, chasing invoices and answering reviews. It connects to the tools they already use, including Gmail, Outlook, Google Calendar, WhatsApp, Messenger, QuickBooks and Xero. The owner decides what Handle can access and what needs their approval.

About the role
You'll be the first person whose whole job is how good our agent is. Owners trust Handle with their customers and their business, so good means more than finishing the task. It means getting the details right, understanding what the customer needs, and knowing when to ask the owner first. You'll build the evals that measure that, and run the loop that makes it better.

Our engineers have laid the groundwork for evals and own the harness, tracing and deployment they run on. You'll take the evals over and own them from here.

What you'll do

  • Build and own our eval suite: scenarios, rubrics, graders, benchmarks, and the quality gates every change has to pass.

  • Run the hill-climb loop on the agent's skills. Find where it goes wrong, form a hypothesis, change the skill or prompt, rerun the suite, and prove the fix worked without breaking something else.

  • Judge conversation quality as well as task success: whether the agent understood the customer, said the right thing, and helped.

  • Test new models as they come out, and separate what the model changed from what the prompts, tools and skills changed.

  • Spot repeated failures that local fixes won't solve, and make the case for restructuring a skill or the agent.

  • Build our view on fine-tuning: when it's worth it, what data it would need, and whether a better skill or model gets there first.

Where you'll start

  • First month: take over the existing evals, learn the agent's main workflows, and set a baseline scorecard.

  • By month two: the hill-climb loop is running, every skill change goes through a regression gate, and the first improvements have shipped and been verified.

  • By month three: a repeatable way to compare models, a ranked backlog of improvements, and a short written view on whether fine-tuning is worth exploring.

Requirements

  • You've built and run evals for an LLM product in production. You've written graders, checked an LLM judge against human ratings, and caught a regression before users did.

  • You've worked on multi-step, tool-using agents, and you know how they fail.

  • You write the code yourself, from graders and harness code to experiments, and you read traces without waiting on anyone.

  • You're disciplined about experiments. You run things more than once, account for variance, and can tell a real gain from noise.

  • You have taste in conversation. You can read a transcript and say what was wrong with it, even when the task got done.

  • You're comfortable setting your own agenda. As the first person in this area, you'll decide what to measure first.

  • You work comfortably in English, which is the language of our team.

Nice to have

  • Hands-on post-training or fine-tuning of open-weight models.

What this role isn't

This isn't a frontier research role. You won't need a GPU cluster, and we're not optimising for papers or leaderboards. You don't need a PhD. You need to love finding out why an agent did the wrong thing, and proving your fix worked.

Why join

  • Own a whole area from day one, and set how we measure quality.

  • Work on an agent that real business owners trust with their customers, with real conversations to learn from.

  • See your work in production within weeks.

  • Grow with the work. As it develops, there's room to go further into training our own models, and to build the team that does it.

First Monday isn’t the employer or a recruiter and isn’t affiliated with On The Spot Development. Details may have changed since; confirm with the employer before you apply.

Open to people in PolandApply for this job Write my CV for this job Early access
Pay
Not stated
Required
multi-step, tool-using agents, English (comfortably)
Benefits
benefits package
Posted
November 13, 2026
Company
On The Spot Development

Similar jobs open to people in Poland

Senior Machine Learning Founding EngineerYard Corporate · PolandAI EngineerB3 Consulting Poland · PolandSenior AI Platform EngineerALTEN Polska · PolandAI Engineering LeadHaddad Brands · Poland
First Monday

Get jobs like this every Monday.

One email a week with new remote jobs open to people in Canada. Free, and one click to unsubscribe. Privacy Policy

Or request early access to the app
Apply for this job