First Monday
Jobs
Jobs open to youOnly jobs that can hire where you liveA CV for each jobWritten from your record, every line checkedMemoryIt remembers, so you don’t repeat yourselfInterview helpResearch, prep and hints only you seeWatch how it works0:28
FAQDocsUse in AI, Free
Your countryNorth AmericaEuropeAsia-PacificLatin AmericaNot on the list? Tell us where you liveRequest early accessEarly access
First Monday

Remote jobs that can actually hire you.

One email to confirm your address, and another when your group is invited. Free during the beta. By requesting access you agree to the Terms & Conditions and the Privacy Policy.

Beta formacOSWindows

Jobs

Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico

Product

Jobs open to youA CV for each jobMemoryInterview helpHow it works

Company

FAQContact

Legal

Privacy PolicyTerms & Conditions
Jobs
Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico
Product
Jobs open to youA CV for each jobMemoryInterview helpHow it works
Company
FAQContact
Legal
Privacy PolicyTerms & Conditions
Remote jobs by country
United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand

Remote jobs by country

United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand
© 2026 First MondayNo tracking cookies. Anonymous page statistics only.
First Monday
Jobs/Data and AI/AI Operations Specialist – LLM & Conversation Systems

AI Operations Specialist – LLM & Conversation Systems

Pavago

Mexico · Colombia · Costa Rica…Data and AIPosted October 21, 2026Recruitment agency
Open to people in Mexico and BrazilApply for this job Write my CV for this job Early access
Pay
Not stated
Required
Proven experience running LLM or AI systems in production, Prompt engineering, system prompt optimization, Webhooks, APIs, Knowledge base, English (Excellent written)
Hiring
Screening call, Interviews, Offer
Posted
October 21, 2026
Company
Pavago

Who it can hire

We read the post for each of the 21 countries we cover.

  • Open to people in Mexico and Brazil

    The employer lists this job as remote there: “Mexico · Colombia · Costa Rica · Argentina · Brazil · Peru”.

  • Not open to people in the other 19 countries we cover, as far as the post says.

The job post

AI Operations Specialist – LLM & Conversation Systems

Position Type: Full-Time, Remote
Working Hours: U.S. Business Hours

About the Role

We are looking for a hands-on AI Operations Specialist to own and improve our production AI conversation systems.

We are an AI-powered SMS lead recovery platform serving telehealth brands, processing millions of messages each month. Our AI systems interact directly with customers, and their performance directly impacts client conversions.

In this role, you'll onboard new clients, configure AI conversation systems, monitor live conversations, diagnose issues, improve prompts and system behavior, and measure whether your changes actually improve conversion.

This is not a theoretical AI role. We're looking for someone who has run LLM systems in production and understands how to continuously monitor, troubleshoot, test, and improve them.

Responsibilities

Client Onboarding & AI Setup

  • Onboard new clients into the AI conversation platform.
  • Set up webhook integrations and client knowledge bases.
  • Configure campaigns and conversation workflows.
  • Ensure new client systems are properly tested before going live.
  • Troubleshoot integration and configuration issues during onboarding.

Production AI Operations

  • Monitor live AI-generated conversations and customer interactions.
  • Review conversations to identify errors, inconsistencies, and missed opportunities.
  • Diagnose AI failure modes and determine their root causes.
  • Identify issues affecting conversation quality and conversion.
  • Escalate technical issues when necessary and follow through until resolved.

Prompt & System Optimization

  • Write, edit, and improve prompts that generate customer-facing messages.
  • Adjust system instructions and conversation logic based on real-world performance.
  • Test changes systematically before and after deployment.
  • Maintain documentation of prompt and system changes.
  • Continuously improve AI behavior while maintaining accuracy and consistency.

Performance & Conversion Analysis

  • Analyze conversation transcripts and performance data.
  • Measure whether system and prompt changes improve conversion.
  • Identify patterns across conversations that can inform future improvements.
  • Track key performance outcomes and communicate findings to leadership.
  • Use data and real customer interactions to guide optimization decisions.

Founder Collaboration

  • Work directly with the founder on daily AI operations and improvements.
  • Communicate issues, findings, and recommendations clearly.
  • Take ownership of problems from identification through resolution.
  • Move quickly while maintaining high standards for customer-facing AI output.

Required Experience & Skills

Must-Have

  • Proven experience running LLM or AI systems in production.
  • Hands-on experience monitoring and improving production AI outputs.
  • Experience with prompt engineering and system prompt optimization.
  • Strong understanding of LLM behavior, failure modes, and iterative testing.
  • Experience analyzing AI-generated conversations or customer-facing outputs.
  • Ability to work with webhooks, APIs, knowledge bases, or similar integrations.
  • Strong analytical and problem-solving skills.
  • Excellent written English — required for editing customer-facing AI messages and documenting system changes.
  • Strong attention to detail.
  • Ability to work independently and take ownership of production systems.

What Makes You a Strong Fit

  • You've actually operated AI systems in production rather than only experimenting with ChatGPT.
  • You are comfortable reviewing hundreds or thousands of AI interactions to identify patterns and problems.
  • You think in terms of outcomes and conversion, not just whether the AI response sounds good.
  • You are methodical about testing changes and measuring their impact.
  • You can quickly identify why an AI system is behaving incorrectly and determine how to improve it.
  • You communicate clearly and document your work thoroughly.
  • You enjoy working directly with a founder in a fast-moving environment.

What Does a Typical Day Look Like?

You'll begin by reviewing the performance of live AI conversations and looking for unusual patterns, failures, or opportunities for improvement.

You may spend part of the day onboarding a new telehealth client, configuring their knowledge base and integrations, and setting up their campaigns.

You'll also review conversation transcripts, identify issues with AI responses, update prompts or system instructions, test your changes, and monitor the resulting conversion performance.

Throughout the day, you'll work directly with the founder to discuss findings, prioritize improvements, and make sure production systems continue performing effectively.

Interview Process

  1. Initial Recruiter Screening
  2. Client Interview
  3. Offer Stage

#LI-AG1

First Monday isn’t the employer or a recruiter and isn’t affiliated with Pavago. Details may have changed since; confirm with the employer before you apply.

Open to people in Mexico and BrazilApply for this job Write my CV for this job Early access
Pay
Not stated
Required
Proven experience running LLM or AI systems in production, Prompt engineering, system prompt optimization, Webhooks, APIs, Knowledge base, English (Excellent written)
Hiring
Screening call, Interviews, Offer
Posted
October 21, 2026
Company
Pavago

Similar jobs open to people in Mexico

AI Automation & Voice Agent SpecialistUptalent.io · Mexico · Colombia · Argentina…AI/ML Engineer (AI Agents and Evaluation) (Volunteer)Metasys · Remote · WorldwideJunior AI/ML EngineerResolv.Global · Remote · MexicoLLM Trainer – Agent Function callXperteez Technology · Remote · Mexico, Brazil, India
First Monday

Get jobs like this every Monday.

One email a week with new remote jobs open to people in Canada. Free, and one click to unsubscribe. Privacy Policy

Or request early access to the app
Apply for this job