First Monday
Jobs
Jobs open to youOnly jobs that can hire where you liveA CV for each jobWritten from your record, every line checkedMemoryIt remembers, so you don’t repeat yourselfInterview helpResearch, prep and hints only you seeWatch how it works0:28
FAQDocsUse in AI, Free
Your countryNorth AmericaEuropeAsia-PacificLatin AmericaNot on the list? Tell us where you liveRequest early accessEarly access
First Monday

Remote jobs that can actually hire you.

One email to confirm your address, and another when your group is invited. Free during the beta. By requesting access you agree to the Terms & Conditions and the Privacy Policy.

Beta formacOSWindows

Jobs

Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico

Product

Jobs open to youA CV for each jobMemoryInterview helpHow it works

Company

FAQContact

Legal

Privacy PolicyTerms & Conditions
Jobs
Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico
Product
Jobs open to youA CV for each jobMemoryInterview helpHow it works
Company
FAQContact
Legal
Privacy PolicyTerms & Conditions
Remote jobs by country
United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand

Remote jobs by country

United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand
© 2026 First MondayNo tracking cookies. Anonymous page statistics only.
First Monday
Jobs/Data and AI/AI Testing Engineer, Amazon Bedrock

AI Testing Engineer, Amazon Bedrock

NTT DATA, Europe & LATAM, Branch in USA

LATAMData and AIPosted September 29, 2026
Open to people in Mexico and BrazilApply for this job Write my CV for this job Early access
Pay
$20–$25 an hour (contract)
Required
Python, JavaScript/TypeScript, Amazon Bedrock, Git, CI/CD, REST APIs, JSON, CSV, Generative AI / LLM behavior, groundedness
Experience
5+ years; 2+ years
Posted
September 29, 2026
Company
NTT DATA, Europe & LATAM, Branch in USA

Who it can hire

We read the post for each of the 21 countries we cover.

  • Open to people in Mexico and Brazil

    The post says: “Location: 100% remote, seeking candidates located in LATAM.”

  • Not open to people in the other 19 countries we cover, as far as the post says.

The job post

Location: 100% remote, seeking candidates located in LATAM.

Contract Length: 1 year, with potential for extension.

Contract Rate: $20-25per hour in USD. Final rate will be determined based on relevant experience and overall fit for the position.

NTT DATA is seeking an AI Testing Engineer, Amazon Bedrock to define and execute the quality strategy for Generative AI applications and AI agents used in Customer Service and Contact Center scenarios. This role will evaluate AI outputs across deterministic and non-deterministic scenarios using automation, semantic and statistical analysis, human review, and adversarial testing while partnering with AI engineers and functional QA teams.

Responsibilities:

  • Define the testing and evaluation strategy for Generative AI applications and AI agents.
  • Create and maintain evaluation datasets, golden datasets, and regression suites covering representative business scenarios.
  • Design single-turn and multi-turn conversational test cases, including context retention and workflow transitions.
  • Evaluate AI outputs for accuracy, relevance, completeness, groundedness, faithfulness, consistency, tone, safety, and compliance.
  • Build automated evaluation pipelines and integrate AI-quality checks into CI/CD and release processes.
  • Detect regressions introduced by changes to prompts, models, knowledge sources, retrieval strategies, tools, or guardrails.
  • Design negative and adversarial test scenarios covering prompt injection, jailbreaks, hallucinations, PII leakage, unsafe responses, and out-of-scope requests.
  • Validate fallback behavior, uncertainty handling, tool execution, error handling, and escalation to human agents.
  • Perform root-cause analysis for incorrect or undesirable AI outputs and collaborate with AI engineers on remediation.
  • Define measurable quality thresholds and release acceptance criteria for AI-agent capabilities.
  • Produce quality metrics, evaluation reports, dashboards, and trend analysis for project and product stakeholders.
  • Support UAT preparation, defect triage, production monitoring, and continuous improvement of AI behavior.

Required Experience:

  • 5+ years of experience in software testing, quality engineering, or test automation with hands-on automation capabilities.
  • 2+ years of experience with Amazon Bedrock.
  • Practical understanding of Generative AI / LLM behavior and the challenges of testing non-deterministic systems.
  • Experience designing test scenarios for conversational applications, chatbots, virtual assistants, or AI agents.
  • Experience with Python and/or JavaScript / TypeScript for test automation and data processing.
  • Experience testing REST APIs and backend services.
  • Experience working with structured test data in formats such as JSON and CSV.
  • Understanding of evaluation concepts such as groundedness, faithfulness, relevance, hallucination detection, semantic similarity, and human evaluation.
  • Experience with Git, CI/CD, automated regression testing, and defect-management workflows.
  • Strong analytical skills and ability to investigate why an AI output failed rather than only identifying that it failed.
  • Professional proficiency in English.

Nice-to-Have:

  • Experience with AI evaluation frameworks such as RAGAS, DeepEval, Promptfoo, LangSmith, TruLens, or equivalent tools.
  • Experience evaluating RAG systems, retrieval quality, or enterprise knowledge assistants.
  • Experience with Amazon Bedrock and AWS services.
  • Experience testing Amazon Connect, Contact Center, agent-assist, or Customer Service applications.

Education & Certifications:

  • Bachelor’s degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, Software Engineering, or a related technical field.

Required Equipment: This role requires the use of a personal laptop. Candidates must be comfortable with company security software and device management tools being installed on their laptop as part of the onboarding process and for the duration of the engagement.

About NTT DATA

NTT DATA is a Top 5 global IT services provider with more than 190,000 professionals across 50+ countries. We combine industry expertise with capabilities in consulting, technology, AI, cloud, applications, infrastructure, and connectivity to help organizations innovate, optimize, and transform. Through responsible innovation, we deliver meaningful business outcomes, accelerate client success, and create a positive impact on society.

Equal Opportunity Employer

NTT DATA is committed to hiring and retaining a diverse workforce. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, creed, sex, sexual orientation, gender identity, marital status, national origin, age, veteran status, disability, or any other protected class. NTT DATA is an Equal Opportunity Employer Male/Female/Disabled/Veteran and a VEVRAA Federal Contractor.

Req ID: 1355

First Monday isn’t the employer or a recruiter and isn’t affiliated with NTT DATA, Europe & LATAM, Branch in USA. Details may have changed since; confirm with the employer before you apply.

Open to people in Mexico and BrazilApply for this job Write my CV for this job Early access
Pay
$20–$25 an hour (contract)
Required
Python, JavaScript/TypeScript, Amazon Bedrock, Git, CI/CD, REST APIs, JSON, CSV, Generative AI / LLM behavior, groundedness
Experience
5+ years; 2+ years
Posted
September 29, 2026
Company
NTT DATA, Europe & LATAM, Branch in USA

Similar jobs open to people in Mexico

Data & AI Forward Deployed EngineerRevStar · District of Columbia, United States…AI Engineer – LLM Applications (US Startup · Remote LATAM)ITMates · Remote · WorldwideAI Engineer – Generative AI and AgentsAzumo · Argentina · Dominican Republic · Mexico…AI Engineer – Generative AI & AWSWorkana · Argentina · Mexico · Colombia · Peru…
First Monday

Get jobs like this every Monday.

One email a week with new remote jobs open to people in Canada. Free, and one click to unsubscribe. Privacy Policy

Or request early access to the app
Apply for this job