First Monday
Jobs
Jobs open to youOnly jobs that can hire where you liveA CV for each jobWritten from your record, every line checkedMemoryIt remembers, so you don’t repeat yourselfInterview helpResearch, prep and hints only you seeWatch how it works0:28
FAQDocsUse in AI, Free
Your countryNorth AmericaEuropeAsia-PacificLatin AmericaNot on the list? Tell us where you liveRequest early accessEarly access
First Monday

Remote jobs that can actually hire you.

Beta formacOSWindows

Jobs

Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico

Product

Jobs open to youA CV for each jobMemoryInterview helpHow it works

Company

FAQContact

Legal

Privacy PolicyTerms & Conditions
Jobs
Jobs open to you in CanadaRemote jobs in the USRemote jobs in PolandRemote jobs in the UKRemote jobs in CanadaRemote jobs in Mexico
Product
Jobs open to youA CV for each jobMemoryInterview helpHow it works
Company
FAQContact
Legal
Privacy PolicyTerms & Conditions
Remote jobs by country
United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand

Remote jobs by country

United StatesPolandUnited KingdomCanadaMexicoBrazilGermanySpainPortugalIrelandNetherlandsIndiaSwedenBelgiumAustriaDenmarkFinlandSwitzerlandNorwayAustraliaNew Zealand
© 2026 First MondayNo tracking cookies. Anonymous page statistics only.
First Monday
Jobs/Engineering/Software Development Engineer in Test – AI

Software Development Engineer in Test – AI

Newfold Digital

Remote · IndiaEngineeringPosted November 2, 2026
Open to people in IndiaApply for this job Write my CV for this job Early access
Pay
Not stated
Required
Python, SQL, Docker
Nice to have
Robot Framework, Playwright
Benefits
Health insurance, Paid time off
Posted
November 2, 2026
Company
Newfold Digital

Who it can hire

We read the post for each of the 21 countries we cover.

  • Open to people in India

    The employer lists this job as remote there: “India - Remote”.

  • Not open to people in the other 20 countries we cover, as far as the post says.

The job post

Who we are.

Newfold Digital is a leading web technology company serving millions of customers globally. Our customers know us through our robust portfolio of brands. We have some of the industry's most prominent and storied go-to-market brands, including Bluehost, HostGator, Domain.com, Network Solutions, Register.com and Web.com. We help customers of all sizes build a digital presence that delivers results. With our extensive product offerings and personalized support, we take pride in collaborating with our customers to serve their online presence needs. The strength of our company lives in the intersection of our people, our customers, and our brands.


Role Summary:

This role goes beyond traditional QA; the ideal candidate should be capable of validating, evaluating, and improving AI-powered systems, agents, copilots, and LLM-driven workflows while building scalable automation frameworks and intelligent quality solutions. You will work closely with Engineering, AI/ML, Platform, Product, and DevOps teams to ensure reliability, safety, observability, scalability, and quality of modern AI-native applications and Agentic systems.


What you'll do & how you'll make your mark.

1. AI/LLM Quality Engineering

  • Design, validate, and optimize testing strategies for AI-powered applications, copilots, chatbots, RAG pipelines, and autonomous AI agents.
  • Develop AI evaluation (Evals) frameworks for: Hallucination detection Groundedness Toxicity & bias validation Response quality Context relevance Prompt regression testing Determinism & consistency Safety and guardrail validation
  • Build automated validation pipelines for LLM outputs using structured assertions, semantic scoring, embeddings, and evaluation metrics.
  • Create synthetic datasets and benchmark datasets for AI testing scenarios.
  • Test multi-agent orchestration workflows and agent memory/context retention.
  • Validate MCP (Model Context Protocol) integrations and tool interoperability.
  • Work on Context Engineering, Prompt Engineering, and Prompt Optimization strategies.
  • Evaluate latency, token usage, model cost optimization, throughput, and scalability.

2. Automation Engineering

  • Develop and maintain scalable UI/API automation frameworks using: Python (mandatory)Robot Framework (preferred)Playwright/Selenium/Cypress.
  • Automate testing for: REST APIs GraphQL APIs Event-driven systems Streaming systems Webhooks Microservices MFE (Micro Frontend) architectures.
  • Build reusable automation libraries, utilities, and AI-assisted testing workflows.
  • Integrate automation into CI/CD pipelines and quality gates.

3. AI Engineering & Platform Validation

  • Validate RAG architectures including Chunking strategies Embedding quality Retrieval accuracy Re-ranking Vector search relevance
  • Work with vector databases such as: Pinecone Weaviate ChromaDB Qdrant FAISS
  • Build or validate AI agents leveraging: LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel OpenAI SDKs
  • Validate AI workflows involving: Tool calling Function calling Agent memory Agent planning/execution Multi-modal AI systems Structured outputs

4. DevOps, Observability & Reliability

  • Build and maintain quality workflows integrated with: GitHub Actions Jenkins GitLab CI/CD Azure DevOps
  • Work with Docker and Kubernetes environments.
  • Perform cloud-based testing and deployments across AWS/Azure/OCI/GCP.
  • Validate distributed systems reliability and resiliency.
  • Monitor AI system behaviors using observability platforms/logging/tracing tools.
  • Contribute to shift-left and quality engineering best practices.

Who you are & what you'll need to succeed.

  • Bachelor’s/Master’s degree in Computer Science, Engineering, AI/ML, or related field preferred.
  • Equivalent hands-on experience and demonstrated projects are highly valued.
  • Strong programming experience in Python
  • Strong understanding of Software Testing & Quality Engineering principles
  • Hands-on experience with: AI/LLM testing Prompt Engineering Context Engineering AI Evals RAG systems AI Agents
  • Experience with: CI/CD pipelines Docker & KubernetesCloud platforms (AWS/Azure/OCI/GCP) API testing & automation Microservices testing UI automation
  • Experience with SQL and data validation
  • Strong debugging, analytical, and troubleshooting skills
  • Experience working in Agile/Scrum environments

Good to have skills:

  • Experience with Robot Framework
  • Experience with Playwright
  • Knowledge of: LangChain, LangGraph, LlamaIndex CrewAI Semantic Kernel MCP (Model Context Protocol)
  • Hands-on experience with: Cursor GitHub Copilot Claude Gemini OpenAI APIs
  • Experience with vector databases and embeddings
  • Exposure to MLOps workflows and ML lifecycle testing
  • Knowledge of performance testing and security testing
  • Experience validating AI governance/compliance requirements
  • Experience with contract testing, chaos testing, or resilience engineering
  • Familiarity with accessibility testing and cross-browser/device testing
  • Exposure to synthetic monitoring and production observability

Bonus Skills (High Impact)

  • AI Red Teaming
  • LLM Security Testing (Prompt Injection, Jailbreaks, Data Leakage)
  • Agent Reliability & Safety Testing
  • AI Cost Optimization
  • Distributed Systems Testing
  • Test Data Engineering
  • Feature Flag Testing
  • AI Observability & Telemetry
  • Reinforcement Learning feedback workflows
  • Human-in-the-loop evaluation systems
  • Knowledge Graph integrations
  • Multi-modal AI testing (vision/audio/text)

What We’re Looking For

  • Engineers who actively experiment with AI tools and build side projects, prototypes, PoCs, or open-source contributions.
  • Strong problem solvers who can combine traditional QE practices with modern AI-native engineering approaches.
  • Passion for innovation, developer productivity, automation acceleration, and intelligent testing.
  • Ability to rapidly learn emerging AI frameworks, tools, and architectures.

Nice Portfolio Signals

Candidates with one or more of the following will stand out:

  • GitHub repositories showcasing AI/automation projects
  • AI agents or copilots built personally or professionally
  • Contributions to open-source automation or AI ecosystems
  • Blogs, talks, workshops, or technical content on AI/testing
  • Experience integrating AI into SDLC or QE workflows
  • Experience with autonomous testing systems or self-healing automation

Why you'll love us.

  • Work-life balance. Our work is thrilling and meaningful, but we know balance is key to living well.

  • We celebrate one another’s differences. We’re proud of our culture of diversity and inclusion. We foster a culture of belonging. Our company and customers benefit when employees bring their authentic selves to work. We have programs that bring us together on important issues and provide learning and development opportunities for all employees. We have 20 + affinity groups where you can network and connect with Newfolders globally.

  • We care about you. We are a family, and we care about you and your family’s physical and mental health by providing competitive HMO benefits – 175k MBL with one free dependent upon one year of service! We also give out Punctuality Bonus, Generous Vacation policy, and much more!

  • Where can we take you? We’re fans of helping our employees learn different aspects of the business, be challenged with new tasks, be mentored, and grow their careers. Unfold new possibilities with #teamnewfold!

This Job Description includes the essential job functions required to perform the job described above, as well as additional duties and responsibilities. This Job Description is not an exhaustive list of all functions that the employee performing this job may be required to perform. The Company reserves the right to revise the Job Description at any time, and to require the employee to perform functions in addition to those listed above.

First Monday isn’t the employer or a recruiter and isn’t affiliated with Newfold Digital. Details may have changed since; confirm with the employer before you apply.

Open to people in IndiaApply for this job Write my CV for this job Early access
Pay
Not stated
Required
Python, SQL, Docker
Nice to have
Robot Framework, Playwright
Benefits
Health insurance, Paid time off
Posted
November 2, 2026
Company
Newfold Digital

Similar jobs open to people in India

QA- Test EngineerUvation · IndiaAI Software EngineerRolling Arrays Technologies · IndiaAI Developer (LLM Products)dexter health · Remote · WorldwideSSoftware Engineer – AISynthio Labs · Remote · India
First Monday

Get jobs like this every Monday.

One email a week with new remote jobs open to people in Canada. Free, and one click to unsubscribe. Privacy Policy

Or request early access to the app
Apply for this job