Senior Automation QA Engineer
- Pay
- 120–130 PLN an hour
- Required
- Python, TypeScript, Playwright, GitLab CI, CI/CD tools, AI-based agents
- Nice to have
- Docker
- Experience
- 5+ years
- Posted
- November 16, 2026
- Company
- AVENGA
Who it can hire
We read the post for each of the 21 countries we cover.
Open to people in Poland
The employer lists this job as remote there: “Poland”.
Not open to people in the other 20 countries we cover, as far as the post says.
The job post
We are looking for a Senior Automation QA Engineer to join a project focused on developing an enterprise AI agent development platform. The platform enables teams to build, deploy, and manage AI agents in a secure, scalable cloud environment. It provides isolated workspaces, browser-based development environments, automated agent deployment, an agent marketplace, and real-time monitoring.
- 5+ years of experience in software quality assurance and test automation.
- Proven experience defining test strategies for web platforms or SaaS products.
- Experience setting up and owning test automation repositories using Python or TypeScript.
- Strong hands-on experience with REST API test automation, including schema validation, asynchronous workflows, authentication, and access control.
- Practical experience with Playwright and end-to-end testing of React or other SPA applications.
- Experience integrating automated tests into GitLab CI or similar CI/CD tools, preferably with Docker.
- Experience testing cloud-based applications and troubleshooting issues using logs, metrics, and traces.
- Understanding of AI-based agents, including LLM behaviour, tools, memory, and non-deterministic outputs.
- Daily experience using AI coding assistants such as GitHub Copilot or Claude Code, with the ability to critically review AI-generated code.
- Strong analytical, problem-solving, and communication skills.
Nice to Have
- Experience with AWS services such as Lambda, API Gateway, DynamoDB, S3, CloudWatch, or Bedrock AgentCore.
- Familiarity with AI agent frameworks such as LangGraph, AWS Strands, or CrewAI.
- Knowledge of MCP and A2A protocols.
- Experience with LLM evaluation tools such as DeepEval, Promptfoo, or Ragas.
- Knowledge of vector databases and retrieval systems.
- Experience with monitoring and observability tools.
- Knowledge of OWASP Top 10 and security testing tools such as Burp Suite or OWASP ZAP.
- Experience working in regulated or GxP environments.
- Define and maintain the test strategy and test pyramid in collaboration with developers and platform engineers.
- Set up, develop, and maintain the test automation repository, including its structure, standards, shared fixtures, and reporting.
- Build and maintain automated REST API tests, covering functional, schema, negative, and regression scenarios.
- Develop and maintain stable end-to-end UI tests using Playwright.
- Test asynchronous processes, including agent deployments, environment provisioning, and WebSocket log streaming.
- Perform integration and smoke testing of AI agents, including authentication, API endpoints, MCP tool calls, logs, traces, and memory persistence.
- Integrate automated tests into CI/CD pipelines and implement quality gates, nightly regression runs, and post-deployment smoke tests.
- Collaborate with development, AI, and platform teams to investigate issues and improve overall software quality.
First Monday isn’t the employer or a recruiter and isn’t affiliated with AVENGA. Details may have changed since; confirm with the employer before you apply.