Lead QA Engineer – Agentic Platforms
Full-Time
Work Arrangement: Remote

About the Role

RVStaffing is seeking an experienced Lead QA Engineer with hands-on experience testing AI agents, LLM-based applications, and agentic workflows. This role will focus on ensuring the quality, reliability, and consistency of AI-driven systems through structured testing, defect investigation, and continuous improvement of QA processes.

The ideal candidate combines strong software testing fundamentals with practical experience validating AI-powered applications, identifying unexpected behavior, and collaborating with engineering and product teams to deliver reliable solutions.

Please note: This position requires flexible working hours to support scheduled releases every two weeks, beginning at approximately 9:00 PM Pacific Standard Time (PST). Availability during these release windows is required.

What You'll Do

AI & Agentic Testing: Design and execute test cases to validate AI agents, LLM-based applications, and agentic workflows, including expected behavior, edge cases, failure scenarios, and consistency of results.
Test Planning & Execution: Develop and execute manual and automated test cases for new features, functional requirements, and regression testing.
Quality Assurance Leadership: Identify gaps in test coverage, recommend process improvements, and strengthen QA practices across the development lifecycle.
Defect Management: Identify, document, prioritize, and track defects with clear reproduction steps, expected versus actual results, and severity levels.
Automation Support: Develop, maintain, or improve automated test scripts using frameworks such as Playwright, Selenium WebDriver, Cypress, or pytest.
API Testing: Perform manual and automated API testing to validate backend services, integrations, and application behavior.
Data Validation: Use database queries and data investigation techniques to validate application behavior and troubleshoot defects.
Cross-Functional Collaboration: Work closely with developers and product stakeholders to clarify requirements, troubleshoot issues, and ensure acceptance criteria are met.
Release Support: Participate in biweekly release cycles, including evening release windows, to validate functionality and support release quality.

What You'll Bring to the Team

Bachelor's degree in Computer Science, Information Technology, or a related field, or equivalent practical experience.
3+ years of professional experience in software quality assurance.
Demonstrated hands-on experience testing AI agents, LLM-based applications, or agentic workflows is required.
Understanding of AI-specific testing challenges, including non-deterministic outputs, response consistency, edge cases, and failure handling.
Strong understanding of manual testing methodologies, test case design, defect lifecycle management, and regression testing.
Experience with functional, integration, API, and cross-browser testing.
Experience identifying QA process improvements and gaps in test coverage.
Proficiency with bug-tracking and test-management tools such as Jira and TestRail.
Basic database skills for validating data and investigating defects, including SQL, MongoDB, and Databricks.
Familiarity with Git and automated testing within CI/CD pipelines.
Experience with at least one test automation framework, such as Playwright, Selenium WebDriver, Cypress, or pytest.
Excellent written and verbal communication skills, with the ability to explain technical issues to both technical and non-technical stakeholders.
Ability to work independently in a remote environment with minimal supervision.
Availability for biweekly releases beginning around 9:00 PM PST and flexibility to work during these release windows are required.

What We're Looking For

A proactive QA professional who combines traditional software testing expertise with practical AI testing experience. The successful candidate will take ownership of product quality, investigate unpredictable AI behavior, collaborate effectively across teams, and help ensure reliable releases in a fast-moving engineering environment.

At RV Holdings S.A.S., we promote an inclusive, diverse, and respectful work environment. We value talent regardless of race, ethnicity, gender, sexual orientation, gender identity, age, beliefs, nationality, or disability status. All selection processes are conducted based on the competencies, skills, and experience of each individual.