Skip to main content
Wolters Kluwer logo

Lead Software Test Automation Engineer | Test Architect - Agentic AI

Wolters Kluwer

Greenwood Village, COJob$90–157K/yrSeen 3w agoIn the employer's feed · checked in the last 2 days

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
$90–157K/yr
Location
Greenwood Village, CO
Work Authorization
Not specified

Olive focuses on jobs in the United States, including remote roles you can work from there.

Job overview

Wolters Kluwer seeks a Lead Test Automation Architect to define enterprise test automation strategy across UI, API, data, and platform layers, while evolving quality engineering for Agentic AI and LLM-based systems. The role architects scalable frameworks, improves automation productivity and maintainability, establishes quality gates, and provides technical leadership. It also develops approaches for evaluating probabilistic AI outputs and partners with ML/AI, Product, and Engineering on measurable quality criteria.

Skills & qualifications

RequiredNice to have

Skills

Test Automation Architecture, requiredAPI Testing, requiredUI Automation, requiredTest Data Strategy, requiredEnvironment Reliability, requiredJava, requiredPython, requiredTypeScript, requiredJavaScript, requiredC#, requiredPlaywright, requiredCypress, requiredSelenium, requiredREST-Assured, requiredSuperTest, requiredPostman, requiredNewman, requiredPytest, requiredGitHub Actions, requiredJenkins, requiredAzure DevOps, requiredFlakiness Reduction, requiredExecution Speed Optimization, requiredLLM Testing, nice to haveAgentic AI Testing, nice to haveEvaluation Harnesses, nice to haveBenchmark Datasets, nice to haveSemantic Scoring, nice to haveDrift Detection, nice to havePrompt Regression Testing, nice to haveStructured Evaluation Reports, nice to haveTest Pyramid, nice to haveContract Testing, nice to haveConsumer-Driven Contracts, nice to haveRisk-Based Testing, nice to haveShift-Left Testing, nice to haveSecurity Testing, nice to havePerformance Testing, nice to haveReusable Automation Libraries, nice to have

Qualifications

8+ Years Quality Engineering, requiredArchitect-Level Framework Ownership, required

Benefits

Medical Insurance
Dental Insurance
Vision Insurance
401(k) Match
Tuition Assistance
Paid Time Off
Parental Leave

Full job description

We are seeking a Lead Test Automation Architect to define and lead an enterprise-grade test automation strategy across UI, API, data, and platform layers—while evolving our quality engineering capabilities to support Agentic AI / LLM-based systems (probabilistic outputs, confidence-based validation, hallucination detection, and model evaluation). This role will architect scalable automation frameworks, drive engineering excellence, and accelerate delivery through highly productive, maintainable automation programs.

Key Responsibilities

Automation Strategy & Architecture

Define and own the test automation architecture and multi-layer strategy (UI/API/contract/integration/e2e), aligned to product risk and release cadence.

Establish automation standards, patterns, and reference implementations (framework structure, page-object/screenplay usage, test data patterns, reporting).

Drive shift-left testing practices (contract testing, component testing, pre-merge validation) and reduce reliance on fragile end-to-end tests.

Agentic AI / LLM Test Model Alignment

Design and implement testing approaches for non-deterministic AI outputs , including:

confidence/threshold-based assertions

semantic similarity checks and rubric-based validation

goldens/benchmark datasets and evaluation harnesses

hallucination and groundedness testing strategies

regression detection across model/prompt/version changes

Define AI test coverage across accuracy, robustness, bias/safety (as applicable), performance, and reliability .

Partner with ML/AI, Product, and Engineering to establish acceptance criteria for AI capabilities and measurable quality gates.

High-Productivity Automation Programs

Build and scale high-throughput automation projects that are reliable, fast, and easy to extend.

Improve test execution time , reduce flaky tests, and optimize CI/CD pipelines (parallelization, test selection, retries with guardrails).

Create actionable, engineering-friendly quality dashboards (pass/fail trends, flake rate, coverage, defect escape metrics).

Tooling, CI/CD, and Quality Gates

Integrate automation into CI/CD with quality gates (PR checks, nightly suites, release readiness criteria).

Standardize observability for testing : logs, tracing hooks, screenshots/video, artifacts, and test run diagnostics.

Define practices for test environments , data management, service virtualization, and dependency control.

Technical Leadership & Mentorship

Serve as the QE technical authority: coach engineers on automation design, code quality, and best practices.

Lead design reviews for test architecture and automation PRs; enforce coding standards and maintainability.

Influence roadmap and delivery by translating risk into clear testing priorities.

Required Qualifications

8–12+ years in Quality Engineering / Test Automation with architect-level ownership of frameworks and strategy.

Proven experience designing scalable automation across:

API testing (contract + integration)

UI automation with stable locator strategies (test IDs, accessibility roles)

Test data strategy and environment reliability

Strong programming skills in at least one modern language (e.g., Java, Python, TypeScript/JavaScript, C# ).

Expertise with automation tools/frameworks (examples):

UI: Playwright / Cypress / Selenium

API: REST-assured / SuperTest / Postman/Newman / pytest

CI: GitHub Actions / Jenkins / Azure DevOps

Demonstrated ability to reduce flakiness and improve execution speed through engineering solutions .

Preferred Qualifications (Strongly Desired)

Experience testing LLM/Agentic AI features or other probabilistic systems, including:

evaluation harnesses, benchmark datasets, semantic scoring, and drift detection

prompt/version regression strategies and structured evaluation reports

Familiarity with modern quality approaches: test pyramid , contract testing , consumer-driven contracts , risk-based testing , shift-left .

Knowledge of security/performance testing integration (baseline understanding; hands-on is a plus).

Experience building reusable automation libraries for multiple teams/products.

Our Interview Practices

To maintain a fair and genuine hiring process, we kindly ask that all candidates participate in interviews without the assistance of AI tools or external prompts. Our interview process is designed to assess your individual skills, experiences, and communication style. We value authenticity and want to ensure we’re getting to know you—not a digital assistant. To help maintain this integrity, we ask to remove virtual backgrounds and include in-person interviews in our hiring process. Please note that use of AI-generated responses or third-party support during interviews will be grounds for disqualification from the recruitment process.

Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.

Compensation:

$89,600.00 - $157,000.00 USD

This role is eligible for Bonus.

Compensation range listed is based on primary location of the position. Actual base salary offer is influenced by a wide array of factors including but not limited to skills, experience and actual hiring location. Your recruiter can share more information about the specific offer for the job location during the hiring process.

Additional Information :

Wolters Kluwer offers a wide variety of competitive benefits and programs to help meet your needs and balance your work and personal life, including but not limited to: Medical, Dental, & Vision Plans, 401(k), FSA/HSA, Commuter Benefits, Tuition Assistance Plan, Vacation and Sick Time, and Paid Parental Leave. Full details of our benefits are available upon request.

EQUAL EMPLOYMENT OPPORTUNITY Wolters Kluwer U. S. Corporation and all of its subsidiaries, divisions and customer/business units is an Equal Opportunity / Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or protected veteran status.

How this job compares

Set against the 2,794 open Quality Engineer jobs on Olive Jobs, counted November 22, 2026.

Pay
  • This posting$90–157K/yr
  • Colorado median$94–147K/yr67 salary ranges
  • US median$90–149K/yr1,880 salary ranges

This range's midpoint, $123K, is 5% above the Colorado median midpoint ($118K) — in the middle half of Quality Engineer salary ranges in Colorado.

Openings
Olive Jobs lists 75 open Quality Engineer jobs in Colorado and 2,794 across the US. 2% of them nationwide are listed as remote or flexible.
Hiring the most in Colorado
Experience
This posting asks for at least 8 years of experience — more than three in four Quality Engineer postings that state a minimum ask for (median 5 years, across 2,253 postings).
Education
1,328 of the 2,794 open Quality Engineer jobs (48%) ask for a bachelor's degree or higher.

Counted across the open, public US postings on Olive Jobs on November 22, 2026. Figures based on too few postings are left out.

Prepare for this role

You've read the whole posting — now see how you match it.