Testing Engineer

Job Description

Build and maintain automated test suites for the Runability workflow. Implement Playwright-based end-to-end tests for key user journeys Build validation-path tests, especially around incomplete data and workflow blocking Build regression tests for AG-UI thread handlers Design and maintain the ground-truth test corpus used to validate AI and analytical behavior. Create golden input/output fixture computations. Build automated CI checks that compare produced results against ground truth. Ensure deliberate mutations or degraded outputs fail the test gate. Contribute to LLM-as-judge evaluation harnesses for AI-generated outputs Define scoring rubrics for AI outputs, including accuracy, completeness, consistency, actionability, and engineering relevance. Maintain eval score reports and support deployment gates based on minimum acceptable AI-quality thresholds. Validate authentication and authorization behavior where it affects testing Support cross-browser smoke tests, but this is secondary to automation and AI validation.

Skills and Requirements

Strong test automation experience. Playwright E2E testing. CI/CD test integration and deployment gating. Experience designing regression test suites for API-driven applications. Ability to create deterministic test fixtures and expected- Well design experience (optional)

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [email protected].