Mid level Quality Engineer, AI-Native

Job Description

Job Description: Insight Global is seeking a Quality Engineer, AI-Native for a top FinTech / intelligent automation client. This candidate will play a critical role in validating AI-driven workflows, agent behavior, and prompt outputs in an environment where testing goes far beyond traditional QA. They will help ensure that business-critical automation systems behave reliably, even when small changes create large output variation. The ideal candidate brings strong automation testing experience, comfort with ambiguity, and a modern approach to quality engineering in AI-powered systems.

Day-to-Day:

Test and validate AI/agentic systems and business workflows Evaluate prompt outputs for consistency, reliability, and decision accuracy Build and execute automated test strategies for probabilistic systems Validate database interactions, API flows, and agent behaviors across workflows Detect inconsistencies caused by prompt or workflow changes Measure deviations in system performance and support regression testing efforts Partner with engineering teams to ensure AI-enabled workflows meet accuracy and quality targets Help define scalable QA approaches for testing prompts and agent behaviors at scale

Skills and Requirements

Strong QA/SDET background with automation experience Experience with API and UI testing Hands-on experience using Playwright or similar modern QA automation frameworks Ability to validate complex systems and test ambiguity/non-deterministic outputs Experience testing database interactions and backend workflows Understanding of prompt engineering, prompt output validation, or AI system behavior Ability to think statistically about validation, regression, variation, and reliability Strong communication skills around testing strategy and quality risks Experience testing AI/LLM systems or agentic workflows Familiarity with LangChain, LangGraph, or LangSmith Experience with model evaluation, prompt testing, or AI quality monitoring at scale Background in data validation, complex workflow testing, or intelligent automation Understanding of regulated or business-critical workflows such as underwriting, funding, or support operations

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [email protected].