← All roles

AI-Enabled QA Engineer

Scale Army· Technology & Product
Apply on company site
RemoteContractPosted 4w ago

About the role

QA engineer focused on testing AI-enabled tools supporting acquisitions, financial analysis, approval workflows, and brand assets. Responsible for automated checks, edge-case testing, AI output validation, and defining production-readiness.

Responsibilities

  • Build test automation using Playwright, Cypress, or Selenium for interfaces and pytest or equivalent for underlying systems.
  • Design test cases from specifications using equivalence classes, boundaries, negative cases, and cases not covered by the original specification.
  • Run tests through CI pipelines so checks execute automatically.
  • Track and triage defects using Jira, Linear, or Azure DevOps.
  • Write actionable bug reports that engineers can use without requiring additional meetings.
  • Use SQL to independently verify data through joins, aggregation, and record-level investigation.
  • Validate data at scale through row counts, duplicate detection, orphaned records, referential integrity, and reconciliation of totals.
  • Test APIs using Postman or code.
  • Validate status codes, payload structures, authentication failures, and behavior on repeated identical requests.
  • Evaluate AI-generated output using a fixed set of examples.
  • Measure precision and recall using historical cases.
  • Run regression tests following prompt changes.
  • Check AI-generated output for fabricated facts.
  • Verify that fields extracted from financial statements can be traced back to their source documents.
  • Test document extraction against scenarios such as scanned documents, rotated pages, and missing signatures.
  • Verify target-analysis summaries against source financials so every number in an AI-generated summary is traceable to a document.
  • Test the diligence data room against cases including missing documents, duplicate uploads, and rooms that appear complete but are not.
  • Test notifications to ensure they trigger when a room stalls without generating unnecessary alerts.
  • Test the approval step on the priority tracker to ensure nothing is submitted without review, including attempts to bypass the approval process.
  • Write test cases for AI-generated outputs that every change must continue to pass.
  • Check generated brand assets across eight to ten new brands.
  • Verify that each generated asset contains the correct brand name, logo, and colours.
  • Ensure assets do not contain branding belonging to another brand.
  • Develop the production-ready checklist with the Product Manager and Head of IT.
  • Apply the production-ready checklist to the first tool.

Qualifications

  • 3+ years of QA experience across both manual and automated testing.
  • Experience testing data-heavy applications rather than only user interfaces.
  • Experience with test automation in code.
  • Experience evaluating AI-generated outputs methodically.
  • Experience with data validation and API testing.
  • Experience testing document-heavy or extraction-based systems is a plus.
  • Experience checking creative or brand output for consistency is a plus.
  • Healthcare data experience is a plus.
  • Finance or accounting testing experience is a plus.
  • Test automation experience within a Microsoft environment is a plus.
  • Experience creating a definition of done that an organization adopted is a plus.

Skills

Required:Playwright/Cypress/Seleniumpytest or equivalentSQLAPI testing (Postman or code)Test designData validation and data reconciliationAI output evaluationDocument extraction testingCI/CD testingDefect tracking toolsClear bug reportingDocumentation
Last updated Sep 23, 2026. Interested? Apply on the company site.

About Scale Army

Global Staffing for Marketing & Sales Growth

Scale Army sources, vets, and deploys high-performing marketers and sellers, helping meet growth targets with nearshore talent, month-to-month contracts, and fast onboarding.

staffingmarketingsalesgrowthnearshoreoffshoretalent sourcing
Company profileWebsite

More open roles at Scale Army