QA Engineer

Senior AI Quality Engineer — About the Role As a Senior AI Quality Engineer-, you will be responsible for the quality, reliability, and automated validation of next-generation on-device and hybrid AI agent experiences. Qira integrates intelligence directly into PCs, tablets, and mobile devices, requiring validation that extends beyond conventional functional testing. In this role, you will design test suites that evaluate non-deterministic agent workflows, build multi-device test automation frameworks, and ensure robust non-functional quality (performance, accessibility, internationalization, and security) across heterogeneous consumer hardware. What Will You Do
- Multi-Device & Platform Validation: Execute and automate cross-device test strategies across Windows PC, Android tablet, and mobile platforms, profiling local OS background daemons, IPC communications, memory footprints, and state synchronization across paired hardware.
- Agentic & Non-Deterministic AI Evaluation: Implement automated evaluation suites to benchmark non-deterministic LLM/SLM behaviors, measuring tool-calling trajectory correctness, hallucination rates, context relevance, and drift across model versions and prompt iterations.
- Test Automation Engineering: Develop scalable, resilient, and non-flaky test automation harnesses for multimodal AI interactions (voice, text, vision, and system UI controls) without relying on brittle screen coordinate matching or exact string assertions.
- Non-Functional Testing Execution: Automate performance benchmarking (latency, NPU/CPU consumption, battery drain, thermal profiling under load), security validation (indirect prompt injection and data boundary defenses), accessibility (WCAG 2.1/2.2 compliance, dynamic ARIA live regions), and internationalization (bi-directional layout and multilingual semantic quality).
- CI/CD Integration & Shift-Left Gating: Integrate test suites into automated CI/CD pipelines, establishing statistical quality gates and defect telemetry to catch regressions before firmware drops or software release freezes.
- Cross-Functional Collaboration: Partner with Software Engineering, Product Managers (PM), and PMO to triage complex non-deterministic issues, define release readiness criteria, and track defects through structured root-cause analysis.
Basic Qualifications
- Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent practical experience.
- 5+ years of software quality engineering and test automation experience across client platforms (Windows, Android, or Linux).
- 2+ years of hands-on experience validating AI/ML-driven applications, LLMs, or agentic systems (e.g., verifying tool invocations, RAG pipelines, or generative responses).
- Demonstrated proficiency in Python, C#, or Java/Kotlin, with experience building test automation frameworks from scratch (e.g., PyTest, Appium, WinAppDriver, Espresso).
- Hands-on experience executing non-functional testing: measuring device performance metrics (battery, memory, CPU/NPU usage), verifying accessibility trees, and validating localized interfaces.
- Working experience integrating automated tests into modern CI/CD ecosystems (e.g., GitHub Actions, GitLab CI, Jenkins, Azure DevOps).
Preferred Qualifications
- Experience using evaluation frameworks for generative AI (e.g., DeepEval, RAGAS, or custom LLM-as-a-judge pipelines).
- Familiarity with on-device hardware acceleration profiling (DirectML, ONNX Runtime, Qualcomm Snapdragon NPU, Intel NPU, or Android NNAPI) using tools like Perfetto or Windows Performance Toolkit.
- Experience running automated security red-teaming or adversarial prompt-injection testing against AI systems.
- Strong cross-functional communication skills, with a track record of partnering with Product and Program Management teams to make data-backed release decisions.