Gen AI QA Lead - RQ01552

Gen AI QA Lead - RQ01552

Duration: 10 months to start

Location: Hybrid - Downtown Toronto (2-3 days per week)

Reporting to the Products & Technology Quality Assurance Leader, this role presents an exciting opportunity to contribute towards cutting-edge Gen AI solutions that will shape how we deliver value to our clients and our people. The successful candidate will have an opportunity to work with a team of talented technology and business specialists to design and develop tailored Generative AI tools that transform key areas of our business and to foster a culture of innovation.

As a Gen AI Quality Assurance Lead, the successful candidate will be accountable for establishing quality assurance frameworks and processes to ensure Gen AI technology solutions undergo comprehensive and efficient testing. The successful candidate will be also overseeing a team that designs, implements, and maintains both manual and automated testing procedures, and conducts functional, system, integration, and performance testing.

Must Have Skills:

• 6+ years experience as a Sr. QA performing different types of testing for complex applications
• Experience leading technology QA teams
• Experience automated testing tools and platforms (Playwright, JMeter, SoapUI, postman)
• Experience with software development tools and defect tracking systems like Azure DevOps, Jira
• Database experience (SQL)
• Knowledge of Python for Gen AI development and testing
• Competent in designing and executing AI prompt engineering and testing scenarios, ensuring the robustness of AI and large language models
• Microsoft Copilot/Agents experience
• Working with high level UX principles such as user stories, persona building, prototype/prototype development approaches and information architecture and customer/product research methodologies (e.g., A/B testing and MVT)
• Experience with Agile methodologies

Nice To Have Skills:

• Undergraduate Degree in a technical field

Deliverables:

Team leadership (50%)

Lead and manage the Gen AI QA team, providing guidance, mentorship and support

Manage allocation of Gen AI QA resources to projects based on capacity and skillset

Develop, implement and enforce comprehensive testing frameworks, processes and tooling and continuously optimize them to enhance their effectiveness, speed and quality

Quality Assurance testing (50%)

Ensure the quality and reliability of Gen AI solutions through rigorous testing and validation

Contribute to insights, risk evaluations and estimation discussions to ensure alignment quality standards and that testing considerations are well-integrated into project planning

Analyze project requirements, user feedback, and architectural and design documents to ensure completeness and testability

Design, document and execute robust test strategies, test cases, scenarios, and scripts tailored to the specific requirements of Gen AI projects to ensure product quality

Utilize defect tracking system to record, trace, and drive the resolution of defects and issues discovered during testing

Investigate and determine the source of problems, which may include identifying test deficiencies or deviations from requirements

Assist in reproducing issues and conducting root cause analysis to facilitate swift resolution