Test Engineer - AI and LLMs
Apply NowCompany: Architech
Location: Toronto, ON M4E 3Y1
Description:
Salary Range: 45-60,00 CAD/hr
Architech is a Toronto-based software company with 20 years of experience in creating technology solutions for clients across North America. We leverage the latest cloud technology and hire top talent to modernize applications so that businesses can succeed in today's digital world.
Our Dream Team has a main hub in Toronto, but expands across Canada and Krakw, Poland. Our team consists of over 100 certified technical experts in our Product, Design, Engineering, and Delivery disciplines. Our values drive our culture of success: Think Big, Be Open & Collaborate, Never Fail a Client, Grow Our People, Do the Right Thing, and Embrace Change.
Be Open & Collaborate: Our Culture Says It All
You'll work very closely with a diverse tight-knit group of creative and talented people who are passionate about technology, software, and solutions. Not only will you work in a collaborative and supportive environment, you'll also grow your existing skills while keeping up with technology trends.
Who We Are
We're passionate about creating an environment where every team member feels empowered to share their unique point of view. We celebrate diverse talents and encourage our teammates to share their whole selves - because our greatest source of inspiration is each other, and we believe diversity drives innovation.
In order to be inclusive, we must be intentional. We have taken a multi-pillar approach to D&I at Architech including: Listening & Learning, Being an Ally, and Accountability.
In 2020 we launched our first Diversity & Inclusion survey. While we are always striving for more equal representation, we are very proud of our results:
What Our People Say
"Employees of different backgrounds interact well within our company" - 97% of employees agree
"Architech respects individuals and values their differences" - 96% of employees agree
Welcome toArchitech.
Test Engineer - AI and LLMs
We are looking for a Test Engineer - AI and LLMs with a strong development backgroundand a passion fortesting AI-driven applications. This is not a traditional QA role-we need someone who understands automationtesting, software development principles, and Gen AI concepts. You will work in atelecom-focused environment, testing AI-driven solutions, ensuring quality, reliability, and performance of applications leveragingLLMs, transformers, and other AI models.
If you're ananalytical thinker, a problem solver, and a fast learnerwho thrives in acutting-edge AI space, this role is for you!
Key Responsibilities
Required Skills & Qualifications
Preferred Qualifications:
Architech is an equal opportunity employer committed to diversity. Should you require any accommodations prior to or during the interview process, please indicate this during the interview process. We strongly encourage applications from racialized people, people with disabilities, people from gender and sexually diverse communities and/or people with intersectional identities.
Architech is a Toronto-based software company with 20 years of experience in creating technology solutions for clients across North America. We leverage the latest cloud technology and hire top talent to modernize applications so that businesses can succeed in today's digital world.
Our Dream Team has a main hub in Toronto, but expands across Canada and Krakw, Poland. Our team consists of over 100 certified technical experts in our Product, Design, Engineering, and Delivery disciplines. Our values drive our culture of success: Think Big, Be Open & Collaborate, Never Fail a Client, Grow Our People, Do the Right Thing, and Embrace Change.
Be Open & Collaborate: Our Culture Says It All
You'll work very closely with a diverse tight-knit group of creative and talented people who are passionate about technology, software, and solutions. Not only will you work in a collaborative and supportive environment, you'll also grow your existing skills while keeping up with technology trends.
Who We Are
We're passionate about creating an environment where every team member feels empowered to share their unique point of view. We celebrate diverse talents and encourage our teammates to share their whole selves - because our greatest source of inspiration is each other, and we believe diversity drives innovation.
In order to be inclusive, we must be intentional. We have taken a multi-pillar approach to D&I at Architech including: Listening & Learning, Being an Ally, and Accountability.
In 2020 we launched our first Diversity & Inclusion survey. While we are always striving for more equal representation, we are very proud of our results:
- 31% women, 57% BIPOC, 14% LGBTQIA+
- 49% of our people were born in countries other than where our offices are located. Our team members collectively speak 19 different languages. 59% of our people speak more than one language
- In the past year Architech has increased the number of women in our technology function by 200%. We strive to do even better as our multi-year strategic plan unfolds.
- We analyzed salaries by gender of persons in the same role and are delighted to report a 0% gender pay gap in our delivery and technology roles!
What Our People Say
"Employees of different backgrounds interact well within our company" - 97% of employees agree
"Architech respects individuals and values their differences" - 96% of employees agree
Welcome toArchitech.
Test Engineer - AI and LLMs
We are looking for a Test Engineer - AI and LLMs with a strong development backgroundand a passion fortesting AI-driven applications. This is not a traditional QA role-we need someone who understands automationtesting, software development principles, and Gen AI concepts. You will work in atelecom-focused environment, testing AI-driven solutions, ensuring quality, reliability, and performance of applications leveragingLLMs, transformers, and other AI models.
If you're ananalytical thinker, a problem solver, and a fast learnerwho thrives in acutting-edge AI space, this role is for you!
Key Responsibilities
- Design, develop, and execute automated test casesfor AI-powered applications, ensuring high-quality releases.
- Develop and maintain test frameworksfor AI/ML-driven software components, including LLMs.
- Validate AI models' outputsby defining and executing test strategies for accuracy, consistency, and fairness.
- Collaborate with developers and data scientiststo identify edge cases, biases, and failure modes in AI models.
- Evaluate AI-driven workflowswithin telecom applications, ensuring compliance with functional and performance benchmarks.
- Continuously improve testing methodologiesfor AI/LLM applications, incorporating best practices in automation and validation.
- Document test results and provide feedbackto enhance AI models' robustness and reliability.
Required Skills & Qualifications
- 3-5 years of experienceinQA automation, software testing, or software development.
- Hands-on programming experiencewith Python, Java, or JavaScript.
- Strong experience in test automation(e.g., Selenium, Playwright, Cypress, or similar).
- Understanding of AI/ML concepts, particularlyLLMs, transformers, and Gen AI.
- Familiarity withAPI testing(Postman, REST Assured, etc.) and CI/CD pipelines.
- Strong analytical skills andproblem-solving mindset.
- Quick learnerwho can adapt to evolving AI technologies.
- Experience with cloud platforms(AWS, GCP, Azure) is a plus.
Preferred Qualifications:
- Exposure totesting AI models(e.g., prompt engineering, model evaluation).
- Familiarity withAI-powered test automation tools.
- Experience working in thetelecom sector(not required but a bonus).
- Knowledge ofAI ethics and bias testing.
Architech is an equal opportunity employer committed to diversity. Should you require any accommodations prior to or during the interview process, please indicate this during the interview process. We strongly encourage applications from racialized people, people with disabilities, people from gender and sexually diverse communities and/or people with intersectional identities.