- Electronic Arts
- Vancouver, BC
- 17 days ago
Software Engineer III - AI/ML.
Before you go
Before you leave us, sign up for our email alerts
We don't do job spam, just the best digital jobs delivered straight to your inbox.
Software Engineer III - AI/ML: our view in 3 lines...
- The Role:This role is for a software engineer who builds automated testing systems and evaluation tools for a football video game.
- The Person:The person will build automated testing frameworks, validation checks, and CI/CD pipeline integrations, develop evaluation frameworks for generative AI and large language model systems, and collaborate with engineering, design, and quality partners to review code and resolve blockers.
- Requirements:The ideal candidate has 5+ years of professional software engineering experience with Python, algorithms, data structures, automated testing, version control, LLMs, generative AI and NLP.
About the role
Electronic Arts creates next-level entertainment experiences that inspire players and fans around the world. Here, everyone is part of the story. Part of a community that connects across the globe. A place where creativity thrives, new perspectives are invited, and ideas matter. A team where everyone makes play happen.
Description
In FC Quality Verification Engineering, our mission is to help our teams identify quality issues earlier. We build automation frameworks and developer tools that combine software development and applied AI/ML to improve how we test EA-SPORTS FC.
Role summary
You will work with us to build automated testing frameworks and evaluation tools, applying AI and machine learning to expand our test coverage. You will report to our Engineering Manager. You will work hybrid, at least 3 days a week, from our office in Burnaby, BC.
Responsibilities
-
You will build automated testing frameworks, validation checks, and CI/CD pipeline integrations with reproducible diagnostics.
-
Develop methods that help us explore complex software behaviour, identify coverage gaps, and detect defects.
-
Build automated checks to validate software behaviour and data against requirements.
-
Develop evaluation frameworks for generative AI and LLMs-based systems (e.g., implementing LLMs-as-a-judge patterns, rubric-based scoring, and automated calibration) alongside deterministic validation checks.
-
Apply generative AI, NLP, and machine learning, balancing heuristic, deterministic, and model-based approaches to solve complex software testing problems.
-
Integrate testing tools into development pipelines, producing reproducible diagnostics, and support the software you deliver.
-
Prototype latest machine learning and LLMs capabilities, and collaborate with engineering, design, and quality partners to clarify requirements, review code, and resolve blockers.
Required qualifications
-
5+ years of professional software engineering experience, including 2+ years of hands-on experience building, integrating, or evaluating AI/ML and LLMs-based systems.
-
A Master of Science in Computer Science, Mathematics or a related field, or equivalent professional engineering experience.
-
In-depth experience with LLMs, generative AI, and NLP concepts, architectures, and failure modes.
-
Experience developing evaluation frameworks for generative systems, including dataset curation, model-as-a-judge design, validation criteria, error analysis, and quantitative/qualitative metrics.
-
Proficiency in Python, algorithms, data structures, modular design, automated testing, debugging, and version control.
-
Experience preparing datasets, defining validation criteria, and performing error analysis using quantitative ML metrics.
-
Experience with natural language processing or large language models, such as prompting, fine-tuning, or retrieval-augmented generation.
Pay Transparency - North America
* British Columbia (depending on location e.g. Vancouver vs. Victoria) *$122,300 - $170,700 CAD

