HOME Software Engineering & Dev Software Engineer, Systems — AI Training Data Infrastructure
  • Meta
  • Bellevue, WA
  • Full-Time
  • 16 days ago
Meta VERIFIED EMPLOYER

Software Engineer, Systems — AI Training Data Infrastructure.

Software Engineering & Dev Full-Time

Software Engineer, Systems — AI Training Data Infrastructure: our view in 3 lines...

  • The Role:This role is for a software engineer building AI training data infrastructure for large-scale distributed training systems.
  • The Person:The person will own the AIRStore data path, improve throughput and latency, build multi-region and multi-cloud dataset access, manage data lifecycle correctness, and handle oncall and incident root-cause analysis.
  • Requirements:The ideal candidate has a bachelor’s degree in Computer Science or a related technical field, 5+ years building production distributed systems or large-scale data infrastructure, and proficiency in C++, Rust or Go plus Python.

About the role

We own the dataset layer that Meta's largest AI training runs read from. AIRStore and the Anywhere Training substrate are what let a multi-petabyte dataset be written once and read at full throughput from any cluster, in any region, in any cloud — without a copy. Our customers are named model programs, not abstract services: when a training job's GPUs go idle waiting on I/O, or a dataset isn't where the scheduler put the job, that is our problem to own and fix.

In 2026 this team moved Anywhere Training blob pointers to 100% Manifold residency, cut AIRStore dataset startup time by 10×, drove the migration of AIRStore datasets onto a standard S3 interface, and reclaimed hundreds of petabytes through lifecycle work — all while holding the line on training reliability across dozens of production incidents.

Responsibilities

  • Own significant components of the AIRStore data path end to end — ingestion, metadata, client, and read path — from design through production operation
  • Attack throughput and latency as a first-class product concern: prefetching, parallelism, caching, and startup cost, measured in GPU utilization and training wall-clock rather than microbenchmarks
  • Build the multi-region and multi-cloud story: make dataset location invisible to the training job, across Meta data centers and third-party clouds
  • Get dataset lifecycle right — TTL, archival, expiration, and deletion — where the correctness bar is absolute in both directions: nothing a live run needs may disappear, and nothing that must be deleted may persist
  • Own a widely embedded client library responsibly: compatibility, rollout safety, and blast-radius control across thousands of callers you don't control
  • Take real operational ownership. Join the oncall rotation, drive root-cause analysis on incidents affecting production model training, and convert each one into a structural fix rather than a mitigation
  • Partner directly with AI research and training teams, Manifold, Warm Storage, Privacy, and Crypto to land changes that cross system boundaries
Minimum Qualifications

  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
  • 5+ years of experience building and operating production distributed systems or large-scale data infrastructure
  • Proficiency in a systems language — C++, Rust, or Go — plus Python
  • Demonstrated ability to diagnose performance problems in production: profiling, tracing, and reasoning about I/O, network, and concurrency behavior at scale
  • Experience owning a service in production, including oncall, incident response, and postmortem follow-through
  • Track record of designing and delivering a substantial system component with limited direction
  • BS/MS in Computer Science or equivalent practical experience
Preferred Qualifications

  • Experience with data lifecycle, retention, and privacy-driven deletion at scale
  • Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
  • Evidence of influencing technical direction beyond your immediate team
  • Background in ML data pipelines — dataloading, checkpointing, dataset versioning, or throughput-bound training I/O
  • Experience with storage systems: object/blob stores, distributed filesystems, caching and prefetch layers, or dataset/columnar formats
  • Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
  • Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
  • Experience running infrastructure across multiple cloud providers or hybrid environments
  • Familiarity with S3-compatible object storage APIs and the practical tradeoffs of compatibility layers

$154,003/year to $217,006/year + bonus + equity + benefits

Published September 18, 2026
Location Bellevue, WA
Job Type Full-Time