We are seeking a Senior ML Data Processing Developer to participate in the development, curation, and scaling of our core data asset pipeline. Sitting at the intersection of data engineering, data curation, and machine learning, you will own the end-to-end pipeline that transforms raw web-scale data into high-signal datasets used to train the Scientist AI. In this role, you will not just manage data; you will engineer its quality.

You will design algorithmic filtering, build model-based scoring mechanics, and ensure rigorous benchmark integrity to power the next generation of AI. And as our models push beyond established paradigms, you will design and implement novel data transformations that don't yet have playbooks, working at the frontier of what training data can be.

Responsibilities

may be distributed across the team based on individual experience, skills, and interests.

Key Responsibilities

Partner with the Research team to define, build, automate, scale, and manage data pipelines that transform raw web-scale data into training datasets for the Scientist AI. Build and maintain data processing pipelines, including deduplication, model-based quality scoring, heuristic filtering, toxicity removal, PII scrubbing, metadata extraction, and proprietary data transformations, with full dataset versioning and provenance tracking, optimizing for throughput and cost at scale.

Requirements

, internal Data Governance policies, and legal obligations. Develop and refine the scoring and filtering toolchain: heuristics, LLM-as-a-judge evaluators, ML classifiers, metadata extraction modules, and human-in-the-loop review workflows required for data processing and quality assurance. Instrument data processing pipelines with data-quality monitoring, guardrails, and alerting to catch regressions before they propagate downstream.

, then identify and acquire large-scale text corpora that meet those

This includes conducting systematic coverage analyses to identify gaps in the corpus and develop targeted acquisition strategies to address them, and working with the Legal & Governance Team to license new data sources. Design and maintain strict leakage detection mechanisms to guard against evaluation contamination across all stages of the data processing pipeline. Build internal tooling and interfaces that let researchers explore, query, and understand available datasets with minimal friction.

into concrete pipeline work. Strong Python proficiency, including experience writing production-grade data-processing code. , Airflow, Prefect, Dagster).

Qualifications

Degree in computer science, software engineering, or a related field. Proven track record of handling massive unstructured text datasets (trillion-token scale), with 5+ years of experience in data processing, machine learning engineering or Natural Language Processing (NLP). , Spark, Ray, Flink), designing and optimizing high-throughput pipelines.

Experience with data privacy implementation (PII scrubbing), content-safety filtering (toxicity, bias), and evaluation-contamination prevention.

Nice To Have

g. vLLM, SGLang). Familiarity with containerized deployment (Docker, Kubernetes) and infrastructure-as-code practices.

g. Weights and Biases). Experience with data licensing workflows or web-scale data acquisition.