Engagement Manager, Agentic AI Workflow Evaluations
Innodata Inc.
$75 - $85 / hour
Need a reasonable accommodation to apply or interview? Contact us.
JobMinglr uses automated technology to recommend jobs based on profile information and job preferences. Match Score does not determine eligibility for a position, prevent a user from viewing or applying to a job, or make hiring decisions on behalf of an employer.
Description
Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.
Scope of the Role:
We are standing up a dedicated onsite team to evaluate complex, real-world agentic AI workflows for a frontier AI customer. The Engagement Manager owns this team and this engagement: a group of ten reviewers and one QA lead working inside isolated test environments, assessing whether AI agents complete tasks safely, respect user intent and consent, and hold up under close scrutiny.
This is a delivery leadership role with a strong customer-facing component. You are the customer's primary day-to-day contact at the site, accountable for throughput, quality, and turnaround, and responsible for identifying where the engagement should grow next. You will not be scoring trajectories as a daily task, but you need enough technical depth to audit the team's work yourself and defend a scoring decision in a room with the customer's technical leads.
What You’ll Own:
- Own end-to-end delivery for the onsite team: throughput, quality, turnaround, and capacity against committed volumes
- Manage and develop a team of eleven, including hiring, onboarding, performance management, and retention
- Serve as the primary onsite point of contact for the customer's program and technical leads; run recurring reviews, report on delivery metrics, and resolve issues before they escalate
- Audit reviewer output directly — sample trajectories, check rubric application, and assess whether scoring rationale would survive customer review
- Partner with the QA lead on calibration cycles, quality metrics and evaluation, and rubric refinement; arbitrate disagreements that calibration does not resolve
- Own the escalation path for safety-relevant and ambiguous findings, including judgment on what reaches the customer and how quickly
- Identify expansion opportunities within the account: adjacent workstreams, new task types, additional capacity, and scope the work with internal delivery and commercial teams
- Forecast staffing and cost against the engagement's commercial model, and flag variance early
- Maintain information security, privacy, and facility access practices required by the customer's onsite environment
You’ll Thrive in This Role If You Have:
- Bachelor's degree or equivalent practical experience
- 5+ years of professional experience, including direct management of a delivery team of 10–20 people
- Track record as the customer-facing owner of a services or delivery engagement with an enterprise or technology client, including metrics reporting and issue escalation
- Significant experience with AI/ML evaluation work: annotation, red-teaming, RLHF, model or agent evaluation, or trust and safety review
- Ability to read rubric-scored work critically and form an independent view of whether a score is correct
- Strong written communication; able to produce reporting and escalation memos that a technical customer will accept without rework
The expected hourly salary range for this position is $75-85 p/hour, based on experience, skills, and qualifications.
Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission’s guide at https://consumer.ftc.gov/articles/job-scams.
If you believe you’ve been targeted by a recruitment scam, please report it to Innodata at verifyjoboffer@innodata.com and consider reporting it to the FTC at ReportFraud.ftc.gov.
About this role
Innodata is launching a dedicated onsite team in San Jose to evaluate how frontier AI agents perform on complex, real-world tasks—assessing safety, intent alignment, and robustness. As Engagement Manager, you'd own this team of eleven reviewers and a QA lead, managing both the people and the customer relationship for a major AI builder. This is a delivery leadership role where you're accountable for throughput, quality, and turnaround while serving as the customer's primary day-to-day contact.
The work requires you to move fluidly between people leadership and technical judgment. You'll hire, develop, and retain your team; run metrics reviews with the customer's technical leads; and audit reviewer output directly to ensure scoring decisions hold up under scrutiny. You'll also partner on calibration cycles, own the escalation path for safety-critical findings, and identify where the engagement should expand next. Forecasting staffing and cost against the commercial model rounds out the operational side.
This role suits someone with five-plus years managing delivery teams of 10–20 people for enterprise clients, combined with substantial hands-on experience in AI evaluation—annotation, red-teaming, RLHF, model evaluation, or trust and safety work. You need the judgment to read scored work critically and form independent views, strong written communication for technical customers, and the ability to operate in a secure onsite environment. The position is full-time, in-office in San Jose, at $75–85 per hour depending on experience.
This role's local market on JobMinglr
Pay for this role
$75 to $85 per hour
That is the employer's posted range, and where you land in it is usually decided in one conversation. Before that conversation, run the numbers: analyze an offer for this role.
Before you apply, worth reading
How JobMinglr reads this job
Every listing here is scored against your profile before you apply: skills overlap, experience level, location and work arrangement, each weighted and explained. You see the score and the reasons, not just a list. How the matching works.