Director of Data Engineering | Allconnect

This role is not open to visa sponsorship or transfer of visa sponsorship including those on H1-B, F-1, OPT, STEM-OPT, or TN visa, nor is it available to work corp-to-corp. For the Allconnect website click here . Curious how Allconnect fits into Red Ventures?

Click here . Curious how Home fits into Red Ventures? Click here .

You'll lead the team that builds and scales Red Ventures' data infrastructure, using Databricks, Spark, and our proprietary platforms to move data across the business and turn raw data into the analytics we run on. As a Director who stays close to the technical details, you'll set multi-year direction and grow a team of engineers and engineering managers, while making sure your team has the context and guidance to move fast on their own. Our data team exists to make data easy to use for everyone, and this role owns how we get there, in close partnership with the business leaders who depend on that data.

What You'Ll Do

Lead a team of data engineers and engineering managers building large-scale, real-time data pipelines in Spark Act as the primary technical liaison to business leadership, translating trade-offs between technology and business priorities in both directions Set the multi-year technical direction for Data Engineering and communicate that strategy across both technical and business audiences Partner with data scientists, analysts, and business stakeholders to turn business

Requirements

into data products the team builds against Represent Data Engineering in cross-functional and executive planning, building alignment across engineering, product, and business leaders Set the architectural direction and standards for how data moves across platforms, from streaming ingestion through ETL to aggregation and analytics, hold your managers and teams accountable to that direction, and establish the engineering and agile practices that support delivery speed and quality Stay close to the technical details to guide architecture design decisions, performance tuning, and tradeoff calls, and to give your team the context they need to move quickly without waiting on you Work with your teams and the RV Data Platform team to design scalable solutions across distributed systems using AWS, Databricks, and modern big data technologies Champion a governance-first mindset across your team, making sure the way we collect, move, and use data protects the privacy of the people behind it Manage and develop the data engineering managers on your team, setting the bar for how they coach, hire, and run their teams Own career development across your org, building growth paths for individual contributors and managers alike, and coaching through regular, direct feedback Hold the team accountable for delivering high-quality, scalable solutions across the full data lifecycle

What We'Re Looking For

8-10 years of data or software engineering experience, including 5+ years managing engineering teams and at least 1 year managing managers Track record of communicating technical strategy and tradeoffs clearly to non-technical executives, and building trust quickly with cross-functional peers Deep curiosity about the systems and processes that produce our data, with enough understanding of the source to help your team strengthen and troubleshoot pipelines rather than treating data as a black box Track record of leading engineering teams in an agile environment, coaching both engineers and managers, and giving teams enough context to operate independently Deep hands-on experience building large-scale Spark applications for batch and streaming workloads Experience with modern lakehouse platforms (Databricks or similar) in addition to traditional Spark-based pipelines Expertise with the Spark RDD and DataFrame/Dataset APIs, with emphasis on DataFrames, for querying and data manipulation Experience with Spark Structured Streaming and SparkSQL, including broadcast joins Strong programming background in Python and/or Scala, with SQL fluency ac