AI Infrastructure Engineer

 

Position Overview:

Interested in finding system issues before they slow validation? Excited by the challenge of keeping large-scale Linux, SLURM, and GPU systems operating smoothly? This role works across Linux clusters, SLURM services, GPU infrastructure, and operational tooling that support large-scale AI and high-performance computing. We are looking for strong systems troubleshooting, sound operational judgment, and a practical approach to improving system health during deployment and validation.

 

Responsibilities:

• Check system health before validation by verifying node readiness, scheduler health, GPU availability, and critical cluster services.
• Support stable operation during validation by troubleshooting Linux, SLURM, GPU, network, and storage issues and by partnering with the right teams to restore healthy system behavior.
• Separate infrastructure faults from workload or test issues and turn findings into clear next steps, remediation actions, and operational guidance.
• Improve recurring issues through better diagnostics, runbooks, alerts, and targeted automation that reduces manual effort and helps teams respond more efficiently.

Qualifications:

Required Qualifications:

• 4+ years of experience in site reliability engineering, infrastructure engineering, systems engineering, or production operations in large-scale Linux environments.
• Strong Linux administration and troubleshooting skills across services, processes, networking, storage, resource contention, and system performance.
• Experience operating or troubleshooting bare-metal, HPC, or GPU clusters, including SLURM administration and an understanding of cluster health and workload execution.
• Practical Python or Bash scripting skills and familiarity with Ansible or similar tools for operational tasks.
• Experience with monitoring, telemetry, logs, dashboards, alerts, or diagnostic tooling such as Prometheus, Grafana, Alertmanager, or similar platforms.
• Demonstrated troubleshooting and remediation experience, including the ability to investigate failures, identify root causes, and restore healthy system behavior.
• A bachelor’s degree in computer science, computer engineering, electrical engineering, or a related field, or equivalent practical experience.

Preferred Qualifications:

• Experience with GPU telemetry and SLURM metrics.
• Experience with bare-metal cluster management platforms.
• Practical knowledge of bare-metal provisioning, InfiniBand, RoCE, NVLink or NVSwitch, and parallel storage failure modes.
• Experience supporting early-life cluster operations, system validation, or customer deployment environments.

Tools and Technologies:

• Linux
• SLURM
• Python
• Bash
• Ansible
• Prometheus
• Grafana
• Alertmanager

Vailexa Technology is an equal opportunity employer. We consider all qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other protected status under applicable law.