Senior/Staff Backend Engineer - Distributed System

About The Company

Our hiring partner is on a mission to make AI compute ubiquitous, seamless, and limitless. They’re building a cloud where AI just works—anywhere, anytime. “AI Power. Everywhere.” Join the team designing the infrastructure for an AI-first world.

Why this role exists

Our hiring partner is looking for a Backend Engineer to build the systems that orchestrate GPU clusters for AI workloads. You’ll develop APIs that manage GPU allocation, memory, compute scheduling, and multi-tenant isolation—challenges unique to AI infrastructure that go far beyond standard backend engineering. On their backend team, you’ll tackle questions like: How can high-cost GPU resources be efficiently shared among users? How do we handle memory constraints for large AI models? How do we maintain quality of service when workloads compete for compute? This is an opportunity to build infrastructure where every API call could allocate thousands of dollars of compute per hour, and where your optimizations directly influence whether AI startups can train their models affordably.

What you’ll do

Design APIs that simplify complex GPU operations for developers

Build scheduling algorithms that maximize GPU utilization while meeting SLAs

Develop systems to manage the full GPU lifecycle: provisioning, allocation, scheduling, and release

Implement usage tracking and billing for GPU-hours, memory, and compute utilization

Create monitoring solutions for GPU-specific metrics, health checks, and automated recovery

Build multi-tenant systems with resource isolation, quota management, and fair scheduling

Optimize cold starts for model serving and efficient model loading

Collaborate with frontend engineers to expose complex infrastructure through intuitive interfaces

Leverage AI-assisted coding tools (GitHub Copilot, Claude Code, Cursor IDE, etc.) to enhance productivity and code quality

You’ll thrive here if you

Have 5+ years of backend engineering experience in distributed systems

Are proficient in Go, Python, or similar backend languages

Have experience with resource scheduling, orchestration, and API design (REST, GraphQL, gRPC)

Understand hardware constraints and system optimization

Have Linux systems and containerization experience (Docker, Kubernetes)

Are comfortable working with expensive resources where efficiency impacts costs

Are excited to solve novel problems in AI infrastructure (beyond CRUD apps)

Bring a startup mindset—comfortable with ambiguity and rapid iteration

Bonus qualifications

Experience with GPU or HPC cluster management

Familiarity with ML/AI workload patterns and requirements

Experience with high-value resource allocation systems

Background in performance optimization for compute-intensive workloads

Knowledge of GPU virtualization and sharing technologies

Experience building billing or metering systems

Details

They offer competitive salary and equity based on experience and skillset

Hybrid role: 3 days in office, 2 days WFH; must be located in Palo Alto

Applicants must be authorized to work in the United States without visa sponsorship