Principal Platform Engineer - Kubernetes Reliability and Distributed Systems

Principal Platform Engineer – Kubernetes Reliability and Distributed Systems

This role has been designed as 'Hybrid' with a requirement that you will work on average 2 days per week from an HPE office.

Who We Are:

Hewlett Packard Enterprise is the global edge-to-cloud company advancing the way people live and work. We help companies connect, protect, analyze, and act on their data and applications wherever they live, from edge to cloud, so they can turn insights into outcomes at the speed required to thrive in today’s complex world. Our culture thrives on finding new and better ways to accelerate what’s next. We know varied backgrounds are valued and succeed here. We have the flexibility to manage our work and personal needs. We make bold moves, together, and are a force for good. If you are looking to stretch and grow your career our culture will embrace you. Open up opportunities with HPE.

Job Description:

Role overview

We are seeking a Principal Platform Engineer with deep expertise in Kubernetes, distributed systems, and highly available infrastructure to help evolve and operate the Object Storage platform.

This role will own the most complex platform-level technical challenges, including Kubernetes scheduling and resource management, pod failover, service discovery and load balancing, control-plane availability, node failure recovery, timeout behavior, and system resilience during degraded operating conditions.

The successful candidate will work across engineering teams to establish clear ownership, diagnose complex production issues, guide architectural decisions, and build scalable, reliable platform capabilities. This is a hands-on technical leadership role that requires both deep implementation experience and the ability to influence engineering practices across the organization.

Key responsibilities

Serve as the senior technical authority for Kubernetes platform architecture, reliability, and operational behavior.

Own the investigation and resolution of complex issues involving: Pod failover and rescheduling

Service availability and load balancing

Control Plane operation as well as Node failover

Network, API, and service timeouts

Resource requests, limits, quotas, scheduling, affinity, taints, tolerations, and workload pinning

Cluster capacity, performance, and scaling

Distributed-system failure modes and recovery behavior

Define and drive improvements to Kubernetes high availability, fault tolerance, and disaster-recovery strategies.

Establish clear technical ownership and operating models for platform capabilities.

Lead architecture, design, and implementation of platform improvements across multiple engineering teams.

Develop repeatable approaches for diagnosing failures across application, Kubernetes, networking, storage, and infrastructure layers.

Improve observability, including metrics, logs, traces, alerts, dashboards, and health indicators for platform services.

Define and monitor service-level objectives and reliability measures for platform components.

Lead or support major incident response, root-cause analysis, and corrective-action programs.

Identify systemic issues rather than treating incidents as isolated events.

Create technical standards, reference architectures, runbooks, and operational best practices.

Mentor senior engineers and raise the organization’s expertise in Kubernetes and distributed systems.

Partner with application, infrastructure, networking, security, and operations teams to resolve cross-functional issues.

Evaluate new Kubernetes capabilities and platform technologies and determine where they provide meaningful value.

Balance short-term incident resolution with long-term architectural improvements.

Required qualifications

Significant experience designing, building, and operating production Kubernetes platforms.

Deep understanding of Kubernetes architecture and core components, including: API server and control-plane services

Scheduler and controller manager

Nodes, kubelet, and container runtimes

Services, endpoints, ingress, and load balancing

Resource requests and limits

Quality of Service classes

Affinity, anti-affinity, taints, tolerations, and topology constraints

Pod disruption budgets and workload availability

Kubelet and CRI (containerd)

Strong knowledge of high-availability and failure-recovery patterns for distributed systems.

Demonstrated experience troubleshooting complex production failures involving Kubernetes, networking, infrastructure, and application services.

Experience diagnosing Kubernetes failover behavior, service timeouts, partial failures, network partitions, and degraded cluster states.

Experience with Kubernetes cluster upgrade process.

Strong understanding of Linux systems, networking, DNS, TCP/IP, containers, and storage fundamentals.

Experience with cloud or on-premises infrastructure and infrastructure-as-code practices.

Proficiency in Go, Python, and Bash programming or scripting language.

Experience developing automation, platform tooling, or operational frameworks.

Strong observability and reliability engineering experience using metrics, logs, traces, alerting, and performance analysis.

Demonstrated ability to lead complex technical initiatives across organizational boundaries.

Excellent written and verbal communication skills, including the ability to explain complex technical issues to both technical and non-technical stakeholders.

Ability to operate effectively during high-severity incidents and drive issues through root cause and permanent resolution.

Preferred qualifications

Experience contributing to or extending Kubernetes, container-runtime, networking, or infrastructure projects.

Experience with Kubernetes distributions such as EKS, AKS, GKE, OpenShift, or upstream Kubernetes.

Experience with service meshes, ingress controllers, CNI plugins, CSI storage systems, or cluster-autoscaling technologies.

Experience with distributed databases, consensus systems, or control-plane technologies such as etcd.

Experience designing multi-cluster, multi-region, or edge-oriented Kubernetes environments.

Knowledge of Linux performance analysis, kernel behavior, and container isolation.

Experience with chaos engineering, failure injection, load testing, and resilience testing.

Experience establishing SLOs, error budgets, production-readiness reviews, and reliability governance.

Experience working in an environment with embedded, edge, fleet, or geographically distributed systems.

Familiarity with GitOps and deployment technologies such as Argo CD, Flux, Helm, or similar tools.

Relevant Kubernetes or cloud certifications, such as CKA, CKAD, or equivalent practical experience.

Principal-level expectations

The person hired into this role should be expected to:

Own problems that cross team and technology boundaries.

Make architecture decisions that affect the broader platform and product roadmap.

Establish techn