Senior Observability Engineer

Responsibilities: — Design, scale, and operate shared metrics, logs, and tracing services for infrastructure health and workload behavior. — Manage declarative Kubernetes deployments using Argo CD and GitLab and GitHub CI/CD, automate host and VM configuration with Ansible and SaltStack. — Improve monitoring for bare-metal hardware, Linux, physical and overlay networks, and Kubernetes, integrating telemetry from IPMI/Redfish, PDUs, SNMP, gNMI, and NetFlow/sFlow. — Build telemetry pipelines and dashboards for GPU clusters, AI training runs, and inference services. — Define shared standards for OpenTelemetry instrumentation, context propagation, and Collector pipelines; support teams instrumenting Go, Node.js, and Python services and APIs. — Build dashboards and alerts, improve notification workflows through PagerDuty, participate in on-call rotations, reduce alert noise, and shorten incident diagnosis and recovery. — Document standards, help teams adopt them, and develop AI-assisted approaches to monitoring and incident investigation. Requirements: — Strong hands-on experience with Grafana and production telemetry systems including VictoriaMetrics and VictoriaLogs or comparable metrics and logging backends. — Strong Linux fundamentals, production Kubernetes experience, and experience monitoring cloud-native applications and distributed systems. — Practical experience with OpenTelemetry instrumentation, metrics, traces, structured logs, trace context propagation, and Collector pipelines. — Experience with declarative deployments, CI/CD, and configuration management using Argo CD, GitLab CI/CD, Ansible, and SaltStack. Benefits: — Cash and equity compensation along with local benefits. — Diverse team with 40+ nationalities and 6 different nationalities on the management team. — Opportunity to make an impact and work alongside world-class engineers, researchers, and partners across the global AI ecosystem.