ML Inference Engineer – Terminal-Bench Task Designer

Responsibilities: — Design expert-level Terminal-Bench tasks that convert real, paid ML inference work from your experience into challenges solvable end to end by AI agents. — Define the files, systems, and environments each task runs on, including model serving with streaming, ensuring challenges are realistic, reproducible, and command-line solvable. — Develop working reference solutions for each task, verifying they run end to end and require about 60 to 75 minutes of expert effort. — Create automated checks that objectively prove correctness of results, including detecting silent ML output errors like numerical drift or metric bugs. — Package tasks using Docker and Git for reproducibility, and document involved systems, tools, data, and rules for reviewer validation. — Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and resolving ambiguities to maintain expert-level, fair, and objectively gradable tasks. Requirements: — Minimum 3 years of hands-on ML inference or serving experience (not just studying, selling, or managing), or a PhD with active research in the subfield. — Recent experience (within last 3 years) naming systems, tools, data, and rules used. — Experience training, serving, or evaluating models at real scale (multi-GPU or production serving), beyond notebooks or API calls. — Hands-on familiarity with PyTorch or JAX internals and serving stacks such as vLLM, SGLang, or TensorRT-LLM. — Ability to explain silent ML errors (numerical drift, metric bugs) and methods to detect them. — Technical fluency in Python, command line, and working with files, CSV/JSON, APIs, or databases. — Availability to start by early November 2026 and complete tasks requiring about 60 to 75 minutes each. Nice-to-have: — Experience with AI labs or LLM infrastructure. — Contributions to PyTorch, vLLM, or SGLang. — GPU profiling experience. — Proficiency with Docker and Git. — Experience writing automated tests. Benefits: — Task vetting replaces traditional interviews; candidates complete a vetting task for client assessment. — Compensation is task-based, paid per task. — Each task takes approximately 60 to 75 minutes. — Work is delivered task by task with reviewer feedback on submissions. — Expected start in early November 2026.