ML Inference Engineer – Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)

Responsibilities: — Design expert-level Terminal-Bench tasks that transform real, paid ML inference work from your own experience into challenges that AI agents must solve end to end. — Define the files, systems, and environments each task runs on, including model serving with streaming, ensuring the challenge is realistic, reproducible, and solvable via command line. — Develop a working reference solution for every task, verifying it runs end to end and requires about 60 to 75 minutes of expert effort. — Build automated checks that objectively prove correctness of results, including handling silent ML output errors such as numerical drift or metric bugs. — Package tasks using Docker and Git to ensure reproducibility, and document involved systems, tools, data, and rules for reviewer validation. — Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and resolving ambiguities to maintain expert-level, fair, and objectively gradable tasks. Requirements: — At least 3 years of hands-on ML inference or serving experience (not just studying, selling, or managing), or a PhD with active research in the subfield. — Recent experience (within last 3 years) with named systems, tools, data, and rules used. — Experience training, serving, or evaluating models at real scale (multi-GPU or production serving), beyond notebooks or API calls. — Hands-on expertise with PyTorch or JAX internals and serving stacks such as vLLM, SGLang, or TensorRT-LLM. — Ability to explain silent ML errors (numerical drift, metric bugs) and methods to detect them. — Technical fluency in Python, command line, and working with files, CSV/JSON, APIs, or databases. — Availability to start by early November 2026 and complete tasks requiring about 60 to 75 minutes each. Nice-to-have: — Experience in AI labs or LLM infrastructure. — Contributions to PyTorch, vLLM, or SGLang. — GPU profiling experience. — Proficiency with Docker and Git. — Experience writing automated tests. Benefits: — Task vetting replaces traditional interviews: candidates complete a vetting task for assessment. — Compensation is task-based; each task takes approximately 60 to 75 minutes. — Work is delivered task by task with reviewer feedback on submissions. — Expected start date is early November 2026.