Вакансии

Thinking Machines Lab is looking for a Software Engineer Production Inference Distributed Inference to build and operate distributed production systems for large-scale model serving.

350 000 – 500 000 $

Bitdeer Technologies Group is looking for a Senior Inference Service SRE and Automation Engineer to ensure inference-service reliability, automate remediation, manage multi-cluster operations, and meet customer-facing SLA targets.

Зарплата не указана

Location: Canada; San Francisco; United States; New York; Ottawa; Toronto; Montreal Workplace: Remote WHO ARE WE? Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems. We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft. We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us! WHY THIS ROLE? This role is for a highly experienced Back-End Software Engineer who wants to work on core systems that shape how the platform (Model Vault) operates. You will help define the architecture of the platform where reliability, security, performance, and simplicity all matter, and where many of the problems do not have off-the-shelf solutions. Model Vault is Cohere’s inference platform for deploying and operating models securely at enterprise scale. The team owns the systems between a customer and the underlying AI infrastructure, including identity, deployments, metering, billing, security, privacy, and confidential computing. We are strengthening these systems as Model Vault grows, with the goal of scaling to millions of users and supporting some of the world’s most demanding enterprise workloads. AS A SENIOR/STAFF SOFTWARE ENGINEER, YOU WILL: - Evolve and scale the backend systems that power Model Vault, including authentication and authorization, deployment management, billing, metering, and customer lifecycle. - Lead major architectural initiatives across identity, payments, usage systems, platform scalability, and other core product foundations. - Build highly reliable control-plane services for provisioning, configuring, scaling, and operating model deployments across clouds and hardware platforms. - Design APIs and distributed systems with strong guarantees around correctness, consistency, isolation, failure recovery, idempotency, and observability. - Build security and privacy into the platform architecture, including tenant isolation, least-privilege access, auditability, secrets management, encryption, and confidential computing boundaries. - Solve complex problems that span backend systems, cloud infrastructure, security, and AI inference, where conventional SaaS patterns are often not enough. - Drive projects end-to-end, from architecture and threat modeling through implementation, rollout, production operations, and iteration based on real customer workloads. YOU MAY BE A GOOD FIT IF: - You are a strong backend engineer with excellent fundamentals in distributed systems, databases, networking, operating systems, and security. - You have strong programming fundamentals and experience building production systems in languages such as Go, Python, Java, C++, Rust, or similar. You write clean, maintainable, well-tested code and make thoughtful tradeoffs between simplicity, performance, and correctness. - You have designed and operated production systems where correctness, availability, security, and performance mattered at significant scale. - You can take ambiguous, high-impact technical problems, define the right architecture, and drive them through to reliable production systems. - You have strong systems-design instincts and understand concepts such as concurrency, consistency, caching, queues, rate limiting, backpressure, failure domains, idempotency, and graceful degradation. - You think deeply about security boundaries and understand authentication, authorization, identity, cryptography, secrets, tenant isolation, and how small implementation choices can create significant security risks. - You are comfortable reasoning across backend services, APIs, databases, distributed systems, and the cloud infrastructure they run on. - You are comfortable diving into unfamiliar codebases and debugging complex problems that cross service or system boundaries. - You care about the details of production engineering, including instrumentation, capacity planning, migrations, backwards compatibility, incident response, and safe rollouts. - You are comfortable challenging existing architecture and evolving foundational components when the system’s needs change. - You have a track record of taking technically difficult projects from an unclear starting point to systems that other engineers and customers depend on. FULL-TIME EMPLOYEES AT COHERE ENJOY THESE PERKS: - A weekly lunch stipend of $75/£75 or equivalent in your local currency for lunch. - Full health and dental benefits, including a separate budget for mental health. - RRSP matching, 401K, Pension Scheme. - 100% Parental Leave top-up for up to 6 months, for either parent. - Annual enrichment benefits: Arts & culture, fitness/wellness, quality time, and a workspace improvement credit. Education & learning stipend for conferences, courses, and coaching. - 6 weeks of paid vacation (30 working days!) - Budget for traveling to other offices if you are remote, plus an annual company offsite. HOW AND WHERE WE WORK: - Cohere is remote-friendly, but we also have offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul with more opening soon. - For those in the office: a daily lunch program, plenty of snacks, and regular community and social events. - For those not near an office: a co-working benefit so you can work alongside others in your city. - Everyone receives a $500 home office stipend to set up your workspace properly. If any of the above doesn’t line up exactly with your experience, we still encourage you to apply. We strive to create an inclusive work environment for all; we welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form https://docs.google.com/forms/d/12a6IrLdF3kI2nonKSr4tiFuz18rLQbaeYV-JM9L4o9Q/edit, and we will work together to meet your needs. We may use AI-enabled tools to screen and assess applicants against the criteria for this position. This helps our recruiters identify potentially qualified candidates, but it doesn't limit the applications our recruiters may review or consider. Beware of Scams: Cohere will never ask for payment or third-party services (e.g., CV writing) as part of our hiring process. All legitimate roles are listed on the Cohere careers page and LinkedIn only, with all communications from Cohere employees coming from an @cohere.com or @cw.cohere email alias. If jobs are viewed on other sites then please verify these through our official careers https://cohere.com/careers page.

Зарплата не указана

Impossible Cloud is looking for Senior Sales Managers to own DACH managed-inference revenue and guide complex technical sales cycles to signed contracts.

Зарплата не указана

SpAItial is looking for an ML Inference Engineer to build and operate scalable inference services and production APIs.

Зарплата не указана

-Инфраструктура и высоконагруженный инференс (Platform Engineering): -Проектирование и поддержка GPU-кластера инференса (Nvidia) для десятков тысяч пользователей: скалирование, балансировка и приоритизация запросов, пользовательские лимиты.

Зарплата не указана

Location: San Francisco; New York Workplace: Hybrid ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLE We're looking for an Engineering Manager to lead part of our Inference Performance team. This team makes the world's most demanding AI workloads run faster and more efficiently on GPUs. You'll manage and grow a team of inference performance engineers working across the inference engine and runtime: kernels, scheduling, batching, KV-cache management, speculative decoding and prefill/decode disaggregation. This is a hands-on technical leadership role. You'll set direction, unblock hard problems and earn the team's trust by going deep on GPU performance, while also hiring, developing and supporting the people doing the work. Your team's output directly affects how fast our customers' models run and how efficiently we serve them. The team is scaling quickly, so you'll help shape how it is structured as it grows. EXAMPLE INITIATIVES Your team will work on these types of projects as part of our Inference Runtime team: - Agentic inference optimization: 50-90% faster engines https://www.baseten.co/blog/agentic-inference-optimization-faster-than-sota/ - Agentic Kernels in Production https://www.baseten.co/blog/agentic-kernels-in-production/ - Live draft model training for speculative decoding https://www.baseten.co/blog/live-draft-model-training-for-speculative-decoding/ - The Baseten Inference Stack https://www.baseten.co/resources/guide/the-baseten-inference-stack/ RESPONSIBILITIES - Lead, mentor and grow a team of inference performance engineers through regular 1:1s, clear feedback, career development and performance reviews. - Hire top GPU and inference engineering talent, and build a strong, collaborative team culture as the runtime team scales. - Own the technical roadmap and execution for runtime performance work, balancing customer needs, new model launches and long-term platform investments. - Stay close to the technical work. Review designs, guide profiling and optimization efforts, and help the team reason from first principles about where time and memory go. - Drive the productionization of inference techniques such as quantization, speculative decoding, KV-cache reuse, chunked prefill and custom scheduling. - Turn performance wins into measurable outcomes: tokens per GPU-hour, utilization, latency and cost. - Help the team bring up and tune new model architectures on new hardware quickly, often in the same week they're released. - Partner with Infrastructure, Inference Platform, Kernels, Model APIs and customer-facing teams to set priorities, coordinate launches and ship wins. - Set high standards for engineering quality, benchmarking, operational excellence and incident response. REQUIREMENTS - Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or a related field. - Experience managing engineers, including hiring, mentoring, giving feedback and running performance reviews. - Experience leading or closely supporting GPU optimization teams in training, inference or recommendation systems. - Strong technical depth in GPU workloads, with a solid understanding of GPU architecture and performance tradeoffs. - Familiarity with ML libraries such as PyTorch, TensorRT or TensorRT-LLM. - A track record of driving roadmaps and shipping complex technical projects with a team. - Clear written and verbal communication, including the ability to align stakeholders across teams. NICE TO HAVE - Familiarity with inference engines such as vLLM, SGLang or TensorRT-LLM. - Experience with LLM optimization techniques (e.g., quantization, speculative decoding, continuous batching) in production. - Experience with GPU kernels (CUDA, Triton, CUTLASS, or similar). - Experience scaling a team through rapid growth at a startup. - A background as a hands-on performance or systems engineer before moving into management. BENEFITS - Competitive compensation, including meaningful equity - (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents - Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!) - Paid parental leave - Fertility and family-building stipend through Carrot - (U.S. only) Company-facilitated 401(k) - Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities. Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you. At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).

• Ускорять и масштабировать LLM в проде: оптимизация инференса очень больших моделей (до 1T+ параметров), SGLang, KV/prefix caching, speculative decoding, квантизация. • Разворачивать распределенный инференс: MoE, tensor/expert/pipeline parallelism на multi-node / multi-GPU кластерах (32x H200 + AWS).

Зарплата не указана

Location: San Francisco; Toronto; New York; Montreal; Seattle Workplace: Hybrid ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLE We're looking for distributed systems engineers and product-minded generalists to build the distributed runtime that powers large-scale LLM inference on Baseten. Our inference platform empowers customers to deploy and operate cutting-edge models with industry-leading performance, scalability, and reliability. It also powers Model APIs, our hosted endpoints for the latest open-source models. You'll work across the stack, from the developer experience customers use to deploy models, through the libraries behind features like tool calling and reasoning, down to the systems that orchestrate deployments on Kubernetes and route traffic efficiently. Your job is to make sure every model on our platform is fast, reliable, and cost-efficient. You'll join a small, high-impact team at the intersection of distributed systems, model performance, infrastructure, and product, helping define how developers use AI models at scale. This role is ideal for engineers who enjoy owning systems in production, solving hard integration problems, and making complex infrastructure simple and reliable for users.   EXAMPLE INITIATIVES You'll get to work on these types of projects on our Inference Platform: - The Baseten Inference Stack at NVIDIA Dynamo Day https://www.baseten.co/blog/nvidia-dynamo-day-baseten-inference-stack/ - 2x faster inference with KV cache-aware routing https://www.baseten.co/blog/how-baseten-achieved-2x-faster-inference-with-nvidia-dynamo/ - How Baseten multi-cloud capacity management (MCM) unifies deployments https://www.baseten.co/blog/how-baseten-multi-cloud-capacity-management-mcm-powers-cloud-self-hosted-and-hybr/#comparing-deployment-options-cloud-vs-self-hosted-vs-hybrid RESPONSIBILITIES - Build the infrastructure and orchestration systems that deploy and run large-scale distributed LLM inference, including routing, autoscaling, scheduling, and runtime management. - Design, build, and operate Model APIs, with a focus on advanced inference capabilities: structured outputs (JSON mode, grammar-constrained generation), tool/function calling, and multimodal serving. - Implement platform fundamentals such as API versioning, validation, usage metering, quotas, and authentication. - Instrument deep observability (metrics, traces, logs) and build repeatable benchmarks for speed, reliability, and quality. Help set best practices for testing, release automation, and operational excellence. - Debug and harden complex production systems spanning Kubernetes, distributed runtimes, networking, and GPU workloads to improve reliability and scalability. - Partner with Inference Performance engineers and other teams to make new optimizations broadly available to customers and easy to configure. - Own projects end to end, from architecture through deployment, monitoring, and iteration on customer feedback. Along the way, make thoughtful tradeoffs between performance, reliability, operational simplicity, and developer experience.   REQUIREMENTS - Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field, or equivalent practical experience. - 3+ years building and operating distributed systems, backend infrastructure, or large-scale APIs where reliability, latency, and scale are first-class concerns. - A proven track record of owning low-latency, reliable backend services, including rate limiting, auth, quotas, metering, and migrations. - Infrastructure instincts with a feel for performance: profiling, tracing, capacity planning, and SLO management. - Comfort debugging performance and reliability issues across multiple layers of the stack, from application behavior down to runtime and infrastructure internals. - A strong sense of developer experience. You think about how systems are used, not just how they work. - Eagerness to learn new languages, frameworks, and systems, and a real interest in inference engineering. Prior ML or LLM experience isn't required, though experience with model serving or inference systems is a plus. - Excellent written communication and collaboration skills, including writing clear design docs and working across functions.   NICE TO HAVE - Experience with or contributions to LLM inference engines and frameworks such as vLLM, SGLang, TensorRT-LLM, TGI, or Dynamo. - Deep Kubernetes experience, including operators and custom resources, plus familiarity with service meshes or API gateways. - Experience with distributed scheduling, autoscaling, or service orchestration. - Experience operating GPU workloads in production. - A background in developer-facing infrastructure or APIs, or contributions to open-source infrastructure or ML systems. - Familiarity with observability tooling, CI/CD systems, or release automation. BENEFITS - Competitive compensation, including meaningful equity - (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents - Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!) - Paid parental leave - Fertility and family-building stipend through Carrot - (U.S. only) Company-facilitated 401(k) - Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities. Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you. At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).

Location: San Francisco; Toronto; New York; Montreal; Seattle Workplace: Hybrid ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLE We're looking for inference performance engineers who want to make the world's most demanding AI workloads run faster and more efficiently. You'll work across the stack, from the inference engine and runtime through scheduling, serving, and routing. Along the way you'll apply techniques like prefill/decode disaggregation, speculative decoding, and KV-cache management. You'll reason from first principles about where time and memory go, find what's holding performance back, and close the gap. Your work directly impacts how fast our customers' models run and how efficiently we serve them. This role is ideal for someone who thrives in a fast-paced startup environment and is eager to make significant contributions to the exciting field of LLM inference.   EXAMPLE INITIATIVES You'll get to work on these types of projects as an Inference Performance engineer: - Agentic Kernels in Production https://www.baseten.co/blog/agentic-kernels-in-production/ - How we built the new fastest API for GLM-5.2 https://www.baseten.co/blog/how-we-built-the-new-fastest-api-for-glm-52/ - Live draft model training for speculative decoding https://www.baseten.co/blog/live-draft-model-training-for-speculative-decoding/#eliminating-traditional-bottlenecks - Making Kimi K3 Tokenization 18x faster https://www.baseten.co/blog/making-kimi-k3-tokenization-18x-faster-for-million-token-agentic-workloads/ - The Baseten Inference Stack https://www.baseten.co/resources/guide/the-baseten-inference-stack/ - Driving model performance optimization https://www.baseten.co/blog/driving-model-performance-optimization-2024-highlights/ RESPONSIBILITIES - Implement and productionize cutting-edge inference techniques, working deep in runtime internals. That includes quantization, speculative decoding, KV-cache reuse, chunked prefill, LoRA, guided generation for structured outputs, and custom scheduling and routing algorithms. - Profile and optimize inference end to end, from kernel launch overhead and memory layout up to request scheduling, prefill/decode disaggregation, and cache-aware routing. Run cross-layer investigations, such as tracing a tail-latency regression from request timing through routing and batching down to a kernel. - Turn performance into cost savings. Improve tokens per GPU-hour, raise utilization, and give customers and internal teams clear latency/throughput/cost tradeoffs. - Bring up and tune new model architectures on new hardware quickly, often in the same week they're released. - Build benchmarking frameworks that measure real-world performance across model architectures, batch sizes, sequence lengths, and hardware configurations. - Contribute upstream to open-source inference engines (vLLM, SGLang, TensorRT-LLM), and partner closely with model, infrastructure, and customer-facing teams to ship wins.   REQUIREMENTS - Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or related field. - Experience with one or more general-purpose programming languages, such as Python or C++. - Familiarity with LLM optimization techniques (e.g., quantization, speculative decoding, continuous batching). - Strong familiarity with ML libraries, especially PyTorch, TensorRT, or TensorRT-LLM. - Demonstrated interest and experience in LLMs. - Deep understanding of GPU architecture.   NICE TO HAVE - Proficiency in enhancing the performance of software systems, particularly in the context of large language models (LLMs) - Contributed to vLLM, SGLang, TensorRT-LLM, or another inference engine. - Worked on large-scale distributed serving: autoscaling, load balancing, multi-region or multi-cloud capacity. - Written or optimized GPU kernels (CUDA, Triton, CUTLASS, or similar) - Worked on quantization (FP8/FP4, AWQ, GPTQ) or speculative decoding in production. - Deep understanding of software engineering principles and a proven track record of developing and deploying AI/ML inference solutions. BENEFITS - Competitive compensation, including meaningful equity - (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents - Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!) - Paid parental leave - Fertility and family-building stipend through Carrot - (U.S. only) Company-facilitated 401(k) - Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities. Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you. At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).

Location: San Francisco Workplace: OnSite ABOUT US: AI NEEDS A NEW INFRASTRUCTURE LAYER. WE'RE BUILDING IT AT MODAL. Every era of computing brought new workloads that previous infrastructure couldn't support: mainframes, databases, and the cloud. Each time, the company that rebuilt the layer underneath defined the decade. AI is no different, except it touches everything instead of one slice, and the window to build the layer underneath it is open right now. Our customers include category-defining companies like Ramp https://modal.com/blog/how-ramp-built-a-full-context-background-coding-agent-on-modal, Cognition, DoorDash, and Suno. They rely on Modal for instant GPU access, sub-second container starts, and native storage, so it's simple to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale. We've passed $1B in annualized run rate, up 15x from a year ago. In October 2026 we raised a $750M Series D led by Accel at a $15.75B valuation. Our team includes creators of popular open-source projects (e.g.,Seaborn https://github.com/mwaskom/seaborn,Luigi https://github.com/spotify/luigi), academic researchers, international olympiad medalists, and experienced engineering and product leaders with decades of experience. THE ROLE: Modal’s Inference Runtime team owns the container runtime stack used to run inference and training workloads across our fleet. We work at the boundary of Linux, containers, filesystems, storage, GPU drivers, and distributed systems.Our goal is to make demanding ML workloads start quickly, run efficiently, and remain securely isolated — whether they use a single GPU, hundreds of gigabytes of memory, or multiple GPUs connected with RDMA.We’re looking for a systems engineer who enjoys working deep in the stack. You’ll build production runtime infrastructure in Rust and Go, diagnose difficult Linux and performance problems, and help determine the architecture of Modal’s container platform. You’ll also work closely with maintainers of gVisor and contribute to the runtime itself when the right fix belongs upstream. THE WORK: - Make container startup, checkpoint, and restore dramatically faster for large inference and training workloads. - Build multi-GPU and accelerator-aware snapshotting, including efficient handling of GPU memory and RDMA-enabled workloads. - Design zero-copy and low-copy data paths between container memory, filesystems, storage, and Modal’s runtime. - Optimize large snapshot pipelines using techniques such as parallel uploads, direct I/O, incremental snapshots, and more efficient memory handling. - Improve container image and filesystem performance across EROFS, FUSE, page caches, overlay filesystems, and remote storage. - Extend our sandboxed runtime to support new GPUs, drivers, profiling tools, and device capabilities across NVIDIA and AMD hardware. - Debug complex failures involving system calls, virtual memory, process lifecycle, kernel behavior, GPU drivers, and container isolation. - Safely roll out runtime and kernel changes across a heterogeneous fleet using compatibility controls, scheduling constraints, feature flags, and observability. - Work across the runtime, scheduler, storage, and GPU infrastructure—and take ambiguous production problems from investigation through deployment. WHAT WE’RE LOOKING FOR: - Strong Linux systems knowledge, particularly processes, virtual memory, filesystems, system calls, scheduling, namespaces, cgroups, and signals. - Experience building or debugging container runtimes, sandboxes, Linux kernel, or similarly low-level infrastructure. - Strong programming ability in Rust, Go, or another systems language, with an interest in becoming productive in both Rust and Go. - Experience profiling and improving systems where memory movement, I/O, synchronization, or kernel interactions dominate performance. - Comfort debugging across abstraction boundaries, from application behavior down through runtimes, drivers, and the kernel. - An ability to turn loosely defined production problems into well-designed, reliable systems. - A desire to own important infrastructure and work closely with both internal teams and customers. RELEVANT EXPERIENCE: Any of the following would be helpful, but none is required: - gVisor, runsc, runc, OCI runtimes, seccomp, or checkpoint/restore systems. - Linux kernel development, virtualization, sandboxing, kernel modules, or device proxying. - CUDA, ROCm, GPU drivers, accelerator virtualization, or GPU profiling. - RDMA and high-performance networking. - FUSE, EROFS, direct I/O, mmap, page-cache behavior, or storage engines. - Large-memory or multi-GPU inference and training systems. - Contributions to open-source systems software. Prior gVisor or machine-learning experience is not required. We care more about strong systems fundamentals, curiosity, and the ability to learn unfamiliar parts of the stack. WHY THIS ROLE: The container runtime is directly on the critical path for inference performance. Improvements here can substantially reduce cold starts, increase token throughput, unlock new accelerator types, and make previously impractical workloads possible.This is an opportunity to work on unusually deep systems problems with immediate production impact. You’ll have room to shape the architecture, contribute to open-source runtime technology, and take ownership of foundational infrastructure used by every inference and training workload on Modal.

You make huge models, up to 1T+ parameters, answer fast and cheap in production. SGLang, caching, distributed inference, post-training.

Зарплата не указана

Location: San Francisco; New York Workplace: Hybrid ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. REQUIREMENTS Product Positioning & Technical Narrative - Own the positioning and messaging for Baseten’s dedicated inference and platform capabilities, including autoscaling, routing, failover, release safety, observability, and cost/performance capabilities. - Translate infrastructure-heavy product work into buyer narratives for ML engineering, platform engineering, security, compliance, procurement, and executive audiences. - Partner with Product and Engineering to understand the technical architecture, customer value, roadmap tradeoffs, and proof points behind each capability. - Define when a capability should be positioned as a platform differentiator, a dedicated inference requirement, a reliability story, a compliance story, or sales enablement. - Build messaging that is technically credible without being overly implementation-focused or generic. Launch Strategy & GTM Execution - Build and execute launch plans for major dedicated inference and serving platform capabilities, from early internal enablement through external announcement. - Decide what deserves a full launch versus what should ship through docs, sales enablement, customer-specific materials, or targeted enterprise outreach. - Create launch assets including messaging briefs, landing pages, blog posts, sales decks, one-pagers, FAQs, demo storylines, competitive talk tracks, and customer-facing proof points. - Sequence launches and supporting assets based on customer demand, product readiness, sales need, and market opportunity. - Partner with Demand Gen, Sales, Solutions, and Customer Marketing to turn product capabilities into pipeline-generating campaigns and enterprise deal support. Market, Customer & Competitive Intelligence - Build a deep understanding of how AI-native companies and enterprises run production inference today, where their current infrastructure breaks, and what they need as workloads scale. - Turn ambiguous customer and market signals into clear recommendations for Product, Sales, and Marketing. - Develop competitive messaging against inference providers, cloud platforms, and internal DIY alternatives Cross-Functional Partnership - Partner with Engineering to ensure technical claims are accurate, defensible, and supported by real product behavior. - Work with Sales and Solutions teams to understand deal blockers, buyer objections, and enablement gaps. - Collaborate with Demand Gen and Customer Marketing to build campaigns, customer stories, and field programs REQUIREMENTS - 6+ years in product marketing, technical marketing, developer marketing, or a related GTM role, ideally at an infrastructure, developer tools, cloud, data, or AI company. - Strong technical fluency and the ability to quickly ramp on infrastructure concepts like autoscaling, routing, failover, capacity, observability, deployment models, and performance optimization. - Experience translating complex technical products into clear, differentiated positioning for both technical practitioners and enterprise decision-makers. - Track record building launch strategies and GTM plans for platform products, infrastructure products, or technical capabilities with long enterprise sales cycles. - Excellent storytelling skills, with the ability to make technical infrastructure feel concrete, credible, and commercially urgent. NICE TO HAVE - Experience with AI/ML infrastructure, inference, model serving, cloud infrastructure, Kubernetes, observability, DevOps, or distributed systems. - Experience marketing products to ML engineers, platform engineers, infrastructure leaders, security/compliance teams, or technical executives. - Experience building competitive positioning against cloud providers, inference providers, model API companies, or internal platforms. BENEFITS - Competitive compensation, including meaningful equity - (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents - Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!) - Paid parental leave - Fertility and family-building stipend through Carrot - (U.S. only) Company-facilitated 401(k) - Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities. Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you. At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).

Location: San Francisco Workplace: OnSite ABOUT US: AI NEEDS A NEW INFRASTRUCTURE LAYER. WE'RE BUILDING IT AT MODAL. Every era of computing brought new workloads that previous infrastructure couldn't support: mainframes, databases, and the cloud. Each time, the company that rebuilt the layer underneath defined the decade. AI is no different, except it touches everything instead of one slice, and the window to build the layer underneath it is open right now. Our customers include category-defining companies like Ramp https://modal.com/blog/how-ramp-built-a-full-context-background-coding-agent-on-modal, Cognition, DoorDash, and Suno. They rely on Modal for instant GPU access, sub-second container starts, and native storage, so it's simple to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale. We've passed $1B in annualized run rate, up 15x from a year ago. In October 2026 we raised a $750M Series D led by Accel at a $15.75B valuation. Our team includes creators of popular open-source projects (e.g.,Seaborn https://github.com/mwaskom/seaborn,Luigi https://github.com/spotify/luigi), academic researchers, international olympiad medalists, and experienced engineering and product leaders with decades of experience. THE ROLE: Modal's LLM inference platform delivers frontier performance https://modal.com/blog/kimi-k3-by-moonshot-now-available-on-modal for open-source models with best-in-class elasticity and developer experience, made in part possible by our custom runtime with GPU memory snapshots https://modal.com/blog/gpu-mem-snapshots and multi-cloud substrate https://modal.com/blog/resource-solver. We're looking for a leader to own the direction and execution of this platform to continue to establish us as the clear market leader, working closely with customers like Cognition, Doordash, Ramp, and many more. You'll be leading a group of highly talented engineers working on our market-leading LLM inference offering, spanning the serving stack, routing infrastructure, internal agentic optimization platform, and the user-facing product surface area. This is a hands-on leadership role — expect to split your time between technical contribution, product shaping and people management depending on what the team needs. You'll set direction, remove blockers, and build a strong engineering culture as your team tackles hard problems in distributed computing, frontier inference serving, and performance optimization. RESPONSIBILITIES: Team - Recruit, hire, and grow a high-performing team of engineers; run regular 1:1s focused on coaching, feedback, and career growth. - Set clear performance expectations, hold a high bar, and build an environment where engineers do their best work. - Foster a culture of ownership, accountability, customer obsession, and continuous improvement. Technical and Product Direction - Drive technical and product decisions through design reviews, code reviews, and architectural discussions. - Lead our efforts working with customers with novel or frontier workloads so they can be successful running on Modal. - Translate learnings from frontier customers into a roadmap for our internal optimization platform and user-facing product, so that gains from optimization and research efforts are accessible to all of our users. - Establish standards for reliability and product excellence; ensure the team owns projects end-to-end, from spec through production. Cross-Functional Leadership - Partner with business operations and compute strategy on continuing to develop our strategy for compute purchases across a variety of accelerators. - Collaborate with GTM on product launches, positioning, and improving win rates for inference opportunities. - Help guide the roadmap for teams building the infrastructure platform underlying inference, as well as adjacent product teams. REQUIREMENTS: - 10+ years of industry experience, including 3+ years in a leadership role - Track record building high-performance systems at scale - Strong background in cloud infrastructure - Deep knowledge of low-level OS foundations (Linux kernel, file systems, containers, etc.) - Nice to have: Experience working with LLM inference in production and familiarity with underlying concepts like engines, kernels, routing, KV cache management and speculative decoding.

Dizzaract is a product-driven company operating at the intersection of gaming, digital platforms, and AI, which builds and scales multiple products, including FAR Labs & Gamed. Languages: English (Advanced, C1). Remote work or on-site (UAE).

Зарплата не указана

Location: San Francisco; Remote; Toronto; New York; Montreal Workplace: Hybrid ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLE As a Forward Deployed Engineer at Baseten, you will partner directly with customers to architect, build, and deploy high-scale production AI applications on Baseten’s platform. You’ll own the journey with customers from initial exploration to production deployment, translating ambiguous business goals into reliable, observable services with clear quality, latency, and cost outcomes. This role is a great fit for entrepreneurial engineers who want a front-row view into how modern companies adopt AI at scale and who enjoy working across product, software development, performance engineering, and customer-facing implementations. To be clear, this is an engineering role with hands-on coding and software development that also includes aspects of product management, technical customer success, and pre-sales solution engineering mixed in. EXAMPLE INITIATIVES Take a look at these blog posts written by members of our Forward Deployed Engineering team: - Forward Deployed Engineering on the frontier of AI https://www.baseten.co/blog/forward-deployed-engineering/ - The fastest, most accurate Whisper transcription https://www.baseten.co/blog/the-fastest-most-accurate-and-cost-efficient-whisper-transcription/ - Deploy production-ready model servers from Docker images https://www.baseten.co/blog/deploy-production-model-servers-from-docker-images/ - Deploy custom ComfyUI workflows as APIs https://www.baseten.co/blog/deploying-custom-comfyui-workflows-as-apis/ RESPONSIBILITIES - Develop and maintain software systems and product features using one or more general-purpose programming languages in a production-level environment, with a preference for Python due to its relevance in ML projects. - Drive customer impact by designing, implementing, and deploying Baseten solutions end-to-end (problem framing → evaluation → production deployment → monitoring). This involves working with customers’ engineering teams at every stage of the customer journey including: sales, implementation, and expansion. - Deliver with velocity: turn vague objectives into clear specs and well-defined PoCs so we can rapidly ship well-tested services and outcomes for our customers - Optimize and enhance AI/ML projects, contributing to the continuous improvement of our technical stack. This includes developing features and PRDs with other engineering and product orgs. - Own products and customer projects end-to-end, functioning as both an engineer, project manager, and product manager, with a focus on user empathy, project specification, and end-to-end execution. - Navigate ambiguity and exercise good judgment on tradeoffs and tools needed to solve problems, avoiding unnecessary complexity. - Demonstrate pride, ownership, and accountability for your work, expecting the same from your teammates. REQUIREMENTS - Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or related field. - 2+ years of professional work experience in a fast-paced, high-growth environment. - Demonstrated experience with one or more general-purpose programming languages in a production-level environment, with a strong preference for Python. - Familiarity with AI/ML pipelines and the lifecycle of ML model development and deployment. - Strong communication skills, particularly on complex technical topics. - Experience in building or optimizing AI/ML projects is highly valued. BENEFITS - Competitive compensation, including meaningful equity - (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents - Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!) - Paid parental leave - Fertility and family-building stipend through Carrot - (U.S. only) Company-facilitated 401(k) - Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities. Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you. At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).

Location: New York; San Francisco ABOUT US: AI NEEDS A NEW INFRASTRUCTURE LAYER. WE'RE BUILDING IT AT MODAL. Every era of computing brought new workloads that previous infrastructure couldn't support: mainframes, databases, and the cloud. Each time, the company that rebuilt the layer underneath defined the decade. AI is no different, except it touches everything instead of one slice, and the window to build the layer underneath it is open right now. Our customers include category-defining companies like Ramp https://modal.com/blog/how-ramp-built-a-full-context-background-coding-agent-on-modal, Cognition, DoorDash, and Suno. They rely on Modal for instant GPU access, sub-second container starts, and native storage, so it's simple to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale. We've passed $1B in annualized run rate, up 15x from a year ago. In October 2026 we raised a $750M Series D led by Accel at a $15.75B valuation. Our team includes creators of popular open-source projects (e.g.,Seaborn https://github.com/mwaskom/seaborn,Luigi https://github.com/spotify/luigi), academic researchers, international olympiad medalists, and experienced engineering and product leaders with decades of experience. THE ROLE: Most of the value of owning a model shows up at serving time. We're building a platform that covers the whole life of an LLM -- train it, deploy it, observe it -- and inference is where teams feel the difference every day. We already run elastic inference, sandboxes, distributed volumes, and multi-node training, and we control the infrastructure underneath, so the serving stack is ours to shape rather than something we resell. You will do hands-on inference research at Modal, working with the research lead to pick high-impact bets and owning them end to end. The bets that matter most are the ones that move cost per token and tail latency on the workloads our customers actually run. WHAT YOU'LL DO: - Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization (FP8, INT4), KV-cache and memory management, autoscaling for spiky serverless traffic, and whatever else the research agenda calls for. - Train custom speculators against real production traffic and feed what you learn back into target models -- acceptance length is the metric that decides the win. - Work directly with customers alongside our Forward Deployed Engineers to deploy and tune models, and bring what you learn back into the research. - Carry and expand collaborations with outside research labs, for example: - our work with ZLab on DFlash https://modal.com/blog/spec-is-all-u-need, a speculator design built on KV injection and blockwise parallel drafting - our work with SGLang on specdec https://modal.com/blog/host-overhead-inference-efficiency and multimodal inference https://modal.com/blog/boosting-multimodal-inference-performance-by-greater-than-10-with-a-single-python-dictionary performance - our work on Flash Attention 4 kernels https://modal.com/blog/flash-attention-4-faster - Work with engineering to turn frontier serving techniques into products: primitives for disaggregation, fast weight refresh for models that keep training after deployment, observability for quality and latency in production, or even a next-generation inference engine. - Help shape the research agenda. None of the above is prescriptive; your work will help guide our future. REQUIREMENTS: - A research-leaning or systems background in LLM inference, with work you can point to. - Fluency in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling. - A record of shipping research or systems that other people build on, whether in a lab or in industry. - The drive to independently take a research bet from idea to result, working in the open with the rest of the team. - Ability to work in-person, in our NYC or San Francisco office.

Location: San Francisco; Remote; Toronto; New York; Montreal; Seattle Workplace: Hybrid ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F https://www.baseten.co/blog/announcing-our-series-f/, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLE We’re looking for a seasoned Frontend Engineer to craft performant and delightful user experiences across Baseten’s core platform. You’ll own critical parts of our web application stack and collaborate cross-functionally with product, design, and backend teams to launch impactful features that help users deploy and manage AI systems at scale. EXAMPLE INITIATIVES - Rolling Deployments https://www.baseten.co/blog/rolling-deployments-zero-downtime-model-updates/ - Model APIs for frontier models https://www.baseten.co/products/model-apis/ - Model training built for production inference https://www.baseten.co/products/training/ RESPONSIBILITIES - Design, implement, and maintain responsive, accessible, and user-friendly frontend interfaces using React and TypeScript - Collaborate closely with product designers to turn complex ideas into elegant, intuitive UIs - Optimize application performance and reliability, with a focus on rendering speed and responsiveness - Drive major frontend initiatives, including partnering with backend teams to define APIs and test and refine end-to-end flows - Establish best practices, and mentor other engineers on frontend technologies - Build reusable component libraries and frontend infrastructure that accelerate product development - Partner with backend and platform teams to define and refine APIs and end-to-end flows REQUIREMENTS - 5+ years of experience building production-grade web applications - Deep expertise in React, TypeScript, and modern web development tooling - Track record of building well-crafted, performant, and scalable UIs - Strong product intuition and empathy for the user experience - Experience working in cross-functional product teams in a fast-paced environment - Excellent communication and collaboration skills NICE TO HAVE - Familiarity with Tailwind CSS, Next.js, or other modern frontend frameworks - Background in developer tools or data/ML infrastructure products - Experience with WebSockets, real-time data, or custom visualization tooling - Contributions to open-source projects or public engineering work BENEFITS - Competitive compensation, including meaningful equity - (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents - Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!) - Paid parental leave - Fertility and family-building stipend through Carrot - (U.S. only) Company-facilitated 401(k) - Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities. Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you. At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).