MLOps-инженер вакансии

Location: Anywhere; Europe, Middle East, Africa, Asia-Pacific (EMEA, APAC) Workplace: Remote ABOUT US Circle is building the world’s leading AI-powered, all-in-one platform for digital businesses. We make it possible for creators, coaches, educators, and businesses to bring together their audience with engaging discussions, live streams, events, chat, courses, and payments — all in one place, all under their own brand. We’re proud to be a fully remote company of around 270 (and growing!) team members from 30+ countries around the world. We seek exceptional individuals around the world, set them up to do the best work of their lives, and in turn, create a meaningful impact in their own lives. We don't track hours, but we do manage for high expectations very closely. We collaborate across time zones, are highly async, and like to document a lot. Twice a year, we bring the whole company together in beautiful places around the world for our company offsites. So far, we’ve hosted offsites in Turkey, Portugal, Mexico, Thailand, Colombia, Italy, Ireland, and more, with still more to come! Check out our Careers https://careers.circle.so/ page for more about working at Circle. ABOUT THE ROLE The AI Quality engineering team at Circle owns the foundation for measuring, diagnosing, and improving the quality of Circle's AI-powered features. This team focuses on building the infrastructure to measure, diagnose, and improve production AI systems, rather than ML research or model training. We're looking for a Lead Engineer to help us build out the evaluation frameworks, observability tooling, and diagnostic infrastructure that tell us whether our AI Agents are working well, where to improve them, and how to make them faster and more cost-efficient. This is a hands-on player-coach role. You’ll also lead and manage the AI Quality engineering team, with an ambitious and growing roadmap. If you're excited about making AI systems work reliably, efficiently, and at scale, this is for you. WHAT YOU'LL BE DOING - Build and own our evaluation infrastructure. Design the CI/CD pipelines, scorers, and datasets that tell us whether Circle's AI agents (planners, tool-callers, and sub-agents) are actually working, from a single tool call to a full multi-turn conversation. - Diagnose exactly where quality breaks down. Trace failures across the agent pipeline including plan creation vs. execution, tool selection, tool trajectory in complex areas like workflows, site builder, and analytics. Turn what you find into prototypes that solve the issue or identify priorities for AI core engineering can help. - Grow the datasets that make evaluation possible. Stand up annotation workflows and build out AI-generated and simulated conversations so we can cover more of the product faster than manual labeling alone. - Run structured experiments across prompts, models, and the agent harness. Evaluate new and open-source models against our production baseline, build the framework we use to decide when to shift models, and chase cost and latency wins through model swaps, caching, and routing by plan complexity. - Lead and grow the AI Quality engineering team. Set technical direction and manage day-to-day priorities, all while staying hands-on in the code yourself. - Partner closely with AI Core engineering. Work with the engineers building Circle's AI products so they have real confidence that their changes are actually improving quality, not just shipping. WHAT YOU'LL NEED TO BE SUCCESSFUL - 7+ years of experience building and shipping production software, ideally including LLM-powered agents that take real actions in a product. You've worked on complex, tool-using systems (multiple tools, planning or orchestration, sub-agents) not just simple, single-turn assistants, and you can walk us through something you shipped and how you knew it was actually working. - Comfortable in Ruby on Rails / Python or ready to pick them up quickly. Ruby on Rails is our production system and the foundation that Circle’s AI Agents are built on and proficiency in Python is a strong plus, especially for the data and evaluation side of the work. - Experience building evaluation or observability infrastructure for ML/AI systems. You've built eval pipelines, scorers, dashboards, or CI/CD for evals before and have experience with evaluation frameworks like Braintrust, LangSmith, or similar. - Experience designing datasets, annotation workflows, or labeling pipelines for ML/AI evaluation. You know how to turn raw examples into a dataset you can trust through human annotation, AI-generated data, or simulation. - Learn fast, experiment aggressively, and thrive in a highly dynamic environment. You’ll need to be comfortable running a lot of experiments and letting the results settle the argument, including the ones that don't work (which will be many of them in the beginning). - Comfortable in a fast-paced environment with ambiguity. You’ll need to learn and pick up new technologies when projects require it. - Strong alignment with our company values https://circle.so/careers. - You are proficient in English (spoken, written, and reading) at a CEFR Level C2 https://magoosh.com/english-speaking/english-proficiency-levels-a-guide-to-determining-your-level/ / ILR Level 5 https://www.govtilr.org/.   COMPENSATION & BENEFITS Circle offers U.S.-benchmarked compensation globally, equity in the company with ongoing refresh grants, and 35 days of paid time off each year. We’re a remote-only team that comes together twice a year for company retreats in incredible destinations around the world. Alongside incredible flexibility and autonomy, we offer a benefits package that supports health, wellbeing, and professional growth. Learn more in our Candidate Hub. https://circleco.notion.site/circle-candidate-hub LEARN MORE - Candidate Safety & Interview Process Notice https://app.notion.com/p/circleco/Circle-Candidate-Hub-374b655313b7803d9580fb7da4f51acf?source=copy_link#374b655313b780e5b681c4d56b962790 - Diversity, Equity & Inclusion https://app.notion.com/p/circleco/Circle-Candidate-Hub-374b655313b7803d9580fb7da4f51acf?source=copy_link#374b655313b7807a8e3ff5c25a1d93ba - How We Use Candidate Data https://app.notion.com/p/circleco/Circle-Candidate-Hub-374b655313b7803d9580fb7da4f51acf?source=copy_link#374b655313b780e49e98f390eec6bb51 - Equal Employment Opportunity https://app.notion.com/p/circleco/Circle-Candidate-Hub-374b655313b7803d9580fb7da4f51acf?source=copy_link#374b655313b7807a967dea67e68f31f0 - Visit our Candidate Hub https://circleco.notion.site/circle-candidate-hub to learn more about working at Circle, our benefits, and our hiring process.

Развитие и поддержка ML-инфраструктуры (Linux, Docker, Kubernetes, Python, GitLab CI, Git, Kubeflow, Prometheus, Grafana Airflow S3 / MinIO HelmPyTorch / TensorFlow Fine-tuning — LoRA, QLoRA, PEFT Дистилляция RAG,Hugging Face Quantization, ONNX, TensorRT). Интеграция LLM и других моделей в продакшн-окружение.

Ищем DevOps/MLOps инженера в проектную команду для сопровождения и развития платформы машинного обучения и AI-сервисов (LLM, агенты, чат-боты). развёртыванием ML-моделей и AI-сервисов, локальных моделей (LLM, ML-модели) в различных средах;

Развитие классификатора данных для полей БД (Tabular ML). Развитие существующей ML-модели (RandomForest), которая выявляет ПДн в полях баз данных на основе метаданных и значений.

Задачи: — Руководство проектированием, разработкой и внедрением on-premises MLOps решений для поддержки доставки моделей машинного обучения — Сотрудничество с дата-сайентистами и разработчиками для создания масштабируемых и эффективных конвейеров данных, систем обучения и деплоя моделей — Разработка и поддержка инструментов мониторинга и управления для обеспечения надежности и производительности MLOps инфраструктуры — Взаимодействие с заинтересованными сторонами для понимания их потребностей в машинном обучении и разработки соответствующих решений — Отслеживание новейших трендов и технологий в области MLOps, машинного обучения и искусственного интеллекта, обмен знаниями с командой — Наставничество младших членов команды и помощь в развитии их навыков в MLOps — Поиск и устранение проблем пользователей Требования: — Опыт работы 5+ лет в MLOps, разработке ПО, инженерии данных или смежных областях — Сильные навыки программирования на современных языках (Java, Scala, Python, Kotlin) — Практический опыт работы с Kubernetes: деплой и эксплуатация рабочих нагрузок, отладка сбоев, работа с основными концепциями (Deployments, ConfigMaps/Secrets, Services, resource limits) — Отличные коммуникативные и командные навыки — Ценность простоты и производительности кода Будет плюсом: — Понимание жизненного цикла ML-проектов — Опыт работы с MLOps платформами, такими как Kubeflow или MLFlow — Опыт с ML-фреймворками: scikit-learn, LGBM, Tensorflow, PyTorch — Опыт работы с Big Data инструментами: Spark, S3, Hadoop — Глубокие знания контейнеризации и оркестрации (Docker, Kubernetes) — Опыт DevOps и CI/CD практик — Опыт создания и масштабирования инструментов для обслуживания моделей, включая построение API с жесткими SLA — Страсть к инженерной стороне машинного обучения и масштабированию — Опыт проектирования и построения MLOps инфраструктуры, включая конвейеры данных, системы обучения и деплоя моделей, а также инструменты мониторинга и управления.

На проекте банковского заказчика предстоит создавать AI-агентов, строить MLOps-инфраструктуру и работать с данными для ML. создавать AI-агенты и платформы для них;

• От 3 лет опыта в ML Engineering, Data Engineering или DevOps с развёртыванием и поддержкой ML-систем в продакшене. • Опыт построения ML training pipelines с experiment tracking и model registry (MLflow, W&B или аналоги).

Задачи: — Управление и валидация кредитных моделей — Управление рисками моделей Требования: — Опыт работы с нейронными сетями, машинным обучением, XGBoost — Знание принципов управления моделями и оценки рисков — Навыки валидации моделей и документации Условия: — Работа в гибридном формате в офисе в Сан-Франциско — Зарплата от $207,600 до $306,600 в год

• Опыт MLOps от 4 лет.

• Глубокое понимание и опыт работы с Docker и Kubernetes (K8s). • Опыт построения CI/CD for ML.

Location: Remote Workplace: Remote We are building AI to simulate the world through merging art and science. We believe that world models are at the frontier of progress in artificial intelligence. Language models alone won’t solve the world’s hardest problems – robotics, disease, scientific discovery. Real progress requires models that experience the world and learn from their mistakes, the same way that humans do. And this kind of trial and error can be massively accelerated when done in simulation, rather than in the real world. World models offer the most clear path to general-purpose simulation, changing how stories are told, how scientific progress is made and how the next frontiers of humanity are reached. Our team consists of creative, open minded, caring and ambitious people who are determined to change the world. We aspire to continuously build impossible things and our ability to do so relies on building an incredible team. If you are driven to do the same, we'd love to hear from you. *Titles posted to our careers page are updated regularly to best advertise the role and expected seniority, but internally we have a flat titling system with two types of titles: one for individual contributors and one for people managers. ABOUT THE ROLE We're looking for an ML infrastructure engineer to own model evaluation at Runway, end to end. Every decision we make about a model – which checkpoint to keep training, what to ship to millions of users, which datamix and architecture shows the most promise – rests on evals. Today that work is spread across the organization. You'll turn it into one platform. Your job will be to design and build the systems that generate samples at scale, score them with automated metrics and human annotations, track results across checkpoints and releases, and put the answers in front of researchers in minutes rather than days. You'll define what "better" means operationally – how we measure it, how confident we are, and how a result becomes a ship/no-ship decision. You'll be embedded within research teams as a member of ML Platform, and you'll set the technical direction for evals across the company. This is an extremely high-leverage role: the quality of our models is bounded by how well we can measure them. A PEEK AT OUR TECHNICAL STACK Our inference and training code is written in Python (PyTorch) and is cloud-native. We leverage multiple clouds and use a combination of Kubernetes native and home-grown tooling for efficient job orchestration. We’re big users of agentic development and operations. We have a robust internal platform and provide broad access to the latest models and harnesses. You’ll have access to best-in-class models, agents, GPUs, storage and cloud services you need to drive a world class evaluation pipeline that helps us deliver frontier models. WHAT YOU'LL DO - Own the evaluation platform end to end: the tooling and systems required to generate, annotate, review and adapt at frontier scale - Define the CLIs, APIs, GUIs and storage layers required to make this process seamless, fast, sophisticated and collaborative - Work directly with research teams on video, image, audio, agents, and robotics to understand what they need to measure and build it, then generalize the result into the platform - Help set the standards for how we evaluate models at Runway: reproducibility, metric definitions, reporting formats, and when a result is trustworthy enough to act on - Support broad adoption of the platform and its integration throughout Runway’s research efforts across training, production model serving and more - Contribute broadly as a member of the ML Platform team to tools and systems that help Runway train and serve frontier models WHAT YOU'LL NEED - 5+ years of experience building ML infrastructure or data platforms in production environments, with at least some of that time spent on evaluation, experimentation, or benchmarking systems - Strong Python and PyTorch, and hands-on experience running large batch GPU workloads on Kubernetes - Experience designing data pipelines and storage for large volumes of media or model outputs, with attention to versioning and reproducibility - Familiarity with experimental statistics: paired comparisons, confidence intervals, multiple-comparison pitfalls, inter-rater agreement - Comfort building internal tools end to end, from the command line to the browser - Ability to lead a broad technical area: gather requirements from many teams, set direction, make tradeoffs, and drive a roadmap without waiting to be told - Familiarity with the full model development lifecycle: data, training, evaluation, serving - Self-starter who can work embedded with research teams and move fast - Strong systems thinking and pragmatic approach to production reliability - Humility and open mindedness; at Runway we love to learn from one another NICE TO HAVE - Track record of building evaluation suites for generative models - Hands-on work with LLM- or VLM-as-judge pipelines - Experience with online experimentation platforms and connecting offline metrics to product outcomes - Prior work evaluating agents or robotics policies Runway strives to recruit and retain exceptional talent from diverse backgrounds while ensuring pay equity for our team. Our salary ranges are based on competitive market rates for our size, stage and industry, and salary is just one part of the overall compensation package we provide. There are many factors that go into salary determinations, including relevant experience, skill level and qualifications assessed during the interview process, and maintaining internal equity with peers on the team. The range shared below is a general expectation for the function as posted, but we are also open to considering candidates who may be more or less experienced than outlined in the job description. In this case, we will communicate any updates in the expected salary range. Lastly, the provided range is the expected salary for candidates in the U.S. Outside of those regions, there may be a change in the range, which again, will be communicated to candidates. WORKING AT RUNWAY Great things come from great teams. https://www.youtube.com/watch?v=kwmj4ato2kw&ab_channel=Runway We’d love to hear from you. We’re committed to creating a space where our employees can bring their full selves to work and have equal opportunity to succeed. So regardless of race, gender identity or expression, sexual orientation, religion, origin, ability, age, veteran status, if joining this mission speaks to you, we encourage you to apply. More about Runway - Universal World Simulator https://runwayml.com/world-simulator.html - GWM-1 https://runwayml.com/research/introducing-runway-gwm-1 - Gen-4.5 https://runwayml.com/research/introducing-runway-gen-4.5 - General World Models https://runwayml.com/research/introducing-general-world-models - Robotics SDK https://runwayml.com/research/introducing-runway-gwm-1#robotics-section - Conversational Real-time Agents https://runwayml.com/research/introducing-runway-gwm-1#avatars-section - Runway Studios https://runwayml.com/studios We're excited to be recognized as a best place to work: Crain's https://www.crainsnewyork.com/awards/best-places-work-2023 | InHerSight https://www.inhersight.com/companies/best/city/new-york-city-ny | BuiltIn NYC https://builtin.com/awards/new-york-city/2024/best-places-to-work | INC https://www.inc.com/best-workplaces/2024

-управлять командой, состоящей из DS, DE, MLOps. -соблюдать требования и стандарты производственного процесса банка.

Создавать систему эффективного хранения больших данных и доступа к ним для ML-разработчиков. • Майнинг данных.

-Развивать и поддерживать инфраструктуру DWH. -Строить и развивать CI/CD-процессы для data и ML-разработки.

— Разрабатывать Python / SQL-пайплайны загрузки и обработки данных из файлов, API и информационных систем. — Проектировать таблицы хранилища: сущности, связи, правила преобразования.

Предстоит: Проводить мониторинг и обслуживание AI-моделей; Управлять инфраструктурой в виде GPU-серверов;

Требуется ML Ops-инженер, который возьмёт на себя задачи автоматизации полного жизненного цикла машинного обучения. В обязанности входит организация процессов разработки, тестирования, развертывания моделей, а также их мониторинг и поддержка в рабочей среде.

As the Senior Staff Machine Learning Platform Engineer, you will own the technical vision and evolution of Faire’s ML platform. You will set standards, influence org-wide architecture, and lead complex, cross-functional initiatives that unlock data science velocity at scale.