Talablar:
— Python, Vibe Coding, Yolo, Opencv, Transformers-deffusion Architecture bo'yicha bilim;
— Ishga mas'uliya bilan yondasish;
— Deadline ga amal qilish;
— Tajirba va o'sishga ochiqlik.
Ish holati: 8:30-17:00, offline, Farg'ona, qo'qon.
Murojaat uchun: +998 97 691 91 91, https://t.me/Otabek_rizayev
Задачи:
— Повышение качества RAG-агентов по трём направлениям: поиск по технической документации, нормативной базе, бухгалтерской документации;
— Промпт-инжиниринг и контекст-инжиниринг для всех агентов команды;
— Подготовка эталонных наборов оценки качества;
— Реализация LLM-as-a-judge для автоматических регрессионных проверок;
— Настройка защит для production-агентов: фильтры от инъекций в промпт, валидация структуры и контента выходов, защита от утечек персональных данных, anti-hallucination механизмы;
— A/B-тестирование промптов и моделей для подбора лучших конфигураций;
— Тюнинг качества по сигналам обратной связи пользователей и трассировкам из Langfuse.
Требования:
— Python от 2 лет коммерческой разработки;
— Практический опыт RAG-систем в продакшене: эмбеддинги, векторные базы (Qdrant, FAISS или pgvector), переранжирование, чанкинг;
— Опыт построения и поддержки как минимум одного RAG-решения от начала до конца, не прототип;
— Практический опыт оценки качества LLM-систем: подготовка эталонных наборов, offline-метрики, LLM-as-judge, регрессионные проверки;
— Опыт работы с фреймворками оценки (Ragas, DeepEval или аналоги);
— Практический опыт настройки защит для LLM-приложений: защита от инъекций в промпт, валидация структуры и контента выходов, защита от утечек персональных данных;
— Опыт промпт-инжиниринга и контекст-инжиниринга в реальных проектах: итеративная настройка промптов, structured output, function calling;
— Понимание архитектуры RAG: стратегии нарезки документов, метаданные, выбор моделей эмбеддингов, переранжирование, точность ссылок на источник;
— Практический опыт работы хотя бы с одним LLM-фреймворком: LangChain, LangGraph, PydanticAI, OpenAI API или аналоги;
— Практический опыт A/B-тестирования промптов и моделей в продакшене;
— Опыт работы с агентскими протоколами (MCP) или собственным tool-layer для агентов;
— SQL и работа с реляционными базами на базовом уровне.
Location: Germany; United Kingdom; Spain; Italy; The Netherlands
Workplace: Remote
ClickHouse is the fastest-growing database in the OLAP space, and AI is reshaping how developers and enterprises interact with their data. From ClickHouse Agents and AI Functions to the Remote MCP Server, LibreChat, and Langfuse, AI/ML capabilities are rapidly becoming a core pillar of the ClickHouse Cloud experience and a key growth vector for the platform.
We are looking for a Senior Product Manager to own core areas of our agentic analytics experience. You will work at the intersection of large language models, developer tooling, and high-performance analytics, shaping how organizations let agents query, reason over, and act on their data. This is a high-ownership role: you will drive products from discovery to launch with autonomy expanding as fast as you build confidence and context.
WHAT WILL YOU DO?
- Roadmap Ownership: Own the roadmap for key surfaces of the agentic analytics portfolio, from discovery through launch and iteration, and ensure they are best-in-class in the analytics database space.
- Customer Discovery: Talk to users constantly — developers, data engineers, and platform teams deploying agents against production data — and turn their feedback into crisp priorities.
- Product Quality: Define and prioritize features that improve agent accuracy, latency, cost, and trust. Instrument the product with evals and observability (Langfuse) so that quality is measured.
- Market & Competitive Intelligence: Track the competitive landscape and translate it into positioning, differentiation, and gaps to close.
- Go-to-Market Partnership: Partner with marketing, sales, and developer relations on pricing, packaging, launch content, and field enablement.
- Communication: Write clearly and often, including public-facing content that explains what we built and why.
WHAT YOU BRING ALONG:
- 3+ years of product management experience in developer tools, data infrastructure, or ML/AI products
- Hands-on familiarity with LLMs and agentic workflows: you have built or shipped something with them
- Comfort with SQL and analytical databases; you can query your own product data without waiting for someone else
- Strong written communication and the ability to drive alignment across engineering, design, and go-to-market
- You have a high level of ownership and a bias toward shipping: you scope pragmatically, cut ruthlessly, and iterate in public
- You thrive in collaborative environments and can effectively communicate technical concepts to diverse stakeholders
NICE TO HAVE
- Experience with LLM observability, evals, or inference infrastructure
- Experience pricing usage-based or token-based products
- Prior work on open source products or with open source communities
- Understanding of database systems and data processing pipelines, with ClickHouse experience being a significant plus
PERKS
- Flexible work environment - ClickHouse is a globally distributed company and remote-friendly. We currently operate in over 25 countries.
- Healthcare - Employer contributions towards your healthcare.
- Equity in the company - Every new eligible team member who joins our company receives equity.
- Time off - Flexible time off in the US, generous entitlement in other countries.
- A USD$500 Home office setup if you’re a remote employee.
- Global Gatherings – We believe in the power of in-person connection and offer opportunities to engage with colleagues at company-wide offsites.
CULTURE - WE ALL SHAPE IT
As part of a rapidly scaling start-up, you will be instrumental in shaping our culture.
Are you interested in finding out more about our culture? Learn more about our values here https://clickhouse.com/company/careers. Check out our blog posts https://clickhouse.com/blog or follow us on LinkedIn https://www.linkedin.com/company/clickhouseinc/life/0c1ee7f4-2129-488d-b413-e7f6270659fd/?viewAsMember=true to find out more about what’s happening at ClickHouse.
EQUAL OPPORTUNITY & PRIVACY
ClickHouse provides equal employment opportunities to all employees and applicants and prohibits discrimination and harassment of any type based on factors such as race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
Please see here https://clickhouse.com/legal/applicant-privacy-notice for our Privacy Statement.
Location: NAMER and LatAm
ClickHouse is the fastest-growing database in the OLAP space, and AI is reshaping how developers and enterprises interact with their data. From ClickHouse Agents and AI Functions to the Remote MCP Server, LibreChat, and Langfuse, AI/ML capabilities are rapidly becoming a core pillar of the ClickHouse Cloud experience and a key growth vector for the platform.
We are looking for a Senior Product Manager to own core areas of our agentic analytics experience. You will work at the intersection of large language models, developer tooling, and high-performance analytics, shaping how organizations let agents query, reason over, and act on their data. This is a high-ownership role: you will drive products from discovery to launch with autonomy expanding as fast as you build confidence and context.
WHAT WILL YOU DO?
- Roadmap Ownership: Own the roadmap for key surfaces of the agentic analytics portfolio, from discovery through launch and iteration, and ensure they are best-in-class in the analytics database space.
- Customer Discovery: Talk to users constantly — developers, data engineers, and platform teams deploying agents against production data — and turn their feedback into crisp priorities.
- Product Quality: Define and prioritize features that improve agent accuracy, latency, cost, and trust. Instrument the product with evals and observability (Langfuse) so that quality is measured.
- Market & Competitive Intelligence: Track the competitive landscape and translate it into positioning, differentiation, and gaps to close.
- Go-to-Market Partnership: Partner with marketing, sales, and developer relations on pricing, packaging, launch content, and field enablement.
- Communication: Write clearly and often, including public-facing content that explains what we built and why.
WHAT YOU BRING ALONG:
- 3+ years of product management experience in developer tools, data infrastructure, or ML/AI products
- Hands-on familiarity with LLMs and agentic workflows: you have built or shipped something with them
- Comfort with SQL and analytical databases; you can query your own product data without waiting for someone else
- Strong written communication and the ability to drive alignment across engineering, design, and go-to-market
- You have a high level of ownership and a bias toward shipping: you scope pragmatically, cut ruthlessly, and iterate in public
- You thrive in collaborative environments and can effectively communicate technical concepts to diverse stakeholders
NICE TO HAVE
- Experience with LLM observability, evals, or inference infrastructure
- Experience pricing usage-based or token-based products
- Prior work on open source products or with open source communities
- Understanding of database systems and data processing pipelines, with ClickHouse experience being a significant plus
PERKS
- Flexible work environment - ClickHouse is a globally distributed company and remote-friendly. We currently operate in over 25 countries.
- Healthcare - Employer contributions towards your healthcare.
- Equity in the company - Every new eligible team member who joins our company receives equity.
- Time off - Flexible time off in the US, generous entitlement in other countries.
- A USD$500 Home office setup if you’re a remote employee.
- Global Gatherings – We believe in the power of in-person connection and offer opportunities to engage with colleagues at company-wide offsites.
CULTURE - WE ALL SHAPE IT
As part of a rapidly scaling start-up, you will be instrumental in shaping our culture.
Are you interested in finding out more about our culture? Learn more about our values here https://clickhouse.com/company/careers. Check out our blog posts https://clickhouse.com/blog or follow us on LinkedIn https://www.linkedin.com/company/clickhouseinc/life/0c1ee7f4-2129-488d-b413-e7f6270659fd/?viewAsMember=true to find out more about what’s happening at ClickHouse.
EQUAL OPPORTUNITY & PRIVACY
ClickHouse provides equal employment opportunities to all employees and applicants and prohibits discrimination and harassment of any type based on factors such as race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
Please see here https://clickhouse.com/legal/applicant-privacy-notice for our Privacy Statement.
В Альфа-Банке расширяем команду AI Delivery и ищем AI/ML Delivery Manager для управления разработкой и промышленным внедрением решений на базе искусственного интеллекта и машинного обучения.
Location: Ukraine; Spain; Serbia; Yerevan; Poland
Workplace: remote
Provectus is an AWS Premier Partner and an Anthropic Strategic Partner, working at the frontier of applied AI. We help enterprises turn Claude, agentic systems, and their own data into measurable business outcomes — through bespoke applications, managed services, and advisory engagements. With offices in North America, LATAM, and EMEA, we partner with clients worldwide.
Our work centers on two verticals — Financial Services & Insurance and Healthcare & Life Sciences — where we deploy five pre-built AI Blueprints: Submission Flow, Portfolio Lens, Asset Flow, Revenue Flow, and Evidence Lens. Each Blueprint rebuilds a critical business process front to back, shipped from working code and tuned to a client's specific book, regulators, and operating posture.
Our team holds 100+ AWS certifications, is Claude Code certified, and co-delivers Anthropic's Agentic SDLC program, Cowork Activation, and AI Blueprint engagements.
Where this role sits
You will work in a senior pod alongside an FDE, an FDX, and Senior AI Engineers — contributing to real delivery work while building toward independent ownership.
Requirements:
Mindset
Proactive and self-directed; you push for clarity rather than waiting for a ticket.
Excellent communication and problem-solving skills.
Comfortable with some ambiguity, with support from senior team members as you take on more.
B2+ English, comfortable collaborating across distributed, multicultural teams.
Technical depth
Hands-on experience building or contributing to RAG systems, ideally in a production or near-production setting.
Solid engineering fundamentals; Python and/or TypeScript proficiency. Productive in an unfamiliar codebase with some ramp-up support.
Practical AWS experience (Lambda, S3, ECS, or similar); ready to grow into Bedrock and Bedrock AgentCore .
Some experience with containers and CI/CD in real projects.
Exposure to evaluating non-deterministic systems — you've contributed to or run test/eval cycles, even if you haven't owned a full eval suite end-to-end.
Basic working knowledge of model/agent monitoring concepts.
Awareness of cost and latency trade-offs when working with LLMs.
Some hands-on exposure to the Claude ecosystem (Claude Code, CLAUDE.md, hooks, skills files) is a plus, or strong ability to ramp up quickly.
Practical experience with LLM APIs (Anthropic, AWS Bedrock, or OpenAI) in real projects.
2+ years of software or ML engineering experience, including some exposure to production systems.
Solid AI/ML foundations — you understand what the models do well enough to reason about common failure modes.
Nice to Have:
Experience in one of the industries: financial services, insurance, healthcare.
Consulting, professional services, or other embedded customer-facing delivery.
AWS and Claude Code Certifications (or actively pursuing them).
A2A: Interest in agent-to-agent interoperability concepts.
CI/CD pipeline experience (GitHub Actions, GitLab CI).
Practical experience with one or more use cases from the following: NLP, LLMs, and Recommendation engines.
Experience in an additional language (Go, TypeScript, or Rust).
Experience with Apache Spark, Apache Airflow, Kafkа.
Experience with the Claude ecosystem — Claude Code, CLAUDE.md, hooks, skills files. Spec-driven development — writing the intent, constraints, and acceptance criteria before you let an agent build.
MCP: you can say why an agent would prefer it to a REST integration, having authored a server is an additional plus.
Responsibilities:
Build and contribute to RAG system components under senior guidance, with growing autonomy.
Write tests and help build out evaluation harnesses for the features you work on.
Write production code across the stack (AI, backend services, data pipelines) with code review support.
Help integrate AI components into backend services and RESTful APIs.
Support deployment of systems to AWS (containerized, CI/CD), taking on more of this independently over time.
Contribute to documentation, runbooks, and client handover materials.
Participate in technical discussions and architectural decisions, with an eye toward taking on more of this independently.
Support model evaluation efforts and help investigate and improve failure modes.
Take on increasing ownership of components and technical decisions as you grow in the role.
What We Offer:
The chance to shape how leading enterprises across LATAM, Europe, and North America adopt AI, from strategy through first deployment
A forward-deployed model working in small, senior teams alongside FDE and FDX
A growing AI delivery practice where you help build the tooling and frameworks, not just use them
Remote-friendly culture
Internal training programs with full support for Claude, AWS, and other professional certifications, conference attendance
Career growth; we actively develop our engineers
Access to the latest AI tools and premium subscriptions
Long-term B2B collaboration
Private medical insurance or a budget for your medical needs
Paid sick leave, vacation, and public holidays
Equipment and all the tech you need for comfortable, productive work
How we hire:
Intro conversation. The role, your background and aspirations, tech questions.
Technical interview with live engineering sessions. Real problems, your own editor, you may use an LLM assistant
HR Interview. Soft skills and expectations
HM interview. Tech questions; a live engineering session is also possible
Location: Ukraine; Spain; Slovakia; Serbia; Yerevan; Slovenia; Poland
Workplace: remote
Provectus is an AWS Premier Partner and an Anthropic Strategic Partner, working at the frontier of applied AI. We help enterprises turn Claude, agentic systems, and their own data into measurable business outcomes — through bespoke applications, managed services, and advisory engagements. With offices in North America, LATAM, and EMEA, we partner with clients worldwide.
Our work centers on two verticals — Financial Services & Insurance and Healthcare & Life Sciences — where we deploy five pre-built AI Blueprints: Submission Flow, Portfolio Lens, Asset Flow, Revenue Flow, and Evidence Lens. Each Blueprint rebuilds a critical business process front to back, shipped from working code and tuned to a client's specific book, regulators, and operating posture.
Our team holds 100+ AWS certifications, is Claude Code certified, and co-delivers Anthropic's Agentic SDLC program, Cowork Activation, and AI Blueprint engagements.
Where this role sits
You will work in a small, senior pod alongside an FDE and an FDX,
Forward Deployed Executives (FDX) own the commercial relationship and the business outcome. Works alongside the client's leadership or C-suite level to move the client's KPIs.
Forward Deployed Engineers (FDE) embed with a client, map the client workflow, identify the business problem underneath it, design and build a working AI solution, present to the client, and transfer the knowledge to the client's team. Owns technical direction of the whole solution.
Senior AI Engineer. When an FDE comes back from the client with the business problem, you will help turn that into an agentic system that runs in production and will be responsible for evaluation, observability, and guardrails. You'll have real ownership of components and of the technical decisions inside them.
Requirements:
Mindset
Proactive and self-directed; you push for clarity rather than waiting for a ticket
Excellent communication and problem-solving skills
Comfort with ambiguity and ownership.
B2+ English, comfortable collaborating across distributed, multicultural teams.
Technical depth
5+ years in software or ML engineering, with production systems you were accountable for.
Solid AI/ML foundations. You understand what the models do well enough to reason about failure modes.
Shipped to production LLM applications and agentic workflows — not demos, not POCs, not notebooks.
Agentic orchestration: multi-step workflows, graph-based orchestration, tool use, state management, and recovery from partial failure
Experience with LLM APIs (Anthropic, AWS Bedrock, or OpenAI) and agent frameworks.
Experience building and optimizing RAG systems in production.
Strong engineering fundamentals. Full-stack mindset, comfortable across AI, backend development, and cloud infrastructure. Python and/or TypeScript proficiency; depth matters more than stack. Dropped into an unfamiliar codebase, you're productive.
Hands-on AWS in production: Bedrock, Bedrock AgentCore, Lambda, ECS, S3, SQS, ECR, or similar. GCP or Azure is a plus.
Cloud-native delivery: containers, ECS or Kubernetes, IaC, and CI/CD applied to AI pipelines.
You evaluate. You have built or owned an eval suite for a non-deterministic system, and you can explain what you measured, how you produced ground truth, and what gated a release.
Model and agent monitoring, drift detection.
Cost and latency discipline: model tiering, caching, and the ability to say what a workload costs to run before it runs.
Hands-on production experience with the Claude ecosystem — Claude Code, CLAUDE.md, hooks, skills files. Spec-driven development — writing the intent, constraints, and acceptance criteria before you let an agent build — is a strong plus.
MCP: you can say why an agent would prefer it to a REST integration. Having authored a server is a plus.
Nice to Have:
Experience in one of the industries: financial services, insurance, healthcare.
Consulting, professional services, or other embedded customer-facing delivery.
AWS and Claude Code Certifications
A2A: you can explain agent-to-agent interoperability
CI/CD pipeline experience (GitHub Actions, GitLab CI)
Practical experience with one or more use cases from the following: NLP, LLMs, and Recommendation engines.
Experience in an additional language (Go, TypeScript, or Rust).
Experience with Apache Spark, Apache Airflow, Kafkа
Responsibilities:
Work in a pair with an FDE and an FDX.
Build and ship production GenAI systems into the customer’s environment (cloud-native data, LLM-based, and agentic AI solutions).
Build and optimize RAG systems for production use cases
Build the evaluation harness before you build the feature.
Write production code across the stack — AI, backend services, data pipelines. We choose tools to fit the customer.
Integrate AI components into backend services and RESTful APIs
Take systems to production on AWS (GCP or Azure where the customer requires it): containerised, CI/CD. Implement LLMOps and AgentOps practices: agent tracing, prompt and version management, cost and latency monitoring, regression testing, drift detection
Start from the blueprint, contribute to enablement and handover: clear documentation, runbooks, and pairing with the client engineers who will inherit the system. Feed reusable components and lessons back into the Provectus Blueprints
Participate in technical discussions and architectural decisions
Conduct model evaluation, improve failure modes you find, optimize model performance, efficiency, and reliability
Mentor junior and mid-level AI engineers, conduct code reviews and share knowledge across the team through documentation, presentations, and workshops.
What We Offer:
The chance to shape how leading enterprises across LATAM, Europe, and North America adopt AI, from strategy through first deployment
A forward-deployed model working in small, senior teams alongside FDE and FDX
A growing AI delivery practice where you help build the tooling and frameworks, not just use them
Remote-friendly culture
Internal training programs with full support for Claude, AWS, and other professional certifications, conference attendance
Career growth; we actively develop our engineers
Access to the latest AI tools and premium subscriptions
Long-term B2B collaboration
Private medical insurance or a budget for your medical needs
Paid sick leave, vacation, and public holidays
Equipment and all the tech you need for comfortable, productive work
How we hire:
Intro conversation. The role, your background and aspirations, tech questions.
Technical interview with live engineering sessions. Real problems, your own editor, you may use an LLM assistant
HR Interview. Soft skills and expectations
HM interview. Tech questions; a live engineering session is also possible
Location: North Macedonia; Bosnia and Herzegovina; Czechia; Croatia; Moldova; Bulgaria; Montenegro
Workplace: remote
Provectus is an AWS Premier Partner and an Anthropic Strategic Partner, working at the frontier of applied AI. We help enterprises turn Claude, agentic systems, and their own data into measurable business outcomes — through bespoke applications, managed services, and advisory engagements. With offices in North America, LATAM, and EMEA, we partner with clients worldwide.
Our work centers on two verticals — Financial Services & Insurance and Healthcare & Life Sciences — where we deploy five pre-built AI Blueprints: Submission Flow, Portfolio Lens, Asset Flow, Revenue Flow, and Evidence Lens. Each Blueprint rebuilds a critical business process front to back, shipped from working code and tuned to a client's specific book, regulators, and operating posture.
Our team holds 100+ AWS certifications, is Claude Code certified, and co-delivers Anthropic's Agentic SDLC program, Cowork Activation, and AI Blueprint engagements.
Where this role sits
You will work in a small, senior pod alongside an FDE and an FDX,
Forward Deployed Executives (FDX) own the commercial relationship and the business outcome. Works alongside the client's leadership or C-suite level to move the client's KPIs.
Forward Deployed Engineers (FDE) embed with a client, map the client workflow, identify the business problem underneath it, design and build a working AI solution, present to the client, and transfer the knowledge to the client's team. Owns technical direction of the whole solution.
Senior AI Engineer. When an FDE comes back from the client with the business problem, you will help turn that into an agentic system that runs in production and will be responsible for evaluation, observability, and guardrails. You'll have real ownership of components and of the technical decisions inside them.
Requirements:
Mindset
Proactive and self-directed; you push for clarity rather than waiting for a ticket
Excellent communication and problem-solving skills
Comfort with ambiguity and ownership.
B2+ English, comfortable collaborating across distributed, multicultural teams.
Technical depth
5+ years in software or ML engineering, with production systems you were accountable for.
Solid AI/ML foundations. You understand what the models do well enough to reason about failure modes.
Shipped to production LLM applications and agentic workflows — not demos, not POCs, not notebooks.
Agentic orchestration: multi-step workflows, graph-based orchestration, tool use, state management, and recovery from partial failure
Experience with LLM APIs (Anthropic, AWS Bedrock, or OpenAI) and agent frameworks.
Experience building and optimizing RAG systems in production.
Strong engineering fundamentals. Full-stack mindset, comfortable across AI, backend development, and cloud infrastructure. Python and/or TypeScript proficiency; depth matters more than stack. Dropped into an unfamiliar codebase, you're productive.
Hands-on AWS in production: Bedrock, Bedrock AgentCore, Lambda, ECS, S3, SQS, ECR, or similar. GCP or Azure is a plus.
Cloud-native delivery: containers, ECS or Kubernetes, IaC, and CI/CD applied to AI pipelines.
You evaluate. You have built or owned an eval suite for a non-deterministic system, and you can explain what you measured, how you produced ground truth, and what gated a release.
Model and agent monitoring, drift detection.
Cost and latency discipline: model tiering, caching, and the ability to say what a workload costs to run before it runs.
Hands-on production experience with the Claude ecosystem — Claude Code, CLAUDE.md, hooks, skills files. Spec-driven development — writing the intent, constraints, and acceptance criteria before you let an agent build — is a strong plus.
MCP: you can say why an agent would prefer it to a REST integration. Having authored a server is a plus.
Nice to Have:
Experience in one of the industries: financial services, insurance, healthcare.
Consulting, professional services, or other embedded customer-facing delivery.
AWS and Claude Code Certifications
A2A: you can explain agent-to-agent interoperability
CI/CD pipeline experience (GitHub Actions, GitLab CI)
Practical experience with one or more use cases from the following: NLP, LLMs, and Recommendation engines.
Experience in an additional language (Go, TypeScript, or Rust).
Experience with Apache Spark, Apache Airflow, Kafkа
Responsibilities:
Work in a pair with an FDE and an FDX.
Build and ship production GenAI systems into the customer’s environment (cloud-native data, LLM-based, and agentic AI solutions).
Build and optimize RAG systems for production use cases
Build the evaluation harness before you build the feature.
Write production code across the stack — AI, backend services, data pipelines. We choose tools to fit the customer.
Integrate AI components into backend services and RESTful APIs
Take systems to production on AWS (GCP or Azure where the customer requires it): containerised, CI/CD. Implement LLMOps and AgentOps practices: agent tracing, prompt and version management, cost and latency monitoring, regression testing, drift detection
Start from the blueprint, contribute to enablement and handover: clear documentation, runbooks, and pairing with the client engineers who will inherit the system. Feed reusable components and lessons back into the Provectus Blueprints
Participate in technical discussions and architectural decisions
Conduct model evaluation, improve failure modes you find, optimize model performance, efficiency, and reliability
Mentor junior and mid-level AI engineers, conduct code reviews and share knowledge across the team through documentation, presentations, and workshops.
What We Offer:
The chance to shape how leading enterprises across LATAM, Europe, and North America adopt AI, from strategy through first deployment
A forward-deployed model working in small, senior teams alongside FDE and FDX
A growing AI delivery practice where you help build the tooling and frameworks, not just use them
Remote-friendly culture
Internal training programs with full support for Claude, AWS, and other professional certifications, conference attendance
Career growth; we actively develop our engineers
Access to the latest AI tools and premium subscriptions
Long-term B2B collaboration
Private medical insurance or a budget for your medical needs
Paid sick leave, vacation, and public holidays
Equipment and all the tech you need for comfortable, productive work
How we hire:
Intro conversation. The role, your background and aspirations, tech questions.
Technical interview with live engineering sessions. Real problems, your own editor, you may use an LLM assistant
HR Interview. Soft skills and expectations
HM interview. Tech questions; a live engineering session is also possible
Location: Canada, Waterloo, ON
Why Sony Interactive Entertainment?
Sony Interactive Entertainment isn’t just the Best Place to Play — it’s also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As a subsidiary of Sony Group Corporation, we’re part of a proud legacy of innovation and excellence. SIE is a dynamic technology company, delivering cutting-edge hardware and network services to more than 100 million people and an entertainment leader, home to some of the most beloved and recognizable intellectual properties (IP) in the world. Our role at SIE is to create and nurture the experiences under the PlayStation brand, a name synonymous with entertainment excellence and creativity.
Sr. Software Development Engineer In Test, ML/AI
At SIE, we believe in making play safer for everyone. Within our Information Security team, we protect our people, platforms, and products with smart, scalable, and thoughtful solutions. Our mission is to provide clear, actionable risk and security insights that guide the business and enable amazing player experiences.
We’re building a team that thrives on collaboration, curiosity, and delivering real value. Come help us make a difference!
Overview
We are hiring a Sr Software Engineer in Test specializing in ML/AI quality, including automation for model evaluation, LLM-assisted test generation, and validation of AI-powered workflows. The role includes guiding quality strategy and developing automation frameworks. You will also manage implementation for complex, cross-functional, machine learning-powered products and services. This role involves more than test development. You will lead quality initiatives from start to finish, ensuring teams stay synchronized, dependencies are managed, and releases are delivered confidently. You will work closely with ML, engineering, product, and infrastructure teams. You will shape how quality is built, monitored, and expanded while leading embedded QE efforts within projects.
What you’ll do
Define and complete quality strategies, test plans, and automation coverage for ML-powered services and platform components.
Use LLMs and other AI-assisted techniques to generate, expand, and maintain high-value test cases for ML-powered workflows.
Design scenario-based test suites for AI features, including adversarial prompts, edge cases, ambiguous inputs, and underrepresented user scenarios.
Lead QE efforts for multi-functional projects, driving risk assessment, dependency management, and release readiness.
Design, develop, and maintain scalable automation frameworks for backend services, APIs, and ML inference systems using Python and/or Java.
Build automated validation for ML and LLM outputs, including ranking behavior, score distributions, prompt/response quality, hallucination indicators, and probabilistic model evaluation.
Debug test failures, service anomalies, model inconsistencies, and AI behavior regressions to identify root causes and drive resolution.
Perform functional, integration, regression, API, end-to-end, performance, and reliability testing for distributed systems.
Improve automation reliability, reduce flakiness, and optimize execution efficiency.
Partner with engineering and ML teams to integrate automated testing into CI/CD pipelines and release workflows.
Collaborate across teams to establish scalable quality standards, tooling, and guidelines.
Required qualifications
Bachelor’s degree in Computer Science or equivalent
5+ years of experience as an SDET or QE engineer focused on backend and distributed systems.
Experience using LLMs to generate, transform, and prioritize test cases for AI-powered experiences.
Experience with AI evaluation tooling, prompt evaluation frameworks, model monitoring, or human-in-the-loop review workflows.
Strong experience testing RESTful APIs, microservices, and distributed architectures.
Proficiency in Python, Java, JS or similar languages for automation development.
Hands-on experience with automation frameworks such as pytest, JUnit, Selenium, Playwright, Cypress, or Appium.
Experience with CI/CD systems and test pipelines (Jenkins, GitHub Actions, etc.).
Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker).
Familiarity with databases, monitoring, and observability tools.
Strong understanding of SDLC, Agile methodologies, and release processes.
Excellent problem-solving, debugging, and communication skills.
Preferred qualifications
Experience validating ML outputs using statistical analysis or scenario-based testing approaches.
Familiarity with ML infrastructure, data pipelines, or model-serving platforms (Seldon, KServe, Ray Serve, etc.).
Prior work in content moderation ML, security, fraud detection, or adversarial ML.
Experience testing high-scale, low-latency online services.
Experience with Databricks or similar ML platform tooling.
Familiarity with Node.js, React, or modern frontend technologies.
Experience testing mobile, console, or other non-PC platforms.
What sets you apart
Strong combination of automation engineering and delivery ownership.
Ability to drive quality across complex cross-functional initiatives.
Practical understanding of how to test non-deterministic AI systems and separate model variance from quality regressions.
Proven risk management and dependency coordination skills.
Ability to influence engineering teams and promote quality guidelines.
Passion for scalable, reliable, and maintainable automation systems.
At SIE, we consider several factors when setting each role’s base pay range, including the competitive benchmarking data for the market and geographic location.
Please note that the base pay range may vary in line with our hybrid working policy and individual base pay will be determined based on job-related factors which may include knowledge, skills, experience, and location.
In addition, this role
is eligible
for SIE’s top-tier benefits package that includes medical, dental, vision, matching 401(k), paid time off, wellness program and coveted employee discounts for Sony products.
This role also may be eligible for a bonus package.
Click
here
to learn more.
The estimated base pay range for this role is listed below.
$114,900
—
$172,300 CAD
Please note, Sony Interactive Entertainment conducts background checks at the offer stage for all new employees (which may include criminal background checks for some roles) and will need to process personal information to support these checks.
Please refer to our Candidate Privacy Notice (where applicable) for more information about what personal information we collect, how we use it, who we share it with, and your data protection rights.
Equal Opportunity Statement:
Sony is an Equal Opportunity Employer. All persons will receive consideration for employment without regard to gender (including gender identity, gender expression and gender reassignment), race (including colour, nationality, ethnic or national origin), religion or belief, marital or civil partnership status, disability, age, sexual orientation, pregnancy, maternity or parental status, trade union membership or membership in any other legally protected category.
We strive to create an inclusive environment, empower employees and embrace diversity. We encourage everyone to respond.
Sony Interactive Entertainment is a Fair Chance employer and qualified applicants with arrest and conviction records will be considered for employment.
Задачи:
— Разработка кросс-ассетных стратегий: поиск альфа-сигналов, построение и валидация математических и ML-моделей для прогнозирования движений рынка на различных горизонтах (от HFT до среднесрочных стратегий).
Обязанности:
• Исследовать бизнес-процессы, найти точки улучшений и показать, как AI-решения могут приносить ощутимую пользу.
• Подготовить полноценные требования к AI...
Требования:
• ...Kubernetes, MLflow), обработке и анализе данных (Python, SQL, DWH, Data Lake, Data Governance), понимании ML-жизненного цикла. Наши ожидания.
Ищу к себе в команду AI/ML Delivery Manager (FinTech). Нужно будет отвечать за скорость и результат поставки ML-решений в прод с последующей интеграцией в бизнес процессы конечных заказчиков.