AI Product Analyst - Voice AI Agents

MEDvidi, a US-based mental healthcare platform, is seeking an AI Product Analyst to focus on Voice AI Agents that engage with patients and automate key parts of their care journey. The role involves analyzing AI behavior in production, identifying failure patterns such as hallucinations and inconsistencies, and assessing AI quality and reliability through examination of call transcripts, conversation logs, and operational data. Responsibilities include developing monitoring frameworks and dashboards to track AI performance, detecting anomalies and shifts in quality, and measuring the impact of AI on patient experience and business outcomes. Collaboration with Product, Engineering, QA, and Operations teams is essential. Candidates should have over three years of experience in Product, Data, or Operational Analytics, proficiency in SQL and Python for handling large text datasets, familiarity with ML pipelines or AI products, and an understanding of LLM limitations and AI failure modes. Experience in building monitoring systems and strong communication skills to translate insights into product decisions are required. Fluency in Russian and English (B2+) and availability during CET-overlapping hours are necessary. Preferred qualifications include experience with conversational AI, evaluating LLM outputs, and knowledge of AI quality metrics. The position offers remote work, involvement with impactful AI healthcare products, and benefits such as health insurance and paid leave.