We’re looking for US-based, actively practicing physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program. Score the clinical reasoning quality of benchmark items: vignette accuracy and completeness, whether the vignette gives the answer away, answer correctness and gradeability, and trap quality;
Responsibilities:
— Evaluate clinical EHR vignettes paired with a question, a proposed answer, and a distractor “trap” answer across diagnosis and treatment tasks covering multiple body systems including cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary.
— Score the clinical reasoning quality of benchmark items, assessing vignette accuracy and completeness, answer correctness and gradeability, and trap quality.
— Verify if the reasoning chain reaches the answer from vignette facts alone, check correct reasoning traces, and write short rationales.
— Participate in three independent blind reads followed by physician adjudication.
Requirements:
— Must be a US-based, actively practicing physician (MD or DO) with an active, unrestricted US medical license verified by the issuing medical board.
— Board certification or completed residency training with recent direct patient care experience (attending-level preferred).
— Broad diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
— Strong clinical reasoning skills including differential diagnosis, next-step management, and application of evidence-based guidelines.
— Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA is a plus.
— Strong written English proficiency (C1+).
— Not suitable for medical students, unlicensed graduates, non-physician clinicians, or non-clinical healthcare roles.
Benefits:
— Project-based participation with estimated 10-20 hours per week during active phases.
— Compensation paid per accepted task, with rates up to the equivalent of $100 per hour depending on qualification tier and task completion efficiency.
— Faster task completion increases effective hourly rate.
Candidates are encouraged to submit their CV in English and indicate their level of English proficiency.
Work is remote and project-based, suitable for US-based physicians open to part-time, non-permanent projects.
Контакты работодателя доступны по кнопке «Откликнуться» после входа.
56/100
Средняя вакансия
Оценка от Job Hunters AI
Licensed Physician (MD/DO) - Freelance AI Trainer; проектная работа; ограничение по локации; понятные задачи; прозрачная оплата.
Из чего сложилась оценка. Нажмите, чтобы узнать подробнее
Проектная работа
Работа проектная, непостоянная, с оценкой времени 10-20 часов в неделю в активные периоды, без гарантии постоянной загрузки.
Из объявления
Participation is project-based. … Project time expectations … For this project, tasks are estimated to require around 10-20 hours per week during active phases, based on project requirements.