Physician (MD/DO) - Medical AI Evaluation Project

Responsibilities: — Evaluate clinical EHR vignettes paired with a question, a proposed answer, and a distractor “trap” answer across diagnosis and treatment tasks covering multiple body systems including cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary. — Score the clinical reasoning quality of benchmark items, assessing vignette accuracy and completeness, answer correctness and gradeability, and trap quality. — Verify if the reasoning chain reaches the answer from vignette facts alone, check correct reasoning traces, and write short rationales. — Participate in three independent blind reads followed by physician adjudication. Requirements: — Actively practicing physician (MD or DO) with an active, unrestricted US medical license verified at onboarding and re-checked monthly. — Board certification or completed residency training with recent direct patient care experience (attending-level preferred). — Broad diagnostic and treatment experience across multiple systems (internal, family, emergency, or hospital medicine). — Strong clinical reasoning skills including differential diagnosis, next-step management, and application of evidence-based guidelines. — Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA is a plus. — Strong written English skills (C1+). — Not suitable for medical students, unlicensed graduates, non-physician clinicians, or non-clinical healthcare roles. Benefits: — Paid per approved task at a flat rate equivalent to about $100 per hour based on estimated task time. — $50 one-time onboarding reward paid after first approved production task. — Flexible, project-based work allowing contributors to choose when and how much to contribute, estimated at 10–20 hours per week during active phases. — Additional earnings incentives for high-quality approved tasks. — No patient contact or malpractice exposure; work involves reviewing written clinical cases remotely. Work format is fully remote. This is a project-based opportunity with flexible time commitment to fit around clinical practice.