Research Engineer

Responsibilities: — Build evaluation frameworks for AI agents, including datasets, sandbox environments and metrics for real-world tasks. — Design experiments to compare models, agent harnesses, tools and execution environments to identify regressions and trade-offs. — Turn qualitative feedback about agent behaviour into concrete research questions, experiments and measurable outcomes. — Investigate agent failures to determine root causes (model, harness, tooling, prompting) and improve performance. — Build and improve agent harnesses, internal tools and integrations, including contributing to Aria and other agents. — Develop shared knowledge and reusable skills across agents so systems can learn and improve over time. — Collaborate with Infrastructure and Security to ensure agents operate reliably and safely in production. Requirements: — Hands-on experience building, evaluating or improving LLM models or agents (professional, research, open-source or personal projects). — Strong Python skills and solid software engineering fundamentals; able to write production-quality, tested code. — Experience with applied research, machine learning or data science, including experiment design and dataset work. — Understanding of how LLM agents work beyond prompting: tool use, harnesses, context management and failure modes. — Ability to decompose open-ended problems into testable questions and work independently toward solutions. — Experience building tools or systems for other teams, with attention to evaluating effectiveness beyond benchmarks. — Comfortable collaborating across research and engineering and adapting to changing priorities. Nice to have: — Experience building LLM evaluation pipelines, benchmarks, RL environments or model testing infrastructure. — Contributions to open-source agent frameworks, harnesses, evaluation datasets or benchmarks. — Experience with agent frameworks such as Hermes, OpenClaw or similar tools. — Experience in applied ML research, model training, fine-tuning or reinforcement learning. — Advanced degree or postgraduate research experience in CS, mathematics, statistics, ML or related field. — Experience in financial markets, quantitative research, DeFi or risk modelling. — Familiarity with AI security considerations, including access controls, prompt injection and data protection. Benefits: — Remote first — work from anywhere in the United States — Regular in-person company retreats and cross-country "office visit" perk — 100% paid medical, dental, and vision premiums for employees — $1,000 WFH stipend — Monthly reimbursement for home internet, phone, and cellular data — Unlimited vacation — 100% paid parental leave of 12 weeks — Fertility benefits — Opportunity for incentive compensation — National pay range: $175,000 - $195,000 base plus additional On Target Earnings potential and equity Applicants must be able to work within the contiguous United States and Canada.