Вакансии

Location: St. Louis, United States; Saint Charles, United States Turn your passion for luxury into a career opportunity. Explore the world of premium brands and make a lasting impact in fashion, beauty, jewelry, or automobiles. Join CXG, the global leader in customer experience, and work alongside iconic brand names. About the Role: As a luxury brand evaluator, you will step into the world of luxury to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques, purchasing online, or returning a product, your voice shapes the future of luxury. • Flexibility - each in-store visit takes approximately 20 minutes, giving you the flexibility to complete multiple assignments. • Frequent opportunities - perform several missions per month and collaborate with some of the most prestigious brands worldwide. What you will be doing: 1. Choose your assignments - align your missions with your personal preferences and profile. 2. Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home. 3. Observe carefully - check the ambiance, service quality, and overall customer experience. 4. Provide honest feedback - use our platform to share your observations through questionnaires. Perks of the Role: • Collaborate with iconic brands across industries. • Flexible assignments tailored to your interests. • Compensation for your time and input, with the potential for reimbursement on purchases. • A user-friendly platform for managing missions and feedback. How to Join the CXG Community: 1. Register: sign up at live.cxg.com and confirm your email. 2. Complete your profile: fill out your details to 100% and get verified. 3. Get certified: pass the General Certification to access missions. 4. Apply for missions: explore assignments with brands that excite you. 5. Start evaluating: begin making a difference in the luxury market. Compensation: • Non-Purchase Evaluations: Earn a fee based on mission complexity. • Purchase-Based Evaluations: Reimbursement for approved items as stated in the mission brief. About CXG At CXG, we connect premium brands with over 80,000 evaluators across 70 countries. With 12 global offices and almost two decades of CX retail expertise, we’re committed to redefining the customer experience for luxury giants. Join us and enjoy the flexibility of working on the go with our mobile platform, CXG Live. On this platform, you can seamlessly manage missions, certifications, and feedback. Requirements Must be 18 years or older. Current customer of premium and luxury brands. Not currently under contract with any retail brands, to ensure impartiality. Punctual, organized, detail-oriented, and reliable. Observant and passionate about customer experience. No prior experience is required; we value honest feedback from genuine customers.

Location: Charleston, United States Turn your passion for luxury into a career opportunity. Explore the world of premium brands and make a lasting impact in fashion, beauty, jewelry, or automobiles. Join CXG, the global leader in customer experience, and work alongside iconic brand names. About the Role: As a luxury brand evaluator, you will step into the world of luxury to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques, purchasing online, or returning a product, your voice shapes the future of luxury. • Flexibility - each in-store visit takes approximately 20 minutes, giving you the flexibility to complete multiple assignments. • Frequent opportunities - perform several missions per month and collaborate with some of the most prestigious brands worldwide. What you will be doing: 1. Choose your assignments - align your missions with your personal preferences and profile. 2. Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home. 3. Observe carefully - check the ambiance, service quality, and overall customer experience. 4. Provide honest feedback - use our platform to share your observations through questionnaires. Perks of the Role: • Collaborate with iconic brands across industries. • Flexible assignments tailored to your interests. • Compensation for your time and input, with the potential for reimbursement on purchases. • A user-friendly platform for managing missions and feedback. How to Join the CXG Community: 1. Register: sign up at live.cxg.com and confirm your email. 2. Complete your profile: fill out your details to 100% and get verified. 3. Get certified: pass the General Certification to access missions. 4. Apply for missions: explore assignments with brands that excite you. 5. Start evaluating: begin making a difference in the luxury market. Compensation: • Non-Purchase Evaluations: Earn a fee based on mission complexity. • Purchase-Based Evaluations: Reimbursement for approved items as stated in the mission brief. About CXG At CXG, we connect premium brands with over 80,000 evaluators across 70 countries. With 12 global offices and almost two decades of CX retail expertise, we’re committed to redefining the customer experience for luxury giants. Join us and enjoy the flexibility of working on the go with our mobile platform, CXG Live. On this platform, you can seamlessly manage missions, certifications, and feedback. Requirements Must be 18 years or older. Current customer of premium and luxury brands. Not currently under contract with any retail brands, to ensure impartiality. Punctual, organized, detail-oriented, and reliable. Observant and passionate about customer experience. No prior experience is required; we value honest feedback from genuine customers.

Location: Netherlands; Germany; Sweden; Switzerland; New Zealand; Spain About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are seeking experienced Subject Matter Experts (SMEs)in psychology, social work, counseling, qualitative research, human-computer interaction (HCI), communication, conflict resolution, coaching, and related advisory fields to support AI conversation evaluation research. You will review conversations between people and AI models, assess whether sufficient context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality. The ideal candidate has at least 3–5 years of relevant professional experience, strong analytical skills, and the ability to distinguish information that could materially change a recommendation from details that are less relevant. Key Responsibilities Review AI conversations involving real-world interpersonal situations and everyday decision-making. Identify missing context and recommend relevant follow-up questions that could improve the quality of AI responses. Evaluate whether gathering additional information would materially change an AI-generated recommendation. Distinguish between load-bearing information that affects a decision and non-essential or merely descriptive details. Conduct qualitative reviews and document findings using structured evaluation guidelines. Assess whether conversations adequately consider relevant personal, psychological, social, financial, practical, and organizational factors. Identify missed opportunities for clarification, deeper understanding, and information gathering. Review conversations involving relationships, work, education, family, wellbeing, planning, and resource constraints. Collaborate with research teams to ensure consistent, evidence-based assessments. Maintain high standards of accuracy, clarity, and consistency while meeting project deadlines. Requirements Candidates must have at least 3–5 years of relevant direct professional experience in one or more of the following domains: 1. Licensed Social Work Master's degree in Social Work (MSW) or an equivalent graduate-level qualification. LCSW, LICSW, LMSW, or an equivalent applicable social work license or credential. Experience in client assessment, case review, interviewing, counseling, or related social-work practice. 2. Counseling, Therapy, or Psychology Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling or therapy license or credential; or PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology. Relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work. 3. Qualitative Research, HCI, or Communication MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline. Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, or qualitative coding. Ability to analyze conversations systematically and identify missing context, communication patterns, and opportunities for clarification. 4. Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields Relevant professional qualification, credential, or demonstrated subject-matter expertise. At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway. Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work. Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations.

Location: United States; Saudi Arabia; Canada; United Kingdom; Australia; Ireland About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are seeking experienced Subject Matter Experts (SMEs)in psychology, social work, counseling, qualitative research, human-computer interaction (HCI), communication, conflict resolution, coaching, and related advisory fields to support AI conversation evaluation research. You will review conversations between people and AI models, assess whether sufficient context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality. The ideal candidate has at least 3–5 years of relevant professional experience, strong analytical skills, and the ability to distinguish information that could materially change a recommendation from details that are less relevant. Key Responsibilities Review AI conversations involving real-world interpersonal situations and everyday decision-making. Identify missing context and recommend relevant follow-up questions that could improve the quality of AI responses. Evaluate whether gathering additional information would materially change an AI-generated recommendation. Distinguish between load-bearing information that affects a decision and non-essential or merely descriptive details. Conduct qualitative reviews and document findings using structured evaluation guidelines. Assess whether conversations adequately consider relevant personal, psychological, social, financial, practical, and organizational factors. Identify missed opportunities for clarification, deeper understanding, and information gathering. Review conversations involving relationships, work, education, family, wellbeing, planning, and resource constraints. Collaborate with research teams to ensure consistent, evidence-based assessments. Maintain high standards of accuracy, clarity, and consistency while meeting project deadlines. Requirements Candidates must have at least 3–5 years of relevant direct professional experience in one or more of the following domains: 1. Licensed Social Work Master's degree in Social Work (MSW) or an equivalent graduate-level qualification. LCSW, LICSW, LMSW, or an equivalent applicable social work license or credential. Experience in client assessment, case review, interviewing, counseling, or related social-work practice. 2. Counseling, Therapy, or Psychology Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling or therapy license or credential; or PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology. Relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work. 3. Qualitative Research, HCI, or Communication MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline. Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, or qualitative coding. Ability to analyze conversations systematically and identify missing context, communication patterns, and opportunities for clarification. 4. Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields Relevant professional qualification, credential, or demonstrated subject-matter expertise. At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway. Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work. Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations.

Location: Serbia; Croatia; Greece; Malaysia; Morocco; South Africa About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are seeking experienced Subject Matter Experts (SMEs) from psychology, social work, counseling, qualitative research, human-computer interaction (HCI), communication, conflict resolution, coaching, and related advisory fields to support AI conversation evaluation research. You will review conversations between people and AI models, assess whether the AI gathered sufficient context before making recommendations, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality. The ideal candidate has at least 3–5 years of relevant professional experience, strong analytical and interviewing skills, and the ability to distinguish essential, decision-changing information from details that are less relevant to a recommendation. Key Responsibilities Review AI conversations involving interpersonal relationships, workplace situations, education, family matters, everyday planning, wellbeing, and other real-world decisions. Evaluate whether the AI gathered sufficient context before providing advice, recommendations, or proposed solutions. Identify missing information and suggest relevant follow-up questions that could improve the quality of the AI's response. Assess whether additional context would materially change the recommendation or decision. Distinguish between load-bearing information that could change the outcome and non-essential or merely descriptive details. Conduct qualitative reviews using structured evaluation guidelines and document findings consistently. Identify missed opportunities for clarification, contextual understanding, and information gathering. Consider relevant personal, psychological, social, financial, practical, and organizational factors when evaluating conversations. Collaborate with research teams to maintain consistent evaluation standards and high-quality assessments. Complete assigned tasks accurately within the required project timelines. Requirements Candidates must have at least 3–5 years of direct professional experience in one or more of the following domains. 1. Licensed Social Work Master's degree in Social Work (MSW) or an equivalent graduate-level qualification. LCSW, LICSW, LMSW, or an equivalent applicable social work credential or license. Experience in client assessment, case review, interviewing, counseling, or related social-work practice. 2. Counseling, Therapy, or Psychology Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling or therapy credential; or PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology. Relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work. 3. Qualitative Research, HCI, or Communication MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline. Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, qualitative coding, or similar research activities. Ability to analyze conversations systematically and identify missing context or patterns in human communication. 4. Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields Relevant professional qualification, credential, or demonstrated subject-matter expertise. At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway. Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work. Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations.

Location: Brazil; Mexico; Argentina; Colombia; Poland; Romania About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are seeking experienced Subject Matter Experts (SMEs) from psychology, social work, counseling, qualitative research, human-computer interaction (HCI), communication, conflict resolution, coaching, and related advisory fields to support AI conversation evaluation research. You will review conversations between people and AI models, assess whether the AI gathered sufficient context before making recommendations, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality. The ideal candidate has at least 3–5 years of relevant professional experience, strong analytical and interviewing skills, and the ability to distinguish essential, decision-changing information from details that are less relevant to a recommendation. Key Responsibilities Review AI conversations involving interpersonal relationships, workplace situations, education, family matters, everyday planning, wellbeing, and other real-world decisions. Evaluate whether the AI gathered sufficient context before providing advice, recommendations, or proposed solutions. Identify missing information and suggest relevant follow-up questions that could improve the quality of the AI's response. Assess whether additional context would materially change the recommendation or decision. Distinguish between load-bearing information that could change the outcome and non-essential or merely descriptive details. Conduct qualitative reviews using structured evaluation guidelines and document findings consistently. Identify missed opportunities for clarification, contextual understanding, and information gathering. Consider relevant personal, psychological, social, financial, practical, and organizational factors when evaluating conversations. Collaborate with research teams to maintain consistent evaluation standards and high-quality assessments. Complete assigned tasks accurately within the required project timelines. Requirements Candidates must have at least 3–5 years of direct professional experience in one or more of the following domains. 1. Licensed Social Work Master's degree in Social Work (MSW) or an equivalent graduate-level qualification. LCSW, LICSW, LMSW, or an equivalent applicable social work credential or license. Experience in client assessment, case review, interviewing, counseling, or related social-work practice. 2. Counseling, Therapy, or Psychology Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling or therapy credential; or PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology. Relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work. 3. Qualitative Research, HCI, or Communication MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline. Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, qualitative coding, or similar research activities. Ability to analyze conversations systematically and identify missing context or patterns in human communication. 4. Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields Relevant professional qualification, credential, or demonstrated subject-matter expertise. At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway. Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work. Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations.

Location: San Marcos, United States Turn your passion for luxury into a career opportunity. Explore the world of premium brands and make a lasting impact in fashion, beauty, jewelry, or automobiles. Join CXG, the global leader in customer experience, and work alongside iconic brand names. About the Role: As a luxury brand evaluator, you will step into the world of luxury to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques, purchasing online, or returning a product, your voice shapes the future of luxury. • Flexibility - each in-store visit takes approximately 20 minutes, giving you the flexibility to complete multiple assignments. • Frequent opportunities - perform several missions per month and collaborate with some of the most prestigious brands worldwide. What you will be doing: 1. Choose your assignments - align your missions with your personal preferences and profile. 2. Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home. 3. Observe carefully - check the ambiance, service quality, and overall customer experience. 4. Provide honest feedback - use our platform to share your observations through questionnaires. Perks of the Role: • Collaborate with iconic brands across industries. • Flexible assignments tailored to your interests. • Compensation for your time and input, with the potential for reimbursement on purchases. • A user-friendly platform for managing missions and feedback. How to Join the CXG Community: 1. Register: sign up at live.cxg.com and confirm your email. 2. Complete your profile: fill out your details to 100% and get verified. 3. Get certified: pass the General Certification to access missions. 4. Apply for missions: explore assignments with brands that excite you. 5. Start evaluating: begin making a difference in the luxury market. Compensation: • Non-Purchase Evaluations: Earn a fee based on mission complexity. • Purchase-Based Evaluations: Reimbursement for approved items as stated in the mission brief. About CXG At CXG, we connect premium brands with over 80,000 evaluators across 70 countries. With 12 global offices and almost two decades of CX retail expertise, we’re committed to redefining the customer experience for luxury giants. Join us and enjoy the flexibility of working on the go with our mobile platform, CXG Live. On this platform, you can seamlessly manage missions, certifications, and feedback. Requirements Must be 18 years or older (21 years or older in the United States). Current customer of premium and luxury brands. Not currently under contract with any retail brands, to ensure impartiality. Punctual, organized, detail-oriented, and reliable. Observant and passionate about customer experience. No prior experience is required; we value honest feedback from genuine customers.

Agile Robots SE is looking for a DevOps Engineer with AI ML Evaluation Focus to operate cloud infrastructure, automate deployments, and evaluate AI models.

Location: United Kingdom; India; Germany; United States About Huzzle Labs Huzzle Labs partners with frontier AI labs to evaluate and improve frontier models through deep human expertise. Contributors directly assess advanced AI systems in their specialist fields, receive competitive compensation, and help define the standards used to measure model performance. About the Role Huzzle Labs is seeking experienced accountants to write realistic accounting tasks that test how well frontier AI systems handle accounting work. You will create prompts that are difficult enough for a frontier model to fail, then write a scoring rubric for each task so reviewers can assess the output consistently. A rubric is a list of objective, checkable criteria that a reviewer marks pass or fail. Your judgement helps define the standards used to measure model performance. This is an expert judgement role focused on realistic task design, careful review of model output and precise, consistent documentation. Training is provided. Prior AI training or data-annotation experience is required. Responsibilities Write realistic accounting tasks: Create requests of the kind a client, controller or business owner would send, covering month-end close, bank and credit card reconciliations, AR/AP ageing, accruals and adjusting entries, sales tax, job costing and the clean-up of incomplete or inconsistent books. Build from real material: Review a deal room containing hundreds of files, select the relevant documents and use them to develop prompts and rubrics for tasks a frontier model struggles to complete correctly. Require usable deliverables: Each request should call for a working output, usually a spreadsheet, a journal entry schedule or a short memo. Test against a frontier model: Run each request and review the output with the care you would apply when signing off accounting work. Where the model succeeds, increase the difficulty until a genuine gap appears. Write scoring rubrics: Define objective criteria that another reviewer can apply without repeating your work, for example, “the reconciled ending balance equals $X” and “the $X deposit is recorded as a customer deposit liability, not revenue.” Apply guidelines consistently: Follow the task and rubric guidelines, flag ambiguous cases and help refine the criteria. Requirements At least 5 years in accounting, bookkeeping or finance operations. Prior AI training or data-annotation experience is required. Hands-on knowledge of month-end close, reconciliations, AR/AP, accruals and adjusting entries, sales tax and job costing. Excellent written English and precision with numbers. Careful and detail-oriented: accurate figures and objective, checkable criteria. Available 20 to 40 hours per week for an initial one-week trial, with potential extension. Example Assignment For a client deal room covering a small business with incomplete and inconsistent books, you would: Review the files and select the relevant documents, such as bank statements, ledgers, invoices and ageing reports. Write a prompt as the business owner or controller would, asking for a reconciliation and the adjusting entries needed to close the month. Run the prompt against a frontier AI model and check the spreadsheet or journal entry schedule line by line. If the model completes the task correctly, increase the difficulty, for example, by adding a customer deposit, a timing difference or a sales tax complication, until it fails. Write the rubric, such as “the reconciled ending balance equals $X” and “the $X deposit is recorded as a customer deposit liability, not revenue,” so another reviewer can grade any answer without repeating the work. Confidentiality An NDA must be signed before the engagement begins. Benefits Compensation: $60 - $90 USD / hour

Location: Shah Alam, Malaysia; Klang, Malaysia; Puchong, Malaysia Turn your passion for luxury into a career opportunity! Explore the world of premium brands and make a lasting impact in fashion, beauty, jewelry, or automobiles. About the Role: As a luxury brand evaluator, you will step into the world of luxury to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques, purchasing online, or returning a product, your voice shapes the future of luxury. Flexibility - each in-store visit takes approximately 20 minutes, giving you the flexibility to complete multiple assignments. Frequent opportunities - perform several missions per month and collaborate with some of the most prestigious brands worldwide. What you will be doing: Choose your assignments - align your missions with your personal preferences and profile. Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home. Observe carefully - check the ambiance, service quality, and overall customer experience. Provide honest feedback - use our platform to share your observations through questionnaires. About CXG At CXG, we connect premium brands with over 80,000 evaluators across 70 countries. With 12 global offices and almost two decades of CX retail expertise, we’re committed to redefining the customer experience for luxury giants. Join us and enjoy the flexibility of working on the go with our mobile platform, CXG Live . On this platform, you can seamlessly manage missions, certifications, and feedback. Requirements Current customer of premium and luxury brands. Not currently under contract with any retail brands, to ensure impartiality. Punctual, organized, detail-oriented, and reliable. Observant and passionate about customer experience. No prior experience is required; we value honest feedback from genuine customers. Benefits This is a freelance, project-based position Flexible working hours Compensation for Non-Purchase Evaluations: Earn a fee based on mission complexity. Compensation for Purchase-Based Evaluations: Reimbursement for approved items as stated in the mission brief.

Location: Paris ABOUT MISTRAL Mistral provides full-stack AI solutions: from frontier models to developer tools, applications, and compute. We partner with enterprises tackling the hardest problems across high-stakes industries like finance, manufacturing, defense, healthcare, and the public sector, co-creating customized AI systems that they can run on their terms. We are a dynamic, collaborative team passionate about AI and its potential to transform society. Our diverse workforce thrives in competitive environments and is committed to driving innovation. Our teams are distributed between Europe, North America, Asia and the Middle East. We are creative, low-ego and team-spirited. THE ROLE This is a chance to build the frontier LLM evals that shape the next generations of flagship models. Could a model design a computer chip, or find the backdoor an attacker left in a network? Public benchmarks can't tell us, as they're generally not a good measure of real work. As an Evaluation Engineer, you'll own evals from start to finish. You'll run workshops with customers, industry experts and our customer-facing teams to understand their most important industry use cases (e.g., an engineer updating a part in CAD software), and help decide which are worth building first. You'll also look beyond today's use cases and design moonshot evals for work no model can do yet, so Mistral gets there first. Then you'll turn each one into a benchmark, often one where an agent has to do the work inside real software. You'll source the data, decide how to score it, and run it on every new Mistral model and on competitor models. When our models fall short, you'll work with the post-training teams to close the gap. When they catch up, you'll make the eval harder. You'll write up and present your results so senior leaders can act on them. They also give our go-to-market teams evidence of where Mistral leads. You'll learn new fields quickly. One month you might be learning how chip designers check their work, and the next how a bank investigates a flagged payment. The team is new and fast-paced, and you'll have a lot of autonomy. You'll spend a lot of time speaking with people across Mistral to work out what matters most, then decide what to build. WHAT YOU WILL DO - Run discovery with customers, industry experts and teams across Mistral, then help decide which use cases are worth building first. - Design new benchmarks for industry use cases and frontier capabilities, including how they're scored. - Build evals where agents work inside real software (e.g., CAD and simulation tools), and run them on our models with checks that the results stay reliable over time. - Stress-test existing evals to find flaws and bugs before we rely on them. - Source data with expert vendors. Where customer data can't leave the customer, build realistic proxy or synthetic data instead. - Write up and present what you find so leaders across Mistral can act on it. - Help other teams build their own evals, with tools and guides they can reuse. WHAT WE'RE LOOKING FOR EXPERIENCE - 2+ years of industry experience (e.g., in software, AI, or consulting), or a PhD. - A software engineering or research background, with experience writing code that others can build on. - Experience working directly with customers, industry experts, or other teams (e.g., through consulting, solutions engineering, or research collaborations). - Experience building something with LLMs. - Comfort working with large datasets, including building and cleaning them. TECHNICAL - Strong Python skills. - Comfortable working in a large, shared codebase and getting your code through reviews by other teams. - A strong understanding of LLM training, inference, and evaluation. COMMUNICATION - You enjoy speaking with people and are good at it. - You write clearly. A short doc or deck from you is enough for a busy leader to make a decision. - Confident presenting to both technical and non-technical audiences. - Fluent in English. HOW YOU WORK - High energy, and you don't need a roadmap. You move independently and handle uncertainty with confidence. - Proactive and high-agency. You spot what needs doing and get it done. - You make good decisions about what to build first and can explain why. - Low-ego, collaborative, and eager to learn. - Curious, with a track record of learning things outside your work or studies. You can get to grips with a new field in a few weeks. NICE TO HAVE - Experience building AI evaluations, especially for industry applications. - Published research on evals or contributions to open-source evaluation frameworks. - Experience building datasets, generating synthetic data, developing labelling guidelines, or working with annotation vendors. - A background in an industry outside software (e.g., engineering, finance, or science). WHAT WE OFFER We offer a comprehensive benefits package designed to support your well-being, growth, and work-life balance. Benefits vary by country and may include healthcare coverage, parental leave, retirement plans, relocation support, wellness programs, meal and transportation allowances, and other location-specific perks. For the most up-to-date details on benefits available in your location, please refer to our Benefits page https://app.notion.com/p/mistralai/Benefits-at-Mistral-36e6ba59a7fe836b93dd01737fcc27ef?source=copy_link. PRIVACY POLICY Your privacy matters to us. You can learn more about how we handle your personal data in our Applicant Privacy Policy https://legal.mistral.ai/terms/applicant-privacy-policy.

Location: United States About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are looking for a detail-oriented Business Analyst – AI Evaluation Specialist to support a project focused on assessing and comparing chatbot responses. You will evaluate multiple AI-generated outputs against defined business requirements and quality criteria, analyze business processes, and deliver structured assessments that help improve the accuracy and usefulness of AI systems. The ideal candidate combines strong business analysis, data interpretation, requirements documentation, and critical evaluation skills with the ability to communicate clear, evidence-based conclusions. Key Responsibilities Gather, analyze, and document business requirements, stakeholder expectations, and project objectives. Compare multiple chatbot responses and assess their accuracy, relevance, completeness, and alignment with defined evaluation criteria. Conduct structured evaluations and provide clear, evidence-based final assessments of AI-generated outputs. Translate business requirements into functional specifications, evaluation criteria, and actionable documentation. Analyze workflows, business processes, and data to identify gaps, inefficiencies, and improvement opportunities. Create reports, dashboards, process maps, and other project documentation. Support project planning, testing, implementation, and user acceptance testing (UAT). Collaborate with technical teams and business stakeholders to ensure project requirements are understood and met. Track project progress, risks, dependencies, and key performance indicators (KPIs). Present findings, insights, and recommendations to support informed decision-making. Maintain consistent evaluation quality and meet project deadlines and task completion targets. Requirements Bachelor's degree in Business Administration, Finance, Computer Science, or a related field. At least 3 years of professional experience as a Business Analyst or in a similar analytical role. Strong analytical thinking, problem-solving, and decision-making skills. Proficiency in Excel and experience with SQL, Power BI, Tableau, or comparable analytical and reporting tools. Experience gathering requirements, documenting functional specifications, and mapping business processes. Strong attention to detail and the ability to compare complex information against defined criteria. Excellent written communication, reporting, and documentation skills. Ability to collaborate with cross-functional teams and manage multiple priorities independently. Willingness to work five days per week, including weekends when required, with two weekdays off during the project.

Location: United States About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are seeking experienced U.S.-licensed attorneys with tax law expertise to support the training and evaluation of next-generation AI systems. You will analyze tax law scenarios, review AI-generated legal analyses and documents, identify errors or gaps in legal reasoning, and provide structured feedback to improve accuracy and completeness. This role is ideal for attorneys with strong legal judgment, excellent written communication, and experience interpreting complex legal issues. Prior AI experience is not required. SENIORITY: Experienced Legal Professional LANGUAGE: Excellent written English LOCATION: Remote — United States only Key Responsibilities Analyze tax law scenarios, identify key legal issues, assess risks, and apply sound legal judgment. Review AI-generated legal analyses, arguments, and documents for legal accuracy, completeness, and consistency. Identify legal errors, missing information, unsupported conclusions, and weaknesses in legal reasoning. Evaluate the application of relevant federal and state tax laws to complex factual scenarios. Provide clear, structured feedback and corrections to improve AI-generated legal outputs. Apply consistent review standards while maintaining strong attention to detail and professional judgment. Requirements Juris Doctor (J.D.) degree from a U.S. law school. Active license to practice law in at least one U.S. state or the District of Columbia. More than 3 years of substantive legal practice, preferably with experience in tax law. Strong legal reasoning, issue-spotting, and legal document review skills. Excellent written English communication and the ability to explain complex legal concepts clearly. Strong analytical skills, attention to detail, and the ability to work independently.

Location: United States About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are seeking an experienced Personal Injury Law Specialist to contribute to a project focused on improving the legal reasoning, accuracy, and reliability of advanced AI systems. In this contractor role, you will analyze legal scenarios, assess liability, evaluate AI-generated legal analyses and documents, and identify errors or gaps in legal reasoning. You will provide clear, structured feedback to help improve AI-generated legal outputs. The ideal candidate has a strong foundation in U.S. law, excellent analytical skills, and experience in personal injury law or civil litigation. SENIORITY: Experienced Legal Professional LANGUAGE: Excellent written English LOCATION: Remote — United States only Key Responsibilities Analyze legal scenarios to identify key legal issues, assess liability, and apply sound legal judgment. Review AI-generated legal analyses, arguments, and documents for accuracy, completeness, and consistency. Identify legal errors, missing facts, unsupported conclusions, and weaknesses in legal reasoning. Provide clear, structured feedback and corrections to improve the accuracy and quality of AI-generated outputs. Evaluate legal arguments critically, applying relevant principles of U.S. law. Maintain high standards of accuracy, clarity, and attention to detail throughout the review process. Requirements J.D. degree from a U.S. law school. Active bar license in at least one U.S. state. More than 3 years of practicing law, preferably in personal injury law or civil litigation. Strong legal reasoning, issue-spotting, and legal document review skills. Excellent written English communication skills. Exceptional attention to detail and the ability to identify subtle legal errors and gaps in reasoning. Ability to provide clear, objective, and well-structured written feedback.

Location: United States About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are seeking an experienced Employment Law Specialist to contribute to a project focused on improving the legal reasoning, accuracy, and reliability of advanced AI systems. In this contractor role, you will analyze employment law scenarios, evaluate AI-generated legal analyses and documents, identify errors or gaps in legal reasoning, and provide structured expert feedback to improve AI-generated outputs. Candidates should have a strong foundation in U.S. law, excellent legal judgment, and meticulous document review skills. Experience in employment or labor law is preferred. SENIORITY: Experienced Legal Professional LANGUAGE: Excellent written English LOCATION: Remote — United States only Key Responsibilities Analyze employment law scenarios, identify relevant legal issues, assess potential risks, and apply sound legal judgment. Review AI-generated legal analyses, arguments, and documents for legal accuracy, completeness, and consistency. Identify legal errors, missing information, unsupported conclusions, and weaknesses in legal reasoning. Provide clear, structured feedback and corrections to improve the quality and reliability of AI-generated legal outputs. Apply strong issue-spotting and critical analysis skills when evaluating complex legal scenarios. Maintain high standards of accuracy, clarity, and attention to detail throughout the review process. Requirements J.D. degree from an accredited U.S. law school. Active bar license in at least one U.S. state or the District of Columbia. More than 3 years of substantive legal practice, preferably in employment law or labor law. Strong legal reasoning, issue-spotting, and legal document review skills. Excellent written English communication skills. High attention to detail and the ability to identify nuanced legal errors and gaps in reasoning. Ability to provide clear, objective, and well-structured written feedback.

Location: Bangladesh; Brazil; Colombia; Egypt; Ghana; India About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are seeking an experienced Chemical Engineering Expert to lead a team of technical trainers developing realistic, terminal-based engineering tasks for advanced AI training and evaluation. You will oversee a pod of approximately 5–10 trainers, review tasks for technical accuracy, reproducibility, and grading reliability, and ensure that each task reflects authentic chemical engineering workflows. This role combines advanced chemical engineering knowledge, computational modeling, technical quality assurance, and team leadership to improve how AI systems solve complex engineering problems. CONTRACT: Contractor assignment, 4 weeks COMMITMENT: Hourly, up to 40h/week LOCATIONS: Remote - Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Pakistan, Indonesia, Kenya, Nigeria, Turkey, Vietnam PROCESS: AI Interview NOTES: PhD Degree is mandatory! Key Responsibilities Lead, mentor, and coordinate a team of 5–10 technical trainers, ensuring consistent quality, productivity, and timely delivery. Review engineering tasks end to end, including problem statements, reaction kinetics, thermodynamic and property data, process flowsheets, computational environments, reference solutions, and automated tests. Validate tasks involving reaction engineering, thermodynamics, phase equilibria, transport phenomena, separation processes, process simulation, process control, optimization, combustion, and electrochemical engineering. Ensure tasks reflect realistic chemical engineering workflows and require meaningful, multi-step technical reasoning. Verify mass and energy balances, thermodynamic consistency, property data, unit handling, numerical accuracy, and solver convergence. Validate computational models, stiff ODE/DAE solvers, tolerances, initial conditions, dependencies, and simulation results for reproducibility. Review automated graders to ensure they assess meaningful engineering outputs, including conversion, yield, selectivity, purity, energy consumption, and process constraint satisfaction. Identify technical inaccuracies, unrealistic assumptions, incorrect tolerances, and opportunities to bypass genuine engineering calculations. Provide actionable technical feedback, track revisions, and ensure issues are resolved before task approval. Onboard trainers, allocate assignments, monitor quality and throughput, resolve technical blockers, and maintain documentation and best practices. Requirements PhD, postdoctoral experience, or equivalent advanced technical expertise in Chemical Engineering. Strong programming skills in Python, Julia, C/C++, or MATLAB/Octave. Proficiency in Linux and command-line environments. Hands-on expertise in at least one major area, such as process modeling, reaction kinetics, thermodynamic modeling, process simulation, or optimization. Strong understanding of chemical engineering fundamentals, numerical methods, computational modeling, and technical validation. Experience reviewing complex technical work, chemical process simulations, computational models, or research outputs. Ability to evaluate mass and energy balances, thermodynamic assumptions, numerical accuracy, solver convergence, and computational reliability. Strong analytical judgment and the ability to identify technical errors, edge cases, and flawed engineering assumptions. Experience mentoring, reviewing, or leading small technical teams. Excellent written communication skills for precise technical feedback and documentation.

Location: Pakistan; Indonesia; Kenya; Nigeria; Turkey; Vietnam About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role We are seeking an experienced Chemical Engineering Expert to lead a team of technical trainers developing realistic, terminal-based engineering tasks for advanced AI training and evaluation. You will oversee a pod of approximately 5–10 trainers, review tasks for technical accuracy, reproducibility, and grading reliability, and ensure that each task reflects authentic chemical engineering workflows. This role combines advanced chemical engineering knowledge, computational modeling, technical quality assurance, and team leadership to improve how AI systems solve complex engineering problems. CONTRACT: Contractor assignment, 4 weeks COMMITMENT: Hourly, up to 40h/week LOCATIONS: Remote - Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Pakistan, Indonesia, Kenya, Nigeria, Turkey, Vietnam PROCESS: AI Interview NOTES: PhD Degree is mandatory! Key Responsibilities Lead, mentor, and coordinate a team of 5–10 technical trainers, ensuring consistent quality, productivity, and timely delivery. Review engineering tasks end to end, including problem statements, reaction kinetics, thermodynamic and property data, process flowsheets, computational environments, reference solutions, and automated tests. Validate tasks involving reaction engineering, thermodynamics, phase equilibria, transport phenomena, separation processes, process simulation, process control, optimization, combustion, and electrochemical engineering. Ensure tasks reflect realistic chemical engineering workflows and require meaningful, multi-step technical reasoning. Verify mass and energy balances, thermodynamic consistency, property data, unit handling, numerical accuracy, and solver convergence. Validate computational models, stiff ODE/DAE solvers, tolerances, initial conditions, dependencies, and simulation results for reproducibility. Review automated graders to ensure they assess meaningful engineering outputs, including conversion, yield, selectivity, purity, energy consumption, and process constraint satisfaction. Identify technical inaccuracies, unrealistic assumptions, incorrect tolerances, and opportunities to bypass genuine engineering calculations. Provide actionable technical feedback, track revisions, and ensure issues are resolved before task approval. Onboard trainers, allocate assignments, monitor quality and throughput, resolve technical blockers, and maintain documentation and best practices. Requirements PhD, postdoctoral experience, or equivalent advanced technical expertise in Chemical Engineering. Strong programming skills in Python, Julia, C/C++, or MATLAB/Octave. Proficiency in Linux and command-line environments. Hands-on expertise in at least one major area, such as process modeling, reaction kinetics, thermodynamic modeling, process simulation, or optimization. Strong understanding of chemical engineering fundamentals, numerical methods, computational modeling, and technical validation. Experience reviewing complex technical work, chemical process simulations, computational models, or research outputs. Ability to evaluate mass and energy balances, thermodynamic assumptions, numerical accuracy, solver convergence, and computational reliability. Strong analytical judgment and the ability to identify technical errors, edge cases, and flawed engineering assumptions. Experience mentoring, reviewing, or leading small technical teams. Excellent written communication skills for precise technical feedback and documentation.

Alpaca is looking for a Senior Data Scientist AI Evaluation to design model evaluations, establish quality standards, and improve AI systems.