Location: United States
About Turing
Turing’s mission is to accelerate superintelligence to drive real economic progress. Headquartered in San Francisco, Turing works with frontier AI labs to generate high-quality datasets, reinforcement learning environments, and frontier research benchmarks that improve model capabilities in software engineering, enterprise knowledge work, and advanced STEM reasoning. In software engineering, Turing is the largest and longest-running data provider in the category. Turing also works with Fortune 500 enterprises across financial services, life sciences, healthcare, retail, automotive, and CPG to build and deploy end-to-end agentic AI systems inside mission-critical workflows. By operating on both sides, Turing closes the loop between frontier research and enterprise deployment, turning real-world deployment signals into better data, evaluations, and more capable models. Learn more at
www.turing.com
.
Senior Software Engineer, AI Evaluation Infrastructure
We build the infrastructure that frontier AI labs use to evaluate coding agents and language models: benchmark datasets, agent evaluation runs at scale, and automated task quality checks. You would own features across the stack, from the data model and background jobs to the screens our users work in every day.
This is a role for an experienced engineer who works independently, makes sound tradeoffs, and turns a loosely defined request into something shipped, tested, and safe to leave alone.
What you will do
Design and ship features end to end: data model, API, background workers, and the UI when the change needs it.
Build and improve how agents and models are evaluated: running trials in sandboxed environments, scoring results, and turning thousands of runs into results people can trust.
Improve how benchmark tasks are quality-checked: verifiers, rubric-based and LLM-as-a-judge grading, and deciding whether a task is clear, solvable, and fair to score.
Integrate new models, agents, and benchmarks in a way that generalizes, without a special case for each one.
Keep the system fast and reliable as usage grows: efficient queries, safe migrations, and concurrency that holds up under load.
Review teammates’ pull requests against our engineering guidelines and help keep the codebase consistent.
What you bring
5+ years building and operating production backend systems, with ownership of at least one service from design through on-call.
Bachelor’s degree or higher in Software Engineering, Computer Science, or a related field.
Strong Python and experience with Django (or a similar framework), PostgreSQL, and a task queue such as Celery. What matters most is having run a real system, not knowing a particular framework.
Hands-on experience in LLM evaluation, beyond calling a model API. You understand how a benchmark task is built, how an agent trial is scored, and how results are aggregated and compared. Experience with Harbor, Terminal-Bench, SWE-bench, Inspect, or a similar harness is a strong plus.
Judgment about task quality. You can spot ambiguous instructions, weak or gameable verifiers, leaked answers, and checks that pass for the wrong reason.
Good tradeoffs. You reach for the smallest design that solves the problem, extend what already exists before adding something new, and can explain what you chose not to build.
Ownership. You take a fuzzy request, find the relevant code, ask questions only when the answer changes the design, and finish the job, including migrations, tests, and edge cases users will hit.
Quick adoption of team conventions. You read the written guidelines and your first pull requests already match them in naming, structure, tests, and error handling.
Comfort with a TypeScript and React frontend when a change is visible there. You don’t need to be a frontend specialist.
Clear writing. Your pull request descriptions and design notes explain why, not just what.
Nice to have
Experience running agents in sandboxed or containerized environments (Docker, Kubernetes, or cloud sandboxes such as Modal or Daytona).
Experience with LLM-as-a-judge or rubric-based grading, and with reducing judge bias and noise.
Familiarity with coding agents such as Claude Code, Codex, or OpenHands, and with MCP.
Experience with distributed job scheduling, concurrency limits, or database locking at scale.
Contributions to open-source evaluation or infrastructure projects.
How we work
Written engineering guidelines, applied consistently, with the same review bar for everyone.
Business logic lives in services. Hot paths stay cheap. Lint and tests pass before a pull request is ready.
The same vocabulary in the code, the API, and the UI.
We prefer deleting code to adding it, and simple designs to clever ones.
A strong first month looks like several changes merged with light review, and one area of the system you can change confidently without a guide.
Compensation:
$170,000 - $200,000 Base Salary + Equity
#LI-VC1
Values
We are client first:
We put our clients at the center of everything we do, because their success is the ultimate measure of our value.
We work at Start-Up Speed:
We move fast, stay agile and favor action because momentum is the foundation of perfection
We are AI forward:
We help our clients build the future of Al and implement it in our own roles and workflow to amplify productivity.
Advantages of joining Turing
Work at the frontier of AI
, helping the world’s leading AI labs improve their most advanced models by building expert datasets, RL environments, and first-of-a-kind benchmarks.
Contribute to leading-edge AI research
and showcase your work at top conferences such as ICLR, ICML, and NeurIPS.
Bring frontier AI innovation to the enterprise
, applying lessons learned from leading AI labs to solve real-world business challenges.
Collaborate with and learn from exceptional colleagues
with deep AI experience from Google, Meta, Amazon, and other leading technology companies.
Move at the pace of AI innovation
, with the speed, ownership, and impact of a startup.
Turing is proud to be an equal opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, gender identity, sexual orientation, age, marital status, disability, protected veteran status, or any other legally protected characteristics. At Turing we are dedicated to building a diverse, inclusive and authentic workplace and celebrate authenticity, so if you’re excited about this role but your past experience doesn’t align perfectly with every qualification in the job description, we encourage you to apply anyways. You may be just the right candidate for this or other roles.
For applicants from the European Union, please review
Turing's GDPR notice here
.
seniorUnited States; For applicants from the European Union, please review
Location: Paris
ABOUT MISTRAL
Mistral provides full-stack AI solutions: from frontier models to developer tools, applications, and compute. We partner with enterprises tackling the hardest problems across high-stakes industries like finance, manufacturing, defense, healthcare, and the public sector, co-creating customized AI systems that they can run on their terms.
We are a dynamic, collaborative team passionate about AI and its potential to transform society. Our diverse workforce thrives in competitive environments and is committed to driving innovation. Our teams are distributed between Europe, North America, Asia and the Middle East. We are creative, low-ego and team-spirited.
THE ROLE
This is a chance to build the frontier LLM evals that shape the next generations of flagship models.
Could a model design a computer chip, or find the backdoor an attacker left in a network? Public benchmarks can't tell us, as they're generally not a good measure of real work.
As an Evaluation Engineer, you'll own evals from start to finish. You'll run workshops with customers, industry experts and our customer-facing teams to understand their most important industry use cases (e.g., an engineer updating a part in CAD software), and help decide which are worth building first. You'll also look beyond today's use cases and design moonshot evals for work no model can do yet, so Mistral gets there first. Then you'll turn each one into a benchmark, often one where an agent has to do the work inside real software. You'll source the data, decide how to score it, and run it on every new Mistral model and on competitor models.
When our models fall short, you'll work with the post-training teams to close the gap. When they catch up, you'll make the eval harder. You'll write up and present your results so senior leaders can act on them. They also give our go-to-market teams evidence of where Mistral leads.
You'll learn new fields quickly. One month you might be learning how chip designers check their work, and the next how a bank investigates a flagged payment.
The team is new and fast-paced, and you'll have a lot of autonomy. You'll spend a lot of time speaking with people across Mistral to work out what matters most, then decide what to build.
WHAT YOU WILL DO
- Run discovery with customers, industry experts and teams across Mistral, then help decide which use cases are worth building first.
- Design new benchmarks for industry use cases and frontier capabilities, including how they're scored.
- Build evals where agents work inside real software (e.g., CAD and simulation tools), and run them on our models with checks that the results stay reliable over time.
- Stress-test existing evals to find flaws and bugs before we rely on them.
- Source data with expert vendors. Where customer data can't leave the customer, build realistic proxy or synthetic data instead.
- Write up and present what you find so leaders across Mistral can act on it.
- Help other teams build their own evals, with tools and guides they can reuse.
WHAT WE'RE LOOKING FOR
EXPERIENCE
- 2+ years of industry experience (e.g., in software, AI, or consulting), or a PhD.
- A software engineering or research background, with experience writing code that others can build on.
- Experience working directly with customers, industry experts, or other teams (e.g., through consulting, solutions engineering, or research collaborations).
- Experience building something with LLMs.
- Comfort working with large datasets, including building and cleaning them.
TECHNICAL
- Strong Python skills.
- Comfortable working in a large, shared codebase and getting your code through reviews by other teams.
- A strong understanding of LLM training, inference, and evaluation.
COMMUNICATION
- You enjoy speaking with people and are good at it.
- You write clearly. A short doc or deck from you is enough for a busy leader to make a decision.
- Confident presenting to both technical and non-technical audiences.
- Fluent in English.
HOW YOU WORK
- High energy, and you don't need a roadmap. You move independently and handle uncertainty with confidence.
- Proactive and high-agency. You spot what needs doing and get it done.
- You make good decisions about what to build first and can explain why.
- Low-ego, collaborative, and eager to learn.
- Curious, with a track record of learning things outside your work or studies. You can get to grips with a new field in a few weeks.
NICE TO HAVE
- Experience building AI evaluations, especially for industry applications.
- Published research on evals or contributions to open-source evaluation frameworks.
- Experience building datasets, generating synthetic data, developing labelling guidelines, or working with annotation vendors.
- A background in an industry outside software (e.g., engineering, finance, or science).
WHAT WE OFFER
We offer a comprehensive benefits package designed to support your well-being, growth, and work-life balance. Benefits vary by country and may include healthcare coverage, parental leave, retirement plans, relocation support, wellness programs, meal and transportation allowances, and other location-specific perks.
For the most up-to-date details on benefits available in your location, please refer to our Benefits page https://app.notion.com/p/mistralai/Benefits-at-Mistral-36e6ba59a7fe836b93dd01737fcc27ef?source=copy_link.
PRIVACY POLICY
Your privacy matters to us. You can learn more about how we handle your personal data in our Applicant Privacy Policy https://legal.mistral.ai/terms/applicant-privacy-policy.
Location: United States
About Gramian
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
About the Role
We are looking for a detail-oriented Business Analyst – AI Evaluation Specialist to support a project focused on assessing and comparing chatbot responses. You will evaluate multiple AI-generated outputs against defined business requirements and quality criteria, analyze business processes, and deliver structured assessments that help improve the accuracy and usefulness of AI systems. The ideal candidate combines strong business analysis, data interpretation, requirements documentation, and critical evaluation skills with the ability to communicate clear, evidence-based conclusions.
Key Responsibilities
Gather, analyze, and document business requirements, stakeholder expectations, and project objectives.
Compare multiple chatbot responses and assess their accuracy, relevance, completeness, and alignment with defined evaluation criteria.
Conduct structured evaluations and provide clear, evidence-based final assessments of AI-generated outputs.
Translate business requirements into functional specifications, evaluation criteria, and actionable documentation.
Analyze workflows, business processes, and data to identify gaps, inefficiencies, and improvement opportunities.
Create reports, dashboards, process maps, and other project documentation.
Support project planning, testing, implementation, and user acceptance testing (UAT).
Collaborate with technical teams and business stakeholders to ensure project requirements are understood and met.
Track project progress, risks, dependencies, and key performance indicators (KPIs).
Present findings, insights, and recommendations to support informed decision-making.
Maintain consistent evaluation quality and meet project deadlines and task completion targets.
Requirements
Bachelor's degree in Business Administration, Finance, Computer Science, or a related field.
At least 3 years of professional experience as a Business Analyst or in a similar analytical role.
Strong analytical thinking, problem-solving, and decision-making skills.
Proficiency in Excel and experience with SQL, Power BI, Tableau, or comparable analytical and reporting tools.
Experience gathering requirements, documenting functional specifications, and mapping business processes.
Strong attention to detail and the ability to compare complex information against defined criteria.
Excellent written communication, reporting, and documentation skills.
Ability to collaborate with cross-functional teams and manage multiple priorities independently.
Willingness to work five days per week, including weekends when required, with two weekdays off during the project.
Location: Remote - United States
Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.
About this Role:
This role is a 3-month, W2, full-time contract (40 hours per week.)
Scope of the Role:
We are looking for detail-oriented Voice Specialists to evaluate the performance of AI models through real-time, voice-based conversations. In this role, you will interact with two different AI models using the same assigned scenario, carefully compare their responses, and provide structured evaluations based on defined quality criteria.
The goal of the evaluation is to create a fair, consistent comparison between models and identify which model delivers the stronger overall conversational experience.
What You’ll Own:
Review an assigned scenario and roleplay a natural conversation with
two different AI models
.
Conduct comparable conversations with Model A and Model B, keeping the scenario, conversational approach, and overall interaction as consistent as possible.
Maintain approximately the same number of conversational turns with each model to support a fair comparison.
Record your voice during each interaction and pay close attention to the quality and behavior of the model's responses.
Evaluate each model across
five defined evaluation dimensions
.
Identify and categorize relevant
error clusters
that may occur in the audio or conversation.
Compare the performance of Model A and Model B based on the evaluation criteria and your observations.
Select the model that demonstrated the stronger overall performance according to the defined evaluation framework.
Write a detailed rationale explaining your final preference, referencing specific examples and observations from both conversations.
Apply evaluation guidelines consistently across different scenarios and model interactions
You’ll Thrive in This Role If You Have
Bachelors Degree
Strong attention to detail and ability to notice subtle differences in conversational quality.
Excellent listening and comprehension skills.
Strong written communication skills, particularly the ability to explain observations clearly and objectively.
Ability to consistently follow detailed evaluation guidelines.
Comfort speaking naturally and role-playing different conversational scenarios.
Ability to compare two interactions fairly without allowing personal preferences to influence the evaluation.
Strong critical-thinking and analytical skills.
Reliability and consistency when completing structured evaluation tasks.
Familiarity with AI assistants, voice-based AI, or conversational systems
The expected hourly salary range for this position is $20-$25 p/hour, based on experience, skills, and qualifications.
Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission’s guide at
https://consumer.ftc.gov/articles/job-scams.
If you believe you’ve been targeted by a recruitment scam, please report it to Innodata at
verifyjoboffer@innodata.com
and consider reporting it to the FTC at
ReportFraud.ftc.gov
.
Location: Khon Kaen, Thailand
As a Freelance Luxury Brand Evaluator, you will step into the world of luxury to discreetly assess customer experiences and provide critical feedback that helps brands refine their services, your insights will shape the future of luxury experiences.
Flexibility
- each in-store visit takes approximately 20 minutes, giving you the flexibility to complete multiple assignments.
Frequent opportunities
- perform several missions per month and collaborate with some of the most prestigious brands worldwide.
What you will be doing:
Choose your assignments - align your missions with your personal preferences and profile.
Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home.
Observe carefully - visit boutiques, check ambience, experience luxury automotive showrooms, purchase online, evaluate after-sales services, or overall customer experience.
Provide honest feedback - use our platform to share your observations through questionnaires.
About CXG
At CXG, we connect premium brands with over 80,000 evaluators across 70 countries. With 12 global offices and almost two decades of CX retail expertise, we’re committed to redefining the customer experience for luxury giants.
Join us and enjoy the flexibility of working on the go with our mobile platform, CXG Live. On this platform, you can seamlessly manage missions, certifications, and feedback.
Requirements
Current customer of premium and luxury brands.
Not currently under contract with any retail brands, to ensure impartiality.
Punctual, organized, detail-oriented, and reliable.
Observant and passionate about customer experience.
No prior experience is required; we value honest feedback from genuine customers.
Benefits
This is a
freelance
, project-based position
Flexible working hours
Location: Chanthaburi, Thailand
As a Freelance Luxury Brand Evaluator, you will step into the world of luxury to discreetly assess customer experiences and provide critical feedback that helps brands refine their services, your insights will shape the future of luxury experiences.
Flexibility
- each in-store visit takes approximately 20 minutes, giving you the flexibility to complete multiple assignments.
Frequent opportunities
- perform several missions per month and collaborate with some of the most prestigious brands worldwide.
What you will be doing:
Choose your assignments - align your missions with your personal preferences and profile.
Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home.
Observe carefully - visit boutiques, check ambience, experience luxury automotive showrooms, purchase online, evaluate after-sales services, or overall customer experience.
Provide honest feedback - use our platform to share your observations through questionnaires.
About CXG
At CXG, we connect premium brands with over 80,000 evaluators across 70 countries. With 12 global offices and almost two decades of CX retail expertise, we’re committed to redefining the customer experience for luxury giants.
Join us and enjoy the flexibility of working on the go with our mobile platform, CXG Live. On this platform, you can seamlessly manage missions, certifications, and feedback.
Requirements
Not currently under contract with any retail brands, to ensure impartiality.
Punctual, organized, detail-oriented, and reliable.
Observant and passionate about customer experience.
No prior experience is required; we value honest feedback from genuine customers.
Benefits
This is a
freelance
, project-based position
Flexible working hours
Location: Remote - Americas
Who We Are:
Alpaca is a US-headquartered, global leader in agent-first brokerage infrastructure
for stocks, ETFs, options, crypto, fixed income, 24/5 trading, and more.
Amongst our subsidiaries, Alpaca is a licensed financial services company, serving hundreds of financial institutions across 40 countries with our institutional-grade APIs. This includes broker-dealers, investment advisors, wealth managers, hedge funds, and crypto exchanges, totalling over 10 million brokerage accounts.
Our global team is a diverse group of experienced engineers, traders, and brokerage professionals who are working to achieve our mission of opening financial services to everyone on the planet. We're deeply committed to open-source contributions and fostering a vibrant community, continuously enhancing our award-winning, developer-friendly API and the robust infrastructure behind it.
Alpaca is proudly backed by $400 million in funding from top-tier global investors including Portage Ventures, Spark Capital, Tribe Capital, Social Leverage, Horizons Ventures, Opera Tech Ventures, SBI Group, Derayah Financial, Unbound, Peak XV, Elefund, and Y Combinator.
Our Team Members:
We're a dynamic team of 400+ globally distributed members who thrive working from our favorite places around the world, with teammates spanning the USA, Canada, Japan, Hungary, Nigeria, Brazil, the UK, and beyond!
We're searching for passionate individuals eager to contribute to Alpaca's rapid growth. If you align with our core values—Stay Curious, Have Empathy, and Be Accountable—and are ready to make a significant impact, we encourage you to apply.
Your Role
: We're looking for a Senior Data Scientist, AI Evaluation to design how Alpaca measures whether our models and agents are actually right. You'll be a senior individual contributor who turns ambiguous quality questions into ground truth, scoring methods, and eval loops that the company can trust—and uses those results to make the systems better. You'll build on an established data foundation, so the focus is raising quality and speeding up safe rollout. You'll own the quality bar, independent of the teams that build and optimize those systems.
This role is for someone who cares as much about whether an answer is correct as about whether a model can generate one. You'll partner with Product, Engineering, Analytics Engineering, and business stakeholders to define what "good" looks like, build the evaluations that test it, and close the loop so evals drive iteration. If you have a strong quantitative background, have shipped rigorous, measurable work (evaluation, experimentation, or model validation), and want ownership over a greenfield eval practice at a fast-growing brokerage-infrastructure company, this is the role.
What You'll Do
Design AI evaluations: Define ground truth, metrics, and scoring methods for models and agents.
Build repeatable eval loops: Track quality over time and catch regressions before release.
Partner on infrastructure: Work with engineering and analytics engineering to operationalize eval harnesses.
Drive iteration: Translate eval results into actionable recommendations for system improvements.
Establish quality standards: Set evaluation guidelines, documentation, and review practices.
Mentor and align: Foster evaluation best practices and build a culture of measurable AI quality across the team.
What We're Looking For
Track record of quantitative measurement rigor (e.g., LLM/model evaluation, metric validation, or experimentation).
Strong statistical and ML foundation—you treat evaluations as experiments (sample sizing, confidence intervals, significance, handling non-determinism) and validate automated graders against human ground truth.
Proficiency in Python and SQL, with experience evaluating models in production environments.
Strong judgment in defining quality metrics and ground truth for ambiguous outputs.
Excellent communication and cross-functional collaboration skills to align technical teams and leadership.
Strong problem-solving ability in fast-paced, greenfield environments.
6–10 years in quantitative data science or ML, with focused experience in measurement or evaluation. A quantitative degree is a plus; equivalent industry experience is equally welcome.
Nice to Have
Hands-on LLM/agent evaluation in production, including eval harnesses, LLM-as-judge calibration, and CI regression gates.
Experience evaluating text-to-SQL, analytics agents, or other systems where correctness is verifiable against data.
Background in fintech, brokerage, or other domains where a wrong answer has real business or risk consequences.
Fluency with AI tools in research and engineering workflows.
How We Take Care of You:
Competitive Salary & Stock Options
Health Benefits
New Hire Home-Office Setup: One-time USD $500
Monthly Stipend: USD $150 per month via a Brex Card
Alpaca is proud to be an equal opportunity workplace dedicated to pursuing and hiring a diverse workforce.
Recruitment Privacy Policy
Location: Pattaya City, Thailand
Turn your passion for luxury into a career opportunity! Explore the world of premium brands and make a lasting impact in fashion, beauty, jewelry, or automobiles.
About the Role:
As a luxury brand evaluator, you will step into the world of luxury to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques, purchasing online, or returning a product, your voice shapes the future of luxury.
Flexibility
- each in-store visit takes approximately 20 minutes, giving you the flexibility to complete multiple assignments.
Frequent opportunities
- perform several missions per month and collaborate with some of the most prestigious brands worldwide.
What you will be doing:
Choose your assignments - align your missions with your personal preferences and profile.
Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home.
Observe carefully - check the ambiance, service quality, and overall customer experience.
Provide honest feedback - use our platform to share your observations through questionnaires.
About CXG
At CXG, we connect premium brands with over 80,000 evaluators across 70 countries. With 12 global offices and almost two decades of CX retail expertise, we’re committed to redefining the customer experience for luxury giants.
Join us and enjoy the flexibility of working on the go with our mobile platform,
CXG Live
. On this platform, you can seamlessly manage missions, certifications, and feedback.
Requirements
Current customer of premium and luxury brands.
Not currently under contract with any retail brands, to ensure impartiality.
Punctual, organized, detail-oriented, and reliable.
Observant and passionate about customer experience.
No prior experience is required; we value honest feedback from genuine customers.
Benefits
This is a
freelance
, project-based position
Compensation for Non-Purchase Evaluations: Earn a fee based on mission complexity.
Compensation for Purchase-Based Evaluations: Reimbursement for approved items as stated in the mission brief.
Location: Bang Nak, Thailand
Turn your passion for luxury into a career opportunity! Explore the world of premium brands and make a lasting impact in fashion, beauty, jewelry, or automobiles.
About the Role:
As a luxury brand evaluator, you will step into the world of luxury to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques, purchasing online, or returning a product, your voice shapes the future of luxury.
Flexibility
- each in-store visit takes approximately 20 minutes, giving you the flexibility to complete multiple assignments.
Frequent opportunities
- perform several missions per month and collaborate with some of the most prestigious brands worldwide.
What you will be doing:
Choose your assignments - align your missions with your personal preferences and profile.
Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home.
Observe carefully - check the ambiance, service quality, and overall customer experience.
Provide honest feedback - use our platform to share your observations through questionnaires.
About CXG
At CXG, we connect premium brands with over 80,000 evaluators across 70 countries. With 12 global offices and almost two decades of CX retail expertise, we’re committed to redefining the customer experience for luxury giants.
Join us and enjoy the flexibility of working on the go with our mobile platform,
CXG Live
. On this platform, you can seamlessly manage missions, certifications, and feedback.
Requirements
Current customer of premium and luxury brands.
Not currently under contract with any retail brands, to ensure impartiality.
Punctual, organized, detail-oriented, and reliable.
Observant and passionate about customer experience.
No prior experience is required; we value honest feedback from genuine customers.
Benefits
This is a
freelance
, project-based position
Compensation for Non-Purchase Evaluations: Earn a fee based on mission complexity.
Compensation for Purchase-Based Evaluations: Reimbursement for approved items as stated in the mission brief.
Location: Calapan City, Philippines
As a Luxury Brand Evaluator, you will step into the world of luxury to discreetly assess customer experiences and provide critical feedback that helps brands refine their services, your insights will shape the future of luxury experiences.
Flexibility
- each in-store visit takes approximately 20 minutes, giving you the flexibility to complete multiple assignments.
Frequent opportunities
- perform several missions per month and collaborate with some of the most prestigious brands worldwide.
What you will be doing:
Choose your assignments - align your missions with your personal preferences and profile.
Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home.
Observe carefully - visit boutiques, check ambience, experience luxury automotive showrooms, purchase online, evaluate after-sales services, or overall customer experience.
Provide honest feedback - use our platform to share your observations through questionnaires.
About CXG
At CXG, we connect premium brands with over 80,000 evaluators across 70 countries. With 12 global offices and almost two decades of CX retail expertise, we’re committed to redefining the customer experience for luxury giants.
Join us and enjoy the flexibility of working on the go with our mobile platform,
CXG Live
. On this platform, you can seamlessly manage missions, certifications, and feedback.
Requirements
Passionate about automobiles and improving customer service and retail environments.
Enjoy interacting with people.
Has a keen eye for discreetly observing and noting various aspects of your shopping experience.
Willingness to adapt to varying assignment types and industries.
Ability to provide unbiased, honest feedback without personal biases and be prompt in filling out online surveys
Benefits
This is a
freelance
, project-based position
Flexible working hours
Location: Tokyo, Japan; Barcelona, Spain; Portland, OR
Workplace: onsite
About Welocalize
Welocalize, a Welo Global brand, serves localization teams through AI-enabled multilingual content solutions that enable enterprises to operate and scale globally. Welocalize combines AI, automation, and human expertise to support enterprises in more than 300 languages, enabling accurate, culturally aligned, and compliant multilingual content at scale.
Welocalize’s Opal Platform comprises patented technology designed to automate multilingual content and improve workflow performance for enterprises. Opal functions as an agentic system that orchestrates AI, automation, and human expertise to coordinate content workflows across the lifecycle, improving speed, scalability, and operational control for global enterprises. These solutions operate within a secure and compliant environment supported by seven ISO certifications. Welocalize is headquartered in New York with offices worldwide. welocalize.com
ROLE OVERVIEW
Welocalize is looking for a native-level Marathi speaker to join a confidential, large-scale AI and consumer technology programme for a major global technology and communications client.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how the product actually performs on a client-supplied device — a smartphone and a next-generation smart glass. If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
Welocalize is looking for a native-level Marathi speaker to join a large-scale AI and consumer technology programme.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how it performs on a real device. You'll be among the first people to see this technology in Marathi, and your judgement shapes how well it works before anyone else uses it.
If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
MAIN RESPONSIBILITIES
Review pre-release AI model output in Marathi and judge it on accuracy, naturalness, appropriateness, and overall quality — would a native speaker really say it this way? This includes fact-checking each response against the source material.
Location: Barcelona, Spain; Tokyo, Japan; Portland, OR
Workplace: onsite
About Welocalize
Welocalize, a Welo Global brand, serves localization teams through AI-enabled multilingual content solutions that enable enterprises to operate and scale globally. Welocalize combines AI, automation, and human expertise to support enterprises in more than 300 languages, enabling accurate, culturally aligned, and compliant multilingual content at scale.
Welocalize’s Opal Platform comprises patented technology designed to automate multilingual content and improve workflow performance for enterprises. Opal functions as an agentic system that orchestrates AI, automation, and human expertise to coordinate content workflows across the lifecycle, improving speed, scalability, and operational control for global enterprises. These solutions operate within a secure and compliant environment supported by seven ISO certifications. Welocalize is headquartered in New York with offices worldwide. welocalize.com
ROLE OVERVIEW
Welocalize is looking for a native-level Kannada speaker to join a confidential, large-scale AI and consumer technology programme for a major global technology and communications client.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how the product actually performs on a client-supplied device — a smartphone and a next-generation smart glass. If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
Welocalize is looking for a native-level Kannada speaker to join a large-scale AI and consumer technology programme.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how it performs on a real device. You'll be among the first people to see this technology in Kannada, and your judgement shapes how well it works before anyone else uses it.
If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
MAIN RESPONSIBILITIES
Review pre-release AI model output in Kannada and judge it on accuracy, naturalness, appropriateness, and overall quality — would a native speaker really say it this way? This includes fact-checking each response against the source material.
Location: Tokyo, Japan; Barcelona, Spain; Portland, OR
Workplace: onsite
About Welocalize
Welocalize, a Welo Global brand, serves localization teams through AI-enabled multilingual content solutions that enable enterprises to operate and scale globally. Welocalize combines AI, automation, and human expertise to support enterprises in more than 300 languages, enabling accurate, culturally aligned, and compliant multilingual content at scale.
Welocalize’s Opal Platform comprises patented technology designed to automate multilingual content and improve workflow performance for enterprises. Opal functions as an agentic system that orchestrates AI, automation, and human expertise to coordinate content workflows across the lifecycle, improving speed, scalability, and operational control for global enterprises. These solutions operate within a secure and compliant environment supported by seven ISO certifications. Welocalize is headquartered in New York with offices worldwide. welocalize.com
ROLE OVERVIEW
Welocalize is looking for a native-level Telugu speaker to join a confidential, large-scale AI and consumer technology programme for a major global technology and communications client.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how the product actually performs on a client-supplied device — a smartphone and a next-generation smart glass. If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
Welocalize is looking for a native-level Telugu speaker to join a large-scale AI and consumer technology programme.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how it performs on a real device. You'll be among the first people to see this technology in Telugu, and your judgement shapes how well it works before anyone else uses it.
If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
MAIN RESPONSIBILITIES
Review pre-release AI model output in Telugu and judge it on accuracy, naturalness, appropriateness, and overall quality — would a native speaker really say it this way? This includes fact-checking each response against the source material.
Location: Tokyo, Japan; Barcelona, Spain; Portland, OR
Workplace: onsite
About Welocalize
Welocalize, a Welo Global brand, serves localization teams through AI-enabled multilingual content solutions that enable enterprises to operate and scale globally. Welocalize combines AI, automation, and human expertise to support enterprises in more than 300 languages, enabling accurate, culturally aligned, and compliant multilingual content at scale.
Welocalize’s Opal Platform comprises patented technology designed to automate multilingual content and improve workflow performance for enterprises. Opal functions as an agentic system that orchestrates AI, automation, and human expertise to coordinate content workflows across the lifecycle, improving speed, scalability, and operational control for global enterprises. These solutions operate within a secure and compliant environment supported by seven ISO certifications. Welocalize is headquartered in New York with offices worldwide. welocalize.com
ROLE OVERVIEW
Welocalize is looking for a native-level Tamil speaker to join a confidential, large-scale AI and consumer technology programme for a major global technology and communications client.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how the product actually performs on a client-supplied device — a smartphone and a next-generation smart glass. If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
Welocalize is looking for a native-level Tamil speaker to join a large-scale AI and consumer technology programme.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how it performs on a real device. You'll be among the first people to see this technology in Tamil , and your judgement shapes how well it works before anyone else uses it.
If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
MAIN RESPONSIBILITIES
Review pre-release AI model output in Tamil and judge it on accuracy, naturalness, appropriateness, and overall quality — would a native speaker really say it this way? This includes fact-checking each response against the source material.
Location: Barcelona, Spain; Portland, OR; Tokyo, Japan
Workplace: onsite
About Welocalize
Welocalize, a Welo Global brand, serves localization teams through AI-enabled multilingual content solutions that enable enterprises to operate and scale globally. Welocalize combines AI, automation, and human expertise to support enterprises in more than 300 languages, enabling accurate, culturally aligned, and compliant multilingual content at scale.
Welocalize’s Opal Platform comprises patented technology designed to automate multilingual content and improve workflow performance for enterprises. Opal functions as an agentic system that orchestrates AI, automation, and human expertise to coordinate content workflows across the lifecycle, improving speed, scalability, and operational control for global enterprises. These solutions operate within a secure and compliant environment supported by seven ISO certifications. Welocalize is headquartered in New York with offices worldwide. welocalize.com
ROLE OVERVIEW
Welocalize is looking for a native-level Traditional Chinese speaker to join a large-scale AI and consumer technology programme.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how it performs on a real device. You'll be among the first people to see this technology in Traditional Chinese, and your judgement shapes how well it works before anyone else uses it.
If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
MAIN RESPONSIBILITIES
Review pre-release AI model output in Traditional Chinese and judge it on accuracy, naturalness, appropriateness, and overall quality — would a native speaker really say it this way? This includes fact-checking each response against the source material.
Location: Barcelona, Spain; Portland, OR; Tokyo, Japan
Workplace: onsite
About Welocalize
Welocalize, a Welo Global brand, serves localization teams through AI-enabled multilingual content solutions that enable enterprises to operate and scale globally. Welocalize combines AI, automation, and human expertise to support enterprises in more than 300 languages, enabling accurate, culturally aligned, and compliant multilingual content at scale.
Welocalize’s Opal Platform comprises patented technology designed to automate multilingual content and improve workflow performance for enterprises. Opal functions as an agentic system that orchestrates AI, automation, and human expertise to coordinate content workflows across the lifecycle, improving speed, scalability, and operational control for global enterprises. These solutions operate within a secure and compliant environment supported by seven ISO certifications. Welocalize is headquartered in New York with offices worldwide. welocalize.com
ROLE OVERVIEW
Welocalize is looking for a native-level Mandarin Chinese speaker to join a large-scale AI and consumer technology programme.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how it performs on a real device. You'll be among the first people to see this technology in Mandarin Chinese, and your judgement shapes how well it works before anyone else uses it.
If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
MAIN RESPONSIBILITIES
Review pre-release AI model output in Mandarin Chinese and judge it on accuracy, naturalness, appropriateness, and overall quality — would a native speaker really say it this way? This includes fact-checking each response against the source material.
Location: Tokyo, Japan; Barcelona, Spain; Portland, OR
Workplace: onsite
About Welocalize
Welocalize, a Welo Global brand, serves localization teams through AI-enabled multilingual content solutions that enable enterprises to operate and scale globally. Welocalize combines AI, automation, and human expertise to support enterprises in more than 300 languages, enabling accurate, culturally aligned, and compliant multilingual content at scale.
Welocalize’s Opal Platform comprises patented technology designed to automate multilingual content and improve workflow performance for enterprises. Opal functions as an agentic system that orchestrates AI, automation, and human expertise to coordinate content workflows across the lifecycle, improving speed, scalability, and operational control for global enterprises. These solutions operate within a secure and compliant environment supported by seven ISO certifications. Welocalize is headquartered in New York with offices worldwide. welocalize.com
ROLE OVERVIEW
Welocalize is looking for a native-level Malay speaker to join a confidential, large-scale AI and consumer technology programme for a major global technology and communications client.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how the product actually performs on a client-supplied device — a smartphone and a next-generation smart glass. If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
Welocalize is looking for a native-level Malay speaker to join a large-scale AI and consumer technology programme.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how it performs on a real device. You'll be among the first people to see this technology in Malay, and your judgement shapes how well it works before anyone else uses it.
If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
MAIN RESPONSIBILITIES
Review pre-release AI model output in Malay and judge it on accuracy, naturalness, appropriateness, and overall quality — would a native speaker really say it this way? This includes fact-checking each response against the source material.
ОфисTokyo, Japan; Barcelona, Spain; Portland, OR
Paris; Benefits vary by country and may include healthcare coverage, parental leave, retirement plans, relocation support, wellness programs, meal and transportation allowances, and other location-specific perks
УдалённоUnited States
Не хотите искать каждый день?
Настройте подбор один раз, и подходящие вакансии будут сами приходить в Telegram.
Настройте подбор, и подходящие вакансии будут приходить в Telegram.