Location: Pune, Maharashtra, India
Marathi AI Product Evaluator
🎯
The Project
Productive Playhouse is building a talent pool of Marathi speakers for an upcoming project testing and evaluating leading AI chatbots. The objective is to enhance response quality through direct user interaction with the various AI models.
Open to freelancers based outside the U.S.
This is more than one job. Apply once, and we'll keep you in mind for this project — and future ones that need your language skills.
The Details
We’re looking for independent contractor engagement (task-based - project-based).
Pay Rate: $8.00 USD per hour
100% remote
You choose which tasks to take, set your own hours, and are free to work with other clients any time. You can stay flexible as the project evolves.
This is an iterative project, work arrives in batches (there may be pauses between them).
What You’ll Do
As an AI Evaluator, you'll play a critical role in shaping and improving the next generation of Generative AI. You'll participate in structured, hands-on evaluations by interacting directly with various AI models to assess their capabilities, safety, and helpfulness. Your insights and data will directly inform model development and optimization.
Exact tasks vary by project and will be spelled out in that project's Statement of Work (SOW) before you start. Depending on the role, your work may include:
Testing and evaluating AI chatbots or language models through structured conversations, using assigned goals and prompts.
Voice-based interactions with AI models — for projects that involve this, sessions are recorded, and audio may be shared with the client as part of the evaluation deliverable.
Submitting deliverables (write-ups, ratings, screenshots, or recordings) in the format each task specifies.
✅ What We're Looking For
Native or expert-level fluency in Marathi, as spoken in India
Strong English reading, writing, and communication skills - assessment maybe required
18 years of age or older
Hands-on experience using AI models or large language models
Comfortable with both written and voice-based conversation — and good at asking sharp follow-ups
Your own smartphone, computer, reliable internet, and comfort with web-based tools
Able to pass a language proficiency and technical literacy assessment
Responsive communicator — quick replies move you through onboarding faster
Able to start within 48 hours
Why PPH? 🌍
🌍 Your language skills are the whole point — and this project pays for them
⚡ Apply once, get contacted directly when a spot opens — no chasing
🏠 Work from anywhere, on your own schedule, alongside any other work you do
🤝 Clear task specs, no ongoing oversight, no micromanagement
💼 Real project experience with a global data and language services company
✨ Work that actually improves how AI understands your language
About Productive Playhouse
We started by teaching kids through award-winning programming. We grew into a global data and language services company trusted by clients worldwide. But the mission never changed: keep language alive.
Transcription, translation, localization, linguistic analysis, AI evaluation — it all comes back to preserving languages and the cultures they carry.
All engagements are contingent upon successful completion of identity verification. Productive Playhouse is an equal opportunity organization headquartered in California and committed to diversity and inclusion across our global workforce. We welcome applicants of all backgrounds and abilities, regardless of location or engagement type. For accommodations or inquiries, please contact HR.
Pune, Maharashtra, India; We welcome applicants of all backgrounds and abilities, regardless of location or engagement type
Location: California; Sunnyvale, CA; San Francisco, CA; Seattle, WA; Redmond, Washington; Burlingame, CA; New York City Metropolitan Area; Los Angeles, CA; Washington, D.C.; Boston, MA
Workplace: onsite
Overview
Welo Data is looking for sharp, curious, and detail-oriented individuals to join our team as Data Labeling Associate.
This is not a traditional annotation role.
You’ll be working directly with cutting-edge AI systems — evaluating outputs, identifying gaps, and helping improve how these systems behave in real-world scenarios. The work sits at the intersection of data quality, model evaluation, and human judgment, where your ability to think critically matters just as much as following guidelines.
We’re looking for people who are naturally curious about AI, comfortable forming opinions, and confident in contributing to conversations with teammates, leads, and stakeholders.
Project Details
Job Title: Data Labeling Associate
Hiring in: Onsite (Bay Area, Seattle, NYC, or client-dependent)
Hours: Full-time, 40 hours per week
Employment Type: W2 Full-Time Employee
Work Authorization: Must be authorized to work in the U.S. (no visa sponsorship)
Pay Rate: $34/hour
Important: This is a 100% onsite position — remote work is not available for this role. To be considered, candidates must be located in or able to commute to one of the following cities: NYC, Seattle, Bellevue, Redmond, San Francisco, Sunnyvale, Burlingame, Austin, Los Angeles, Washington DC, Chicago, Boston, Austin. Please only apply if you meet this location requirement.
What You’ll Do
Evaluate AI model outputs and provide structured, high-quality feedback
Perform audit-based reviews of data and model behavior — identifying patterns, edge cases, and failure modes
Apply guidelines thoughtfully — and flag when they don’t reflect real-world scenarios
Contribute to improving evaluation frameworks, not just executing them
Identify trends in model performance and communicate insights clearly
Participate in team discussions, calibrations, and stakeholder syncs
Partner with leads and cross-functional teams to refine quality standards
Document findings in a clear, concise, and actionable way
What We’re Looking For
Native-level Marathi language proficiency
and a
university degree
(Bachelor’s or higher).
B2 or superior level of English.
1–2 years of professional writing experience with strong, structured writing skills
Ability to apply complex writing rules and guidelines consistently
Strong understanding of safety considerations in GenAI data delivery, with 2+ years of relevant experience
Strong critical thinking and attention to detail
Ability to make sound judgment calls in ambiguous situations
Naturally curious about AI, technology, and how systems behave
Comfortable speaking up, asking questions, and contributing ideas
Strong written and verbal communication skills
Ability to stay consistent while working with evolving guidelines
Experience in data quality, QA, annotation, or analysis is helpful — but not required
Benefits
Paid Vacation: 6 days
Paid Company Holidays: 2 days (Memorial Day and Labor Day)
Paid Sick Leave: accrued per applicable state law and company policy
Medical, Dental, and Vision Insurance (eligibility applies)
Health Savings Account (HSA)
401(k) Retirement Plan
Employee Assistance Program
Additional voluntary benefits (life, accident, critical illness, etc.)
Free Gourmet Food: Free breakfast, lunch, and dinner are provided, featuring a wide variety of cuisines in multiple cafes.
Micro-kitchens & Snacks: Offices are stocked with free snacks and beverages, including premium coffee and La Croix.
Unique Campus Features: Some locations include roof-top nature parks
Commuter Benefits: Free transport, shuttles, and sometimes bike-to-work perks.
Please note that in order to verify work authorization as is required by Federal law (I-9 process), all new employees must complete a live video verification with their selected IDs and provide photos of these selected IDs within their first 3 days of employment.
To know more details (Click here)
In compliance with federal law, all persons hired will be required to verify identity and eligibility to work in the United States and to complete the required employment eligibility verification form upon hire. In addition, we employ anti-fraud checks to ensure all candidates meet the requirements of the program.
As a trusted global transformation partner, Welocalize accelerates the global business journey by enabling brands and companies to reach, engage, and grow international audiences. Welocalize delivers multilingual content transformation services in translation, localization, and adaptation for over 250 languages with a growing network of over 400,000 in-country linguistic resources. Driving innovation in language services, Welocalize delivers high-quality training data transformation solutions for NLP-enabled machine learning by blending technology and human intelligence to collect, annotate, and evaluate all content types. Our team works across locations in North America, Europe, and Asia serving our global clients in the markets that matter to them. www.welocalize.com
To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform essential functions.
Why This Role
This is an opportunity to move beyond traditional data work and play a direct role in how AI systems are evaluated and improved. The work is fast-moving, collaborative, and increasingly central to how modern AI systems are built and deployed.
Location: Tokyo, Japan; Barcelona, Spain; Portland, OR
Workplace: onsite
About Welocalize
Welocalize, a Welo Global brand, serves localization teams through AI-enabled multilingual content solutions that enable enterprises to operate and scale globally. Welocalize combines AI, automation, and human expertise to support enterprises in more than 300 languages, enabling accurate, culturally aligned, and compliant multilingual content at scale.
Welocalize’s Opal Platform comprises patented technology designed to automate multilingual content and improve workflow performance for enterprises. Opal functions as an agentic system that orchestrates AI, automation, and human expertise to coordinate content workflows across the lifecycle, improving speed, scalability, and operational control for global enterprises. These solutions operate within a secure and compliant environment supported by seven ISO certifications. Welocalize is headquartered in New York with offices worldwide. welocalize.com
ROLE OVERVIEW
Welocalize is looking for a native-level Marathi speaker to join a confidential, large-scale AI and consumer technology programme for a major global technology and communications client.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how the product actually performs on a client-supplied device — a smartphone and a next-generation smart glass. If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
Welocalize is looking for a native-level Marathi speaker to join a large-scale AI and consumer technology programme.
It's a hands-on, on-site role that combines two things: evaluating a pre-release AI model in your language, and testing how it performs on a real device. You'll be among the first people to see this technology in Marathi, and your judgement shapes how well it works before anyone else uses it.
If you enjoy noticing the detail other people miss, this is work you'll find satisfying.
MAIN RESPONSIBILITIES
Review pre-release AI model output in Marathi and judge it on accuracy, naturalness, appropriateness, and overall quality — would a native speaker really say it this way? This includes fact-checking each response against the source material.
Location: India
Workplace: remote
We are looking for qualified raters to support a project focused on evaluating personalized search and location recommendations. This role involves rating the relevance and usefulness of suggested places based on personalized results and defined project guidelines.
𝗜𝗳 𝘆𝗼𝘂’𝗿𝗲 𝘀𝗼𝗺𝗲𝗼𝗻𝗲 𝘄𝗵𝗼:
✅Notices when recommendations actually make sense
🧠Thinks critically about usefulness, intent, and context
🖥️Cares about accuracy in digital experiences
This project is built for you.
Contract type: Freelance
Language: Marathi (India)
Location: Mumbai
Estimated volume: 8 - 10 hours
Start date: The project runs on a weekly basis.
What you will do:
Review search queries with results personalized based on your activity history
Evaluate suggested places (e.g., restaurants or locations) related to places you’ve previously visited
Rate whether the suggested places are relevant and useful
Follow project guidelines and complete tasks accurately
Requirements:
To be located and have the right to work in India.
An existing, actively used
Gmail account
with prior usage history on
Google Maps.
Participation is limited to maximum three project rounds. Please note that the task request will be dynamic and we will be providing you with a heads-up as soon as possible once there is a project round available for you.
Please note:
This project requires opt-in, consent-based access to limited elements of your activity history. A consent form will be provided for you to review and sign before participation. Please note that the consent form and its terms may be subject to change as project requirements evolve.
A desktop or laptop computer is required to complete the application and onboarding process. Mobile devices are not supported.
Location: India; Remote, Worldwide; Remote - Europe
Workplace: remote
Welo Data, part of Welocalize, is a global AI data company with 500,000+ contributors delivering high-quality, ethical data to train the world’s most advanced AI systems. We’re building smarter, more human AI with a diverse community in 100+ countries.
At Welo Data, Limitless AI. Limitless You. isn’t just a slogan—it’s our promise. We build smarter AI through the power of human contribution, offering limitless opportunities for our global community to grow, contribute, and work on their terms.
Are you detail-oriented and curious about tech?
Welo Data is building a global network of linguists, language enthusiasts, and culturally aware contributors to shape safer, smarter AI.
By joining our talent community, you’ll be first in line for flexible, remote projects in annotation, evaluation, and prompt creation—always on your terms.
Note: This is not an active job opening. By applying, you’ll join our global contributor network. When a project aligns with your skills and availability, we’ll reach out with all the details, including scope, timeline, and compensation.
What You’ll Need
- Native or near-native fluency in Marathi
- Based in: India
- English proficiency
- Comfortable with digital tools
- Detail-oriented and curious
Why Join Us?
🚀 Limitless You – Work when and where you want, with room to grow
🤖 Limitless AI – Contribute to smarter, more inclusive AI
🌎 Be Part of Us – Join a global, supportive community
About our projects:
Welo Data projects vary by scope and complexity, but all play a role in building more accurate, inclusive, and human-centered AI.
Depending on your skills, you might:
📝 Annotate or label text, images, or audio to train AI systems
🔍 Evaluate search results, ads, or chatbot responses for quality and relevance
💬 Create and test prompts to improve large language models (LLMs)
🌍 Ensure content reflects cultural, linguistic, or industry-specific accuracy
Some projects last a few days; others may extend over several weeks. Before each one, you’ll receive clear details on tasks, timelines, and compensation, so you can decide what works best for you.
How to Apply?
Click the “Apply” button on our website to fill out a short form and join our talent database. It only takes a few minutes to get started and become part of our growing community.
📬 Questions?
If you have any questions about the process, feel free to contact us at TalentRequests@welocalize.com.
УдалённоIndia; Remote, Worldwide; Remote - Europe
ОфисCalifornia; Sunnyvale, CA; San Francisco, CA; Seattle, WA; Redmond, Washington; Burlingame, CA; New York City Metropolitan Area; Los Angeles, CA; Washington, D.C.; Boston, MA; To be considered, candidates must be located in or able to commute to one of the following cities: NYC, Seattle, Bellevue, Redmond, San Francisco, Sunnyvale, Burlingame, Austin, Los Angeles, Washington DC, Chicago, Boston, Austin
Настройте подбор один раз, и подходящие вакансии будут сами приходить в Telegram.
Настройте подбор, и подходящие вакансии будут приходить в Telegram.