Remote Freelance AI Conversation Evaluator

Outlier is an innovative AI training platform powered by Scale AI, where Outlier Experts tackle diverse projects from generating training data to assessing model performance.

About the opportunity

We are on the lookout for detail-oriented English speakers who can effectively review AI-generated media.

Engage in live conversations with AI voice models, score these exchanges, compare AI-generated images and videos to determine which aligns better with the prompts, and articulate in writing any discrepancies you observe, all from the comfort of your home and on your own schedule.

Your contributions at Outlier will play a crucial role in honing how advanced AI systems reason, learn, and evolve. By participating in Outlier's feedback loop, you'll help transform cutting-edge research into AI that excels in real-world applications.

Responsibilities

• Engage in live scenario-driven conversations in English with AI voice models, evaluating their naturalness, utility, and clarity.
• Review pairs of AI-generated images and videos, deciding which best adheres to the given prompt.
• Conduct meticulous inspections of outputs, including zooming in or replaying segments, to identify issues like distorted features, unnatural movements, textual errors, and misaligned audio.
• Assess each output on various metrics such as adherence, quality, and realism, and provide specific written feedback highlighting your observations.

Common tasks

• Image evaluations: Compare two AI-generated images from the same text prompt and select the superior one based on adherence to instructions, visual quality, and apparent AI artifacts. Some tasks may require evaluating edited images against originals.
• Video evaluations: Rank AI-generated videos based on prompts or evaluate edited videos against their source materials.
• Video caption labeling: Label first-person videos of individuals or robots performing tasks. Contributors will break down clips into individual actions, documenting precise captions starting from an AI draft and enhancing them as needed.
• Audio and voice evaluations: Assess AI-generated speech regarding its naturalness, pronunciation accuracy, and alignment with a target voice. Some evaluations may require you to record a brief voice explanation of your assessment.

Ideal qualifications

• Proficient in English with strong written communication skills and adept in conversational speech.
• Meticulous attention to detail and patience to closely analyze outputs, replaying or zooming in if necessary.
• Ability to think creatively and engage in realistic conversational scenarios.
• Experience in writing concise, evidence-based explanations or reviews.
• Comfortable making clear decisions even when both options present challenges.
• Preferred: Currently pursuing or holding a Bachelor’s degree or higher.

What you'll need to sign up

• A valid ID and mobile phone from your country of residence.
• An updated resume showcasing your qualifications.
• A LinkedIn profile detailing your educational background and work history.

Payment

• Project work: Earn up to USD $15 per hour.
• Additional incentives: Contributors, on average, earn an extra 7.5% through Missions, Outlier's version of surge pricing; the top quartile averages an 11% boost in earnings.
• Payment schedule: Payments are issued weekly on Tuesdays, covering tasks completed during the preceding pay cycle (Tuesday to Monday at midnight UTC), via PayPal, AirTM, or ACH transfer.
• Payment rates may differ based on skill level, location, project requirements, and other factors.

Notably higher rates may be available for applicants with advanced degrees.

Please refer to the payment terms for specific projects.

Why join Outlier

• Enjoy the freedom to work remotely at your convenience, with no minimum hour requirements. Decide your workload, whether that's five hours or over 40 hours weekly.
• Access premium LLMs valued at over $200 monthly for free.
• Join a community of over 900K Outlier Experts across 52 countries, with $600M+ compensated to contributors.
• Collaborate with professionals in your field to advance the capabilities of pre-release models.

How to apply

• Create an account on Outlier.
• Upload your resume.
• Submit your valid ID.
• Begin earning by completing tasks.

Currently accepting applicants from: Australia, Canada, Ireland, New Zealand, United Kingdom, and United States.

Please note: This is a 1099 contract opportunity available on the Outlier.ai platform.

You will be joining Outlier as a freelancer/independent contractor, not an employee.

As this is a freelance role, we do not provide internships, sponsorship, or employment benefits.

Authorization to work in your country of residence is required.

International students may be eligible to join Outlier while on a visa, and should consult their tax and/or immigration advisor for specific inquiries.