Generalist Rater

Title: Generalist Rater – AI Content & Prompt Evaluation

Location: Remote (Candidates based in US/UK preferred)

About the Role:

We are seeking Generalist Raters to work with cutting-edge Large Language Models (LLMs). As a rater, you will design prompts, evaluate AI-generated responses, and provide detailed feedback that helps improve the quality, accuracy, and creativity of AI systems.

Responsibilities:

• Design and write effective prompts to elicit desired responses from LLMs.
• Generate creative and informative text formats (stories, articles, summaries, dialogues).
• Evaluate and compare AI-generated outputs on fluency, coherence, factual accuracy, and bias.
• Analyze LLM outputs and provide detailed feedback for model improvement.
• Collaborate with engineers, researchers, and product teams to refine strategies.
• Contribute to guidelines and best practices for AI evaluation.

Qualifications:

• Bachelor’s degree in English, Creative Writing, Linguistics, Computer Science, or related field.
• 3+ years of experience in content writing, copywriting, or a related area.
• Strong understanding of LLMs and their capabilities/limitations.
• Excellent writing and editing skills, adaptable to different styles and tones.
• Analytical, detail-oriented, and able to work independently.
• Familiarity with data annotation, prompt engineering, or evaluation tools is a plus.

Preferred:

• Experience with LLM architectures, NLP concepts, or AI research.
• Multilingual ability (Chinese, Spanish, Portuguese, Arabic, French, German).
• Tech savvy and eager to grow technical skills.