Generalist (Portuguese) | $15/hr Remote

Position: LLM – AI Quality Analyst (Personalization) – Portuguese

Type: Short-Term Contract (2 months)

Compensation: $11-$15 per hour

Location: Remote

Commitment: Part-time (20–40 hrs/week)

Role Responsibilities

• Evaluate a personalization feature for Gemini
• Design and execute multi-turn conversational prompts that require the AI to leverage personal context
• Assess how effectively the model uses personal context from past conversations and activity to generate relevant responses
• Evaluate responses based on intent and appropriate application of personalization
• Analyze responses for grounding issues, flawed inferences, or hallucinations
• Assess integration quality to ensure personal data is incorporated naturally into responses
• Perform side-by-side evaluations and stack-rank model outputs based on usefulness, clarity, and overall experience
• Write clear rationales for rankings referencing specific conversation turns
• Extract and verify debug information to confirm correct use of summaries and data sources
• Maintain data hygiene by deleting evaluation conversations after completion

Requirements

• Experience in data annotation, AI quality evaluation, content moderation, or related roles
• Fluency in Portuguese (reading and writing)
• Willingness to use a primary personal Google account and enable personal data sources for assessment
• Strong analytical thinking and attention to detail
• Experience with creative prompt engineering and personalization concepts
• Ability to provide structured feedback and clear written explanations
• Ability to work independently in a remote setup
• Desktop or laptop with a stable internet connection

Application Process

• Apply/Easy Apply and check email for application form
• Fill Google form
• Assessment Link (After shortlisting – to be completed within 24 hours)