Software Engineer - Evaluation Author

About the role

Mercor is looking for a seasoned software engineer to author evaluation tasks that improve AI model performance. You will work remotely, creating coding challenges, rubrics, and automated verifiers that help assess AI-generated content.

Key responsibilities

• Write non‑trivial coding tasks with reference (golden) solutions and automated verification scripts.
• Design rubrics and grade agent trajectories and model outputs.
• Iteratively improve task and rubric quality through structured reviews.
• Assess the accuracy and depth of AI‑generated content to strengthen reasoning and rigor.
• Work independently and asynchronously, meeting deadlines while enhancing model performance.

Required profile

• 5+ years of software engineering experience at a product‑focused organization (big tech or venture‑backed startup).
• Demonstrated discipline in code quality, systems design, debugging, and testing.
• Excellent written communication skills for creating clear instructions.

Required skills

• Software engineering fundamentals (code quality, systems design, debugging, testing).

What we offer

• Contract engagement with flexible hours (30+ hours/week).
• Competitive hourly rate ranging from $35 to $120.
• Remote work environment.