Freelance Agent Evaluation Engineer (AI Evaluation)

About the role

Mindrift is looking for a freelance Engineer to design and evaluate AI coding agents. You will build realistic developer environments and create challenging tasks that test the limits of current AI models.

Key responsibilities

• Construct virtual company environments with codebases, infrastructure, tickets, documentation, and conversation histories.
• Design intermediate‑state tasks, write clear prompts, and define objective success criteria.
• Develop automated tests that accept all correct solutions while rejecting incorrect ones.
• Iterate on tasks and tests based on QA feedback, analyzing agent failures and refining evaluation criteria.

Required profile

• 5+ years of professional software development experience.
• Strong command of English (minimum B2 level).
• Hands‑on experience with Python (FastAPI) and JavaScript/TypeScript (React).
• Familiarity with Docker, PostgreSQL, Kafka, and Redis.
• Proven ability to write functional and integration tests.

Required skills

• Python
• FastAPI
• JavaScript
• TypeScript
• React
• Docker
• PostgreSQL
• Kafka
• Redis
• Functional testing
• Integration testing

What we offer

• Up to $40 USD per hour, based on experience and pace.
• Fully remote, flexible schedule – you choose when and how to work.
• Project‑based engagement with clear deliverables and deadlines.