This job is no longer available
This job expired on 22/08/2026. It no longer accepts applications.
Freelance AI Agent Evaluation Engineer
Mindrift
Job description
About the role
Mindrift is looking for a freelance engineer to help build a high‑quality dataset for evaluating AI coding agents. You will design realistic developer environments and craft challenging tasks that test the limits of current models.
Key responsibilities
- Construct virtual company environments with codebases, infrastructure, tickets, documentation, and conversation histories.
- Design tasks from intermediate states, write clear prompts, and define objective success criteria.
- Develop functional and integration tests that accept all correct solutions while rejecting incorrect ones.
- Iterate on tasks and tests based on QA feedback, analysing agent failures and refining evaluation criteria.
Required profile
- 5+ years of professional software development experience.
- Strong command of English (minimum B2 level).
- Hands‑on experience writing functional and integration tests.
Required skills
- Python (FastAPI)
- JavaScript / TypeScript (React)
- Docker
- PostgreSQL
- Kafka
- Redis
What we offer
- Project‑based freelance contract with flexible hours.
- Compensation up to $40 USD per hour, based on experience and pace.
- Remote work – you choose when and how to complete the tasks.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in Uruguay.
Salaries by job title
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Mindrift