Semi Senior QA Engineer – AI Applications
Rootstrap · Département Montevideo
Job description
About the role
Rootstrap is building a pipeline for future AI‑focused QA talent. The Semi Senior QA Engineer will test non‑deterministic AI systems such as large language models, AI agents, and Retrieval‑Augmented Generation (RAG) applications, while contributing to evaluation frameworks that ensure quality and detect regressions.
Key responsibilities
- Create comprehensive test plans, test cases and automation scripts (Playwright‑based).
- Review client requirements and technical specs with developers, providing timely feedback.
- Estimate, prioritize, and coordinate testing activities, tracking bugs and quality metrics.
- Design, develop and execute regression tests, including automation integration.
- Build and maintain AI evaluation suites to validate model behavior and detect hallucinations, grounding issues, etc.
- Analyze AI outputs, identify failure patterns, and collaborate with engineers on preventive solutions.
Required profile
- 3+ years of experience as a QA Engineer with mobile testing expertise.
- Advanced English proficiency and a background in Software Engineering or Computer Science.
- Hands‑on experience testing AI‑powered applications and evaluating non‑deterministic systems.
Required skills
- Playwright test automation.
- Mobile testing tools and frameworks.
- AI tools such as Claude for test case generation and bug reporting.
- Evaluation frameworks (LangSmith, Braintrust, Arize Phoenix, Promptfoo, OpenEvals) and benchmark dataset creation.
- CI/CD pipeline integration for automated testing.
What we offer
- Remote work flexibility with optional office presence in Montevideo, Buenos Aires, or Medellín.
- Professional development through certifications, mentorship, and learning bonuses.
- Well‑being benefits including gym access, psychological counseling, and weekly lunch reimbursements.
- On‑boarding kit, cutting‑edge tools, and a workation program.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in Uruguay.
Salary: Senior AI Engineer Based on 5 job offers in UruguayApply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 1 month ago
Expires 4 hours from now
54 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Rootstrap
Département Montevideo
Related job offers
-
HubSpot Administrator (B2C Marketing Automation)
Plan A Technologies Département Montevideo -
Desarrollador Android (Kotlin) - Trabajo Remoto
BairesDev Département Montevideo -
Analista de Soporte Técnico
Randstad Uruguay Département Montevideo -
Freelance Forward Deployed AI Engineer
Toloka -
LLM Engineer (Remote, Punta del Este)
Blacksmith Agency