Machine Learning Engineer – ML Evaluation & Experiment Design

About the role

Anyone AI is seeking an experienced Machine Learning Engineer to lead the review and evaluation of ML challenges used in AI model training and assessment. You will analyze experiments, datasets, metrics, and pipelines to ensure challenges are technically sound, reproducible, and truly require strong ML reasoning.

Key responsibilities

• Review ML challenges for sound experiment design, model selection, and difficulty calibration.
• Evaluate datasets for meaningful signals, data quality, and potential shortcuts such as leakage or label noise.
• Assess evaluation metrics, improvement thresholds, and detect metric gaming or evaluation flaws.
• Verify reproducibility across the full data → model → evaluation pipeline.
• Provide clear written feedback and recommendations for improving or recalibrating challenges.

Required profile

• 3+ years of hands‑on applied machine‑learning experience.
• Strong background in experiment design, model selection, hyperparameter tuning, and model evaluation.
• Deep understanding of train/validation/test splits and ability to spot data leakage, distribution shift, spurious correlations, and feature leakage.
• Experience assessing statistical significance of performance gains.
• Proficiency debugging ML workloads on both CPU and GPU environments.

Required skills

• Machine Learning experiment design
• Model selection and hyperparameter tuning
• Model evaluation and metric selection
• Data preprocessing and validation
• Train/validation/test splitting
• Statistical testing, confidence intervals, effect size analysis
• Synthetic data generation and validation
• CPU/GPU debugging of ML pipelines