Senior Data Engineer

About the role

You will join the Data Platform team of a global mining organization, designing and delivering a modern Azure-native data infrastructure. The role is hands-on, focusing on building and maintaining robust pipelines that feed analytics across finance, operations and supply chain.

Key responsibilities

• Gather and analyse business requirements to define integration priorities.
• Design, build and maintain scalable data pipelines with Azure Data Factory.
• Develop and optimise transformation logic in Azure Synapse Analytics (dedicated and serverless SQL pools, Synapse Pipelines).
• Create and manage Databricks notebooks and jobs using PySpark / Spark SQL for large-scale processing and feature engineering.
• Implement reliable API extraction from REST and SOAP services, handling pagination, OAuth 2.0, API keys, rate limiting and error recovery.
• Design and maintain a lakehouse architecture (raw, curated, serving layers) on Azure Data Lake Storage Gen2.
• Ensure data quality through validation frameworks, monitoring, alerting and lineage documentation.
• Collaborate with analytics engineers and data consumers to model data for downstream use.
• Apply CI/CD, infrastructure-as-code (Azure DevOps, Terraform/Bicep) and manage environments.
• Participate in data governance, metadata management and access control.

Required profile

• 4+ years of experience in data engineering or a closely related software engineering role.
• Strong analytical and communication skills.
• Ability to work autonomously in a distributed team.

Required skills

• Azure Data Factory – pipelines, triggers, linked services, data flows.
• Azure Synapse Analytics – SQL pools, Synapse Pipelines, ADLS Gen2 integration.
• Azure Databricks – PySpark, Delta Lake, job clusters (Unity Catalog a plus).
• API extraction – REST, SOAP, OAuth 2.0, API keys, pagination, error handling.
• SQL (T‑SQL or Spark SQL) and Python for data manipulation.
• Data modelling – star/snowflake, data vault, medallion architecture.
• Azure DevOps, Git, CI/CD pipelines, Terraform/Bicep.
• Data security – RBAC, encryption in Azure.