Job Summary
Job Description: Lead DevOps Engineer (Data & AI Platform)
About the Role
At Anglo American, we are advancing a world-class data and AI platform that powers decision-making across the entire mining value chain. Built on Azure and Databricks, our platform enables scalable data products, advanced analytics, and emerging AI capabilities—from digital twins to intelligent automation and natural language querying.
We are looking for a Lead DevOps Engineer to play a critical role in evolving our platform into a highly automated, AI-enabled delivery ecosystem. This role goes beyond traditional DevOps—focusing on platform engineering, AI-assisted development, and intelligent automation of software delivery and operations.
You will lead the design and implementation of modern DevOps practices, integrating AI tools, copilots, and automation frameworks to significantly improve developer productivity, pipeline efficiency, and platform reliability.
Role Overview
As a Lead DevOps Engineer, you will refine and enhance CI/CD processes, integrate new capabilities into release pipelines, and drive automation across the software delivery lifecycle. You will also play a key role in embedding AI-driven practices into DevOps, enabling faster and more reliable delivery of data and AI products.
You will work closely with data engineers, platform teams, and product teams to ensure seamless, scalable, and intelligent deployment processes across the Anglo American Data Platform.
Key Responsibilities
Key Responsibilities
DevOps Strategy and Platform Evolution
- Define and implement a modern DevOps and platform engineering strategy aligned with data and AI platform goals.
- Develop roadmaps that incorporate AI-assisted development, testing, and operations.
- Drive the evolution from traditional DevOps to intelligent, self-service platform capabilities.
- Continuously evaluate emerging technologies (e.g., GenAI, LLMOps, AIOps) and incorporate them where relevant.
AI-Enabled CI/CD and Automation
- Design and optimize CI/CD pipelines using AI-assisted tools (e.g., code generation, test generation, pipeline optimization).
- Integrate AI copilots and automation agents into development and deployment workflows.
- Implement intelligent quality gates (e.g., automated code reviews, anomaly detection in pipelines).
Enable self-healing pipelines and automated failure diagnostics where possible
Automation and Framework Enhancement
- Build scalable automation frameworks leveraging AI, scripting, and infrastructure as code.
- Automate repetitive tasks using AI agents, prompt-based workflows, or orchestration frameworks.
- Enhance DevOps pipelines to support data products and AI/ML workloads (MLOps/LLMOps).
- Standardize reusable templates and pipeline components for platform-wide adoption.
Data & AI Platform Integration
- Analyze and optimize integrations across the Anglo American Data Platform, including:
- Databricks (data processing, workflows, DABs)
- Airflow (orchestration)
- Azure services (compute, storage, identity)
- Power BI / downstream consumption layers
- Support deployment patterns for AI/ML models, feature pipelines, and inference services.
- Enable end-to-end lifecycle management for AI applications (training → deployment → monitoring).
Governance, Security, and Reliability
- Implement governance practices across pipelines, including policy-as-code and automated compliance checks.
- Manage access control and ensure secure DevOps practices across environments.
- Introduce AIOps practices for monitoring, alerting, and incident management.
- Ensure high availability, scalability, and observability of DevOps processes.
Skill Requirements
Technical Expertise
- Strong experience with CI/CD tools (e.g., Azure DevOps, GitHub Actions).
- Expertise in infrastructure as code (Bicep, ARM or similar).
- Proficiency in scripting (PowerShell, Python, Bash).
- Deep understanding of DevOps principles, Git workflows, and release strategies.
- Experience with Azure services and cloud-native architectures.
- Familiarity with data platforms (Databricks, ADF, Airflow, SQL, AAS or equivalent).
AI & Modern DevOps Capabilities
- Hands-on experience or strong familiarity with:
- AI-assisted development tools (e.g., GitHub Copilot, ChatGPT, code assistants)
- MLOps / LLMOps concepts (model deployment, monitoring, versioning)
- AIOps tools for monitoring and incident management
- Understanding of how AI can be applied to:
- Code generation and testing
- Pipeline optimization
- Incident detection and resolution
Experience integrating APIs or services for AI capabilities into workflows is a plus