Job Summary

We are seeking an experienced CI/CD Engineer or Lead to design, automate, and govern delivery pipelines across the Azure ecosystem (Azure DevOps, Databricks, ADF/Synapse) and adjacent CI/CD platforms. While the final tool choice may vary, the candidate must have significant hands-on experience with Jenkins and strong DevOps fundamentals. The role includes in-sprint testing enablement, platform governance (monitoring, administration, configuration), and cost-effective technology selection. Excellent communication and documentation skills are essential. 

Key Responsibilities

• Pipeline Architecture & Delivery: Design and maintain end-to-end CI/CD pipelines using Azure DevOps, Jenkins, GitHub Actions, or GitLab CI. Implement linting and code review checks, integrate static analysis tools (SonarQube/CodeQL), architect multi-stage workflows, build reusable templates for Databricks, and manage artifact repositories. 
• Azure Platform Governance & Operations: Monitor, administer, and configure Azure services (Key Vault, Storage/ADLS, ADF/Synapse, Databricks, AKS). Apply RBAC, secrets management, observability (Azure Monitor, Log Analytics), and cost governance. 
• Source Control & ALM: Define branching strategy (GitFlow/Trunk), enforce PR policies in Bitbucket, and integrate Jira workflows with CI/CD. 
• Automated & In-Sprint Testing: Embed unit, integration, SIT, and E2E/regression tests into pipelines for in-sprint execution. Collaborate with QA to align automated test coverage with sprint goals. 
• DevSecOps & Policy as Code: Integrate SAST/DAST, dependency scanning, SBOM generation, and enforce quality/security gates. Apply policy-as-code (OPA/Conftest, Azure Policy). 
• Infrastructure as Code (IaC): Provision and promote infrastructure using Terraform (preferred) and/or Bicep; manage remote state and modules. Standardize container builds and AKS deployments. 
• Data Storage Layer Management & Governance: Govern data storage across Unity Catalog (Databricks), Azure SQL / SQL Server, and ADLS. Implement data lineage, optimize storage tiers, and coordinate schema migrations. 
• Communication & Collaboration: Communicate effectively with cross-functional teams including QA, Development, Product Management, and Architecture. Produce clear documentation, runbooks, and present status updates to stakeholders. 
• Leadership (for Lead level): Define CI/CD standards, mentor engineers, lead design reviews, and drive continuous improvement. 

Skill Requirements

• Significant hands-on experience with Jenkins (declarative pipelines, shared libraries). 
• Hands-on with Azure services (Key Vault, Storage, ADF, Databricks, AKS). 
• Expertise in CI/CD best practices across multiple tools (Azure DevOps, GitHub Actions, GitLab CI). 
• Solid understanding of in-sprint testing and test automation frameworks. 
• Experience with linting, static analysis, and enforcing code quality gates. 
• Scripting: Python, Bash, YAML proficiency. 
• Familiarity with IaC using Terraform (preferred) and containerization (Docker/Kubernetes). 
• Strong communication and documentation skills. 

• Databricks asset bundling/dbx, MLflow model promotion; Unity Catalog administration. 
• Data quality frameworks (e.g., Great Expectations). 
• GitOps (Argo CD/Flux), Helm, Kustomize. 
• SRE practices; DORA metrics instrumentation. 

• CI/CD: GitHub Actions, GitLab CI, Jenkins (self-hosted), CircleCI. 
• Code Quality: SonarQube/CodeQL, pre-commit, Black/Flake8, ESLint. 
• Observability: Azure Monitor, Log Analytics; Cost Management tools like Azure Advisor. 
• IaC: Terraform (preferred), Bicep; Secrets via Azure Key Vault. 
• Data Storage: Unity Catalog, Azure SQL / SQL Server, ADLS Gen2; Catalog alternatives like DataHub or Azure Purview. 

• Lead time for changes, deployment frequency, change failure rate, MTTR (DORA). 
• % of pipelines with linting, static analysis, test/security gates. 
• % infra under Terraform; environment drift incidents; secrets coverage. 
• Platform cost adherence: budget vs. actual; ingestion/storage optimization achieved. 
• Data governance adoption: catalog completeness, Unity Catalog policies applied. 

Other Requirements