Job Summary
We are looking for an experienced Data Engineer – Azure Data Platform to design, develop, and maintain scalable data solutions across cloud-based data ecosystems. The ideal candidate should have strong expertise in Azure ELT pipelines, Data Modeling, Data Integration & Ingestion, Data Processing, and modern data engineering tools. The role involves building enterprise-grade data platforms, optimizing data workflows, implementing CI/CD processes, and supporting advanced analytics initiatives.
Key Responsibilities
- Design and develop scalable ELT/ETL pipelines using Azure cloud technologies.
- Build and manage data ingestion frameworks for structured, semi-structured, and streaming data.
- Develop enterprise data models to support reporting, analytics, and business intelligence requirements.
- Implement data transformations and processing using Databricks, SQL, and cloud-native services.
- Create and optimize data workflows using Azure Data Factory, Airflow, and other orchestration tools.
- Integrate data from multiple sources into Azure data platforms.
- Design and maintain data lakes, warehouses, and analytical data stores.
- Develop CI/CD pipelines using GitHub Actions and Azure DevOps.
- Monitor, troubleshoot, and optimize data processing performance and reliability.
- Enforce data quality, governance, security, and compliance standards.
- Collaborate with architects, business stakeholders, analysts, and development teams to deliver data-driven solutions.
Skill Requirements
Data Engineering & Architecture
- Azure ELT/ETL
- Data Modeling
- Data Integration & Ingestion
- Data Manipulation and Processing
- Data Warehousing Concepts
- Data Lake Architecture
Azure & Cloud Technologies
- Azure Data Factory (ADF)
- Azure Databricks
- Azure Synapse Analytics
- Azure SQL Database
- Azure Stream Analytics
Programming & Querying
- SQL
- Python
- PySpark
- Data Transformation Frameworks
Data Orchestration & Integration
- Apache Airflow
- AWS Glue
- Kinesis
- Redshift
- Stream Processing
DevOps & CI/CD
- GitHub
- GitHub Actions
- Azure DevOps
- CI/CD Automation
Quality & Testing
- SonarQube
- PyTest
- Code Quality Management
- Automated Testing
Other Requirements
- 5+ years of experience in Data Engineering.
- Strong understanding of cloud-native data platforms.
- Experience working with large-scale data migration and modernization projects.
- Knowledge of Azure Data Lake, Delta Lake, and Lakehouse architectures.
- Experience in Agile/Scrum delivery environments.
- Azure Data Engineer Associate certification is preferred.