Job Summary
The Azure Databricks Specialist is responsible for designing, building, and optimizing scalable data platforms and analytics solutions on Microsoft Azure using Azure Databricks. The role involves architecting end-to-end data pipelines, ensuring data governance, and enabling advanced analytics and AI solutions.
Key Responsibilities
Key Responsibilities:
Design and implement scalable data architectures using Azure services
Build and optimize ETL/ELT pipelines using Azure Databricks and Apache Spark
Architect data lake and data warehouse solutions using Azure Data Lake and Azure Synapse Analytics
Define data modeling strategies (batch & real-time processing)
Ensure data security, governance, and compliance standards
Collaborate with stakeholders to understand business requirements and translate them into technical solutions
Optimize performance and cost efficiency of data workloads
Implement CI/CD pipelines for data engineering workflows
Mentor and guide data engineers and development teams
Integrate advanced analytics, AI, and ML solutions when required
Skill Requirements
-
Required Skills & Qualifications:
Strong experience with Azure Databricks and Spark (PySpark/Scala)
Hands-on experience with Azure services (ADF, ADLS, Synapse, Event Hub)
Expertise in big data architecture and distributed systems
Strong knowledge of SQL, Python, and data engineering concepts
Experience with data modeling techniques (star schema, dimensional modeling)
Understanding of real-time streaming (Kafka/Event Hub)
Knowledge of DevOps and CI/CD practices
Other Requirements
1.Relevant certifications in Azure Data Factory (ADF), Azure Databricks, SQL, Python are a plus.