Job Summary

The Senior Technical Lead will be responsible for leading technical teams and overseeing the implementation of projects related to Snowflake, Azure Data Factory (ADF), and Data Bricks. The role involves ensuring the successful delivery of data solutions and optimizing data pipelines for efficient performance.

Key Responsibilities

1. Lead and manage technical teams, providing guidance and support in the implementation of snowflake, azure data factory (adf), and data bricks projects.
2. Design, develop, and maintain data pipelines using snowflake, azure data factory (adf), and data bricks to support business requirements.
3. Work closely with stakeholders to gather requirements, identify opportunities for data analytics, and propose data driven solutions.
4. Monitor and troubleshoot data pipelines, ensuring data quality, reliability, and performance.
5. Stay updated with the latest trends and technologies in data management and analytics, incorporating best practices into projects.
6. Collaborate with cross functional teams to integrate data solutions with existing systems and applications.
7. Provide technical expertise and mentorship to team members, promoting a culture of learning and innovation.

Skill Requirements

Databricks Certified Architect – Responsibilities & Skills

• Design and implement enterprise-scale solutions on the Databricks Lakehouse Platform.
• Architect end-to-end data pipelines for batch and real-time processing using Apache Spark and PySpark.
• Develop scalable data ingestion, transformation, and data quality frameworks.
• Design and implement Medallion Architecture (Bronze, Silver, Gold) using Delta Lake.
• Build and optimize data warehouses, data marts, and analytical solutions.
• Implement data governance, security, lineage, and access controls using Unity Catalog.
• Develop and support AI/BI dashboards, semantic models, and self-service analytics solutions.
• Configure and optimize Genie Spaces to enable natural language business queries and conversational analytics.
• Design and deploy Generative AI and RAG-based solutions using Databricks Mosaic AI and Vector Search.
• Collaborate with business users to translate requirements into scalable data and AI solutions.
• Optimize Databricks workloads for performance, scalability, reliability, and cost efficiency.
• Lead cloud-native implementations across  Azure environments.
• Define architecture standards, best practices, and reusable design patterns.
• Mentor data engineers, analysts, and architects on Databricks technologies and platform adoption.
• Lead migration initiatives from legacy data warehouses and analytics platforms to Databricks.
• Build and maintain Genie Spaces for business self-service analytics.
• Create semantic models, metrics, and trusted data assets for AI-driven reporting.
• Develop natural language-to-SQL analytics solutions using Databricks Genie.
• Implement RAG solutions using enterprise data and Vector Search.
• Optimize AI/BI dashboards and conversational analytics experiences.
• Troubleshoot Spark performance, query optimization, and workload management.
• Automate data validation, monitoring, and governance controls.
• Support AI use cases using Mosaic AI model serving and inference endpoints.

Technical Skills

• Databricks Lakehouse Platform
• Apache Spark, PySpark, Spark SQL
• Python, SQL
• Delta Lake, Delta Live Tables, Lakeflow
• Unity Catalog
• Databricks AI/BI and Genie
• Mosaic AI, Vector Search, RAG
• Data Modeling (Dimensional & Data Vault)
• Structured Streaming
• Data Quality and Data Governance
• Azure
• Terraform, Git, Azure DevOps, Jenkins
• REST APIs and Data Integration
• Performance Tuning and Cost Optimization

Other Requirements

Databricks Certified Architect – Responsibilities & Skills

• Design and implement enterprise-scale solutions on the Databricks Lakehouse Platform.
• Architect end-to-end data pipelines for batch and real-time processing using Apache Spark and PySpark.
• Develop scalable data ingestion, transformation, and data quality frameworks.
• Design and implement Medallion Architecture (Bronze, Silver, Gold) using Delta Lake.
• Build and optimize data warehouses, data marts, and analytical solutions.
• Implement data governance, security, lineage, and access controls using Unity Catalog.
• Develop and support AI/BI dashboards, semantic models, and self-service analytics solutions.
• Configure and optimize Genie Spaces to enable natural language business queries and conversational analytics.
• Design and deploy Generative AI and RAG-based solutions using Databricks Mosaic AI and Vector Search.
• Collaborate with business users to translate requirements into scalable data and AI solutions.
• Optimize Databricks workloads for performance, scalability, reliability, and cost efficiency.
• Lead cloud-native implementations across  Azure environments.
• Define architecture standards, best practices, and reusable design patterns.
• Mentor data engineers, analysts, and architects on Databricks technologies and platform adoption.
• Lead migration initiatives from legacy data warehouses and analytics platforms to Databricks.
• Build and maintain Genie Spaces for business self-service analytics.
• Create semantic models, metrics, and trusted data assets for AI-driven reporting.
• Develop natural language-to-SQL analytics solutions using Databricks Genie.
• Implement RAG solutions using enterprise data and Vector Search.
• Optimize AI/BI dashboards and conversational analytics experiences.
• Troubleshoot Spark performance, query optimization, and workload management.
• Automate data validation, monitoring, and governance controls.
• Support AI use cases using Mosaic AI model serving and inference endpoints.

Technical Skills

• Databricks Lakehouse Platform
• Apache Spark, PySpark, Spark SQL
• Python, SQL
• Delta Lake, Delta Live Tables, Lakeflow
• Unity Catalog
• Databricks AI/BI and Genie
• Mosaic AI, Vector Search, RAG
• Data Modeling (Dimensional & Data Vault)
• Structured Streaming
• Data Quality and Data Governance
• Azure
• Terraform, Git, Azure DevOps, Jenkins
• REST APIs and Data Integration
• Performance Tuning and Cost Optimization