Senior Data Lead - Azure Data Factory (ADF), Databricks
India
Job Description
Senior Data Lead - Azure Data Factory (ADF), Databricks
Pune, Maharashtra

Job Summary

Job Description: Snowflake Engineer (8+ Years Experience)

Role Overview

We are seeking a highly skilled Snowflake Engineer with 8+ years of overall IT experience, including 3+ years of strong hands-on experience in Snowflake-based data engineering. The ideal candidate brings deep expertise in modern data pipeline engineering, metadata-driven ingestion, and AI-enabled data processing, along with strong consultative and stakeholder engagement skills.

Key Responsibilities

Key Responsibilities

1. Data Pipeline Engineering

  • Design and develop scalable, high-performance data pipelines using SnowSQL, Snowpark, and Snowflake native capabilities
  • Build and optimize ELT/ETL frameworks supporting both batch and real-time workloads
  • Implement robust ingestion pipelines from diverse enterprise systems

2. Metadata-Driven Ingestion

  • Design and implement metadata-driven ingestion frameworks for scalable onboarding of datasets
  • Handle ingestion across multiple formats including XML, CSV, JSON, and Parquet
  • Automate ingestion, validation, and transformation processes

3. Streaming & Kafka Integration

  • Develop real-time data ingestion pipelines using Kafka
  • Integrate streaming data into Snowflake ensuring reliability and scalability
  • Implement monitoring and failure handling for streaming workflows

4. AI/GenAI Integration (Cortex AI)

  • Leverage Snowflake Cortex AI capabilities to enable AI-driven data transformations, enrichment, and insights generation
  • Collaborate with business and analytics teams to enable AI-powered data consumption use cases
  • Contribute to building intelligent, automated data pipelines and semantic layers

5. Data Modeling & Transformation

  • Develop scalable data models supporting analytics and reporting layers
  • Build transformations using Snowpark, SQL, and Snowflake native features
  • Optimize query performance and data access patterns

6. Data Delivery & Reporting Enablement

  • Deliver analytics-ready datasets in predefined formats for reporting tools (Power BI, Tableau, etc.)
  • Work closely with business stakeholders to ensure data usability and alignment with reporting requirements

7. Performance Optimization & Governance

  • Implement best practices for performance tuning, clustering, and cost optimization
  • Ensure data quality, governance, security, and compliance standards
  • Monitor, troubleshoot, and continuously improve pipelines

8. Stakeholder Engagement & Technical Consulting

  • Engage with stakeholders to understand business needs and translate them into data solutions
  • Provide technical consultation on data architecture, ingestion strategies, and Snowflake capabilities
  • Act as a trusted advisor in data engineering and platform decisions

Skill Requirements

Required Skills & Experience 8+ years of overall IT experience with 3+ years in Snowflake engineering Strong expertise in: Snowflake (SnowSQL, Snowpark, Snowpipe, Streams/Tasks) Pipeline design and data engineering frameworks Metadata-driven ingestion architectures Hands-on experience with: Kafka-based ingestion (mandatory) File formats: Parquet (must), CSV, XML, JSON Experience delivering data in predefined formats for reporting and analytics Strong proficiency in SQL and data modeling Hands-on experience or exposure to Snowflake Cortex AI (required) Excellent communication, presentation, and stakeholder management skills Strong technical consulting and problem-solving mindset Good-to-Have Skills Experience with DBT (Data Build Tool) Knowledge of PL/SQL or procedural SQL programming Exposure to cloud platforms (AWS/Azure/GCP) Familiarity with CI/CD pipelines and version control systems Skill Matrix Must-Have Skills Skill Area Expected Competency Relevance to Role Snowflake Engineering Strong hands-on experience with Snowflake, SnowSQL, Snowpark, Snowpipe, Streams, and Tasks. Core capability required to design, build, and optimize Snowflake-based data engineering solutions. Data Pipeline Development Ability to design scalable batch and real-time ELT/ETL pipelines using Snowflake native capabilities. Required for building enterprise-grade ingestion and transformation frameworks. Metadata-Driven Ingestion Experience designing metadata-driven ingestion frameworks for onboarding multiple datasets and source systems. Critical for scalable, reusable, and automated data onboarding. Kafka Integration Hands-on experience with Kafka-based data ingestion and streaming pipeline integration. Mandatory for real-time data ingestion and event-driven data processing. File Format Handling Strong experience working with Parquet, CSV, JSON, and XML formats. Required to support structured, semi-structured, and enterprise file-based ingestion scenarios. SQL and Data Modeling Strong SQL proficiency with experience in data modeling, transformations, query optimization, and analytics-ready datasets. Essential for developing reliable reporting, analytics

Other Requirements

Should-Have Skills

Skill Area

Expected Competency

Relevance to Role

DBT

Exposure to DBT for modular transformations, model management, and analytics engineering practices.

Useful for improving transformation maintainability and governance.

PL/SQL or Procedural SQL

Knowledge of PL/SQL or procedural SQL concepts for complex data logic and stored procedure migration scenarios.

Helpful when integrating or modernizing legacy database workloads into Snowflake.

Cloud Platforms

Exposure to AWS, Azure, or GCP services relevant to data engineering, storage, security, and integration.

Supports cloud-native deployment and integration of Snowflake data solutions.

CI/CD and Version Control

Familiarity with Git, GitLab, Bitbucket, Jenkins, or similar tools for code versioning and deployment automation.

Supports reliable release management and DevOps-driven data engineering delivery.

BI and Reporting Tools

Exposure to Power BI, Tableau, Qlik, or similar reporting platforms.

Helps align data delivery with downstream analytics and business reporting needs.

Data Governance and Security

Understanding of data quality, access control, masking, lineage, and compliance practices.

Supports secure, compliant, and trusted enterprise data platforms.

Monitoring and Troubleshooting

Experience with monitoring pipeline failures, data quality exceptions, and operational alerts.

Useful for maintaining production stability and improving operational resilience.

 

Information at a Glance

Why HCLTech?

At HCLTech, you'll supercharge your potential. You'll find your career. And you'll find your spark. All at a place that knows that helping its customers stay on top starts by putting its people first.

HCLTech is a global technology company, home to more than 223,000 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending June 2026 totaled $14.8 billion.