SME - Python
India
Job Description
SME - Python
Noida, Uttar Pradesh

Job Summary

Job Title: Senior Data Engineer (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. ey Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues. ________________________________________ Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills. 

Key Responsibilities

Job Title: Senior Data Engineer (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. ________________________________________ Key Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues. ________________________________________ Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills.

Skill Requirements

 Job Title: Senior Data Engineer (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. ________________________________________ Key Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues.Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills.

Other Requirements

Job Title: Senior Data Engineer (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. Key Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues. ________________________________________ Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills. 

Information at a Glance

Why HCLTech?

At HCLTech, you'll supercharge your potential. You'll find your career. And you'll find your spark. All at a place that knows that helping its customers stay on top starts by putting its people first.

HCLTech is a global technology company, home to more than 223,000 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending June 2026 totaled $14.8 billion.