Senior Apache Spark Technical Lead - Scala, Python
India
Job Description
Senior Apache Spark Technical Lead - Scala, Python
Pune, Maharashtra

Job Summary

6+ years of experience in production support, business operations support, or data platform operations.
Strong hands-on experience with Apache NiFi (flow design, error handling, queues, templates, and real-time monitoring).
Proficiency in Apache Spark (batch and streaming), with knowledge of tuning, job monitoring, and debugging in production.
Solid understanding of ETL processes, data ingestion, and transformation.
Experience supporting data platforms in a Linux-based production environment.
Proficient in writing and maintaining scripts in Bash, Python, or Shell for automation and monitoring.
Strong knowledge of log analysis, monitoring tools (e.g., ELK stack, Splunk, Grafana), and incident management systems (e.g., xMatters, BMC Remedy).
Excellent troubleshooting skills and the ability to work under pressure during production incidents.

Key Responsibilities

6+ years of experience in production support, business operations support, or data platform operations.
Strong hands-on experience with Apache NiFi (flow design, error handling, queues, templates, and real-time monitoring).
Proficiency in Apache Spark (batch and streaming), with knowledge of tuning, job monitoring, and debugging in production.
Solid understanding of ETL processes, data ingestion, and transformation.
Experience supporting data platforms in a Linux-based production environment.
Proficient in writing and maintaining scripts in Bash, Python, or Shell for automation and monitoring.
Strong knowledge of log analysis, monitoring tools (e.g., ELK stack, Splunk, Grafana), and incident management systems (e.g., xMatters, BMC Remedy).
Excellent troubleshooting skills and the ability to work under pressure during production incidents.

Skill Requirements

6+ years of experience in production support, business operations support, or data platform operations.
Strong hands-on experience with Apache NiFi (flow design, error handling, queues, templates, and real-time monitoring).
Proficiency in Apache Spark (batch and streaming), with knowledge of tuning, job monitoring, and debugging in production.
Solid understanding of ETL processes, data ingestion, and transformation.
Experience supporting data platforms in a Linux-based production environment.
Proficient in writing and maintaining scripts in Bash, Python, or Shell for automation and monitoring.
Strong knowledge of log analysis, monitoring tools (e.g., ELK stack, Splunk, Grafana), and incident management systems (e.g., xMatters, BMC Remedy).
Excellent troubleshooting skills and the ability to work under pressure during production incidents.

Other Requirements

null
Information at a Glance

Why HCLTech?

At HCLTech, you'll supercharge your potential. You'll find your career. And you'll find your spark. All at a place that knows that helping its customers stay on top starts by putting its people first.

HCLTech is a global technology company, home to more than 227,000 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending March 2026 totaled $14.7 billion.