Senior Apache Spark Technical Lead - Scala, Python
India
Job Description
Senior Apache Spark Technical Lead - Scala, Python
Noida, Uttar Pradesh

Job Summary

We’re looking for a hands-on Scuba Developer cum Support Consultant to design, build, and support solutions on the Scuba platform. The role blends new feature development, data/modeling work, and support, ensuring platform stability, performance, and ongoing enhancements for business users and engineering stakeholders.

To collaborate with product managers, platform engineers, data engineers, QA, and business SMEs, contributing across the SDLC—from requirements and design through deployment, monitoring, and incident/problem management.

Key Responsibilities

Development

  • handling design and development, including data ingestion, quality, integration, analysis, modeling, and ETL
  • Design, develop, and optimize Scuba dashboards/queries/models/pipelines (customize to the exact components your Scuba platform supports).
  • Implement data integrations (batch/streaming), transformations, and enrichment aligned with data quality, lineage, and governance standards.
  • Build reusable components, utilities, and automations (e.g., query templates, data loaders, CI/CD jobs).
  • Participate in solution design reviews, code reviews, and performance tuning; ensure secure and scalable implementations.
  • Author technical design documents, runbooks, and user guides; contribute to internal knowledge base.

Support

  • Provide support and solution for incidents, service requests, and problem records across Scuba environments (Dev/Test/Prod).
  • Troubleshoot query performance, data freshness, access/permissions, and platform reliability issues; drive root cause analysis (RCA) and permanent fixes.
  • Manage releases and change requests following ITSM and Change Advisory Board (CAB) processes; support UAT and production cutovers.
  • Monitor SLAs/SLOs, capacity, and costs; implement alerts, health checks, and observability dashboards.
  • Partner with stakeholders to prioritize backlog, communicate status/ETAs, and escalate risks proactively.

Skill Requirements

  •  
  • Strong proficiency in apache spark, scala, and python 
  • Hands-on experience with Scuba (replace with: Scuba Analytics / your internal Scuba platform) for building dashboards, complex queries, and data models.
  • Strong SQL (window functions, CTEs, performance tuning) and comfort with large datasets.
  • Scripting for automation (e.g., Python or Shell) and basic CI/CD (Git, pipelines).
  • Understanding of data pipelines/ETL/ELT, schema design, and data quality patterns.
  • Familiarity with monitoring & logging (e.g., Grafana/Prometheus/CloudWatch, Kibana) and alerting.
  • Solid grasp of ITSM (Incident/Problem/Change), SLAs, and on-call best practices.
  • Security & compliance awareness (access controls/roles, PII handling, audit readiness).

 

Other Requirements

1.Relevant certifications in apache spark, scala, or python are a plus

Information at a Glance

Why HCLTech?

At HCLTech, you'll supercharge your potential. You'll find your career. And you'll find your spark. All at a place that knows that helping its customers stay on top starts by putting its people first.

HCLTech is a global technology company, home to more than 226,300 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2025 totaled $14.5 billion.

23 Benefits At HCLTech, we believe in empowering our employees with comprehensive benefits that support their professional growth and enhance their well-being. When you sign up for a career with us, you gain access to: https://rmkcdn.successfactors.com/147eb21f/a701dca9-f32d-4fc9-9447-6.svg Industry-benchmarked compensation https://rmkcdn.successfactors.com/147eb21f/b0c54381-ddcc-4a33-9b35-9.svg Best-in-class healthcare benefits https://rmkcdn.successfactors.com/147eb21f/b73027be-7aae-4d36-a090-4.svg Personal time off https://rmkcdn.successfactors.com/147eb21f/d5b4fdfd-2e99-4e26-9878-9.svg Maternity and paternity benefits https://rmkcdn.successfactors.com/147eb21f/3d42b0fc-4652-435a-9ece-c.svg Access to skills / higher education programs/resources https://rmkcdn.successfactors.com/147eb21f/aeddeaf2-9e25-4584-ad11-d.svg Discounts on products and services via Benefit Box https://rmkcdn.successfactors.com/147eb21f/a9609a3b-2700-4b3c-9d90-a.svg Participate in CSR programs and live life with a purpose https://rmkcdn.successfactors.com/147eb21f/c6e33851-710f-4634-bd69-f.svg Opportunities to grow and advance your career Note: The benefits listed above vary depending on the nature of your employment and the country where you work. Some benefits may be available in some countries but not in all.