Senior Technical Lead
Australia
Apply now
Job Description
Senior Technical Lead
Sydney, New South Wales

Job Summary

We are seeking a seasoned Databricks Tech Lead to design, build, and scale our next-generation data ingestion and processing framework. In this role, you will lead a team of data engineers, champion best practices in software development, and build reusable, highly scalable pipelines to ingest and transform data from diverse enterprise sources into our Lakehouse platform.

Key Responsibilities

  • Framework Development: Design and build metadata-driven, generic data ingestion frameworks (batch, streaming, and CDC) to automate the onboarding of new data sources.
  • Architecture & Design: Define architectural standards for the Medallion architecture (Bronze, Silver, Gold layers) utilizing Delta Lake.
  • Technical Leadership: Lead technical design discussions, mentor junior/mid-level engineers, and conduct rigorous code reviews.
  • Pipeline Optimization: Optimize Spark jobs for performance, scalability, and cost reduction across the Databricks platform.
  • CI/CD & DevOps: Implement and mature CI/CD pipelines (Git, automated testing) for data platforms and enforce DataOps best practices.
  • Governance & Security: Integrate ingestion pipelines with Unity Catalog for enterprise-grade data governance and lineage tracking.

Skill Requirements

Other Requirements

  • Experience: 8+ years in Data Engineering, with at least 3–4 years of hands-on technical leadership experience leading engineering pods.
  • Core Databricks & Spark: Advanced proficiency in PySpark and Databricks SQL.
  • Ingestion Expertise: Proven experience building custom or highly configurable data ingestion frameworks for structured and unstructured data.
  • Cloud Platforms: Strong experience working with AWS, Azure, or GCP cloud-native services (e.g., S3, ADLS).
  • Engineering Standards: Deep understanding of data modeling (star/snowflake schemas), SCD (Slowly Changing Dimensions), and software engineering principles in a data context.
  • Soft Skills: Excellent stakeholder management and the ability to translate complex technical architectures into business value.
Apply now
Information at a Glance

Why HCLTech?

At HCLTech, you'll supercharge your potential. You'll find your career. And you'll find your spark. All at a place that knows that helping its customers stay on top starts by putting its people first.

HCLTech is a global technology company, home to more than 223,000 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending June 2026 totaled $14.8 billion.