Senior Technical Lead
Senior Technical Lead
Job Summary
|
Capabilities covered |
Data Contracts; Data Transformation; Data Validation; Data Observability Module; Ingestion Standardization; Master Data Management (data side); Data Toolkit Design; Standardised data platform architecture; Common CLI/SDK; Synthetic Data Generator |
|
Location |
Prudential VCC Bangalore (preferred) |
Role summary
Complete remaining engineering and LBU rollout of the UDP data toolkit and standardised ingestion, working on Azure Databricks and GCP BigQuery.
Key Responsibilities
Key responsibilities
- Build and roll out ingestion standardisation patterns and connectors (file, API, database) using procured ingestion tools.
- Implement and roll out the Data Validation, Data Transformation, Data Contracts, Platform/Data Observability modules and the common CLI/SDK, including versioning and support.
- Integrate the Synthetic Data Generator on UDP and support PACS deployment.
- Support MDM integration with UDP and downstream applications.
- Write Python/SQL and Spark/BigQuery code with unit and integration tests; use Git workflows and CI/CD.
- Execute discovery, prerequisites, deployment, acceptance and hypercare for each LBU rollout using playbooks.
Document runbooks, known issues and release records in Prudential-approved repositories.
Skill Requirements
Must-have skills and experience
- 3–5 years of data engineering experience with Azure Databricks (Spark, Delta) and/or GCP BigQuery.
- Strong Python and SQL; experience with dbt and Airflow (or Data Factory / Composer).
- CI/CD with GitHub/Azure DevOps and basic Terraform.
- Understanding of data quality, schema evolution and data contract concepts.
Good to have
- Experience with Kafka, Fivetran, Informatica or Reltio integrations.
- Experience building reusable SDKs or CLIs (Python packaging, semantic versioning).
- Exposure to data observability tooling.
Certifications (within 90 days of assignment)
- Databricks Data Engineer Associate/Professional
- GCP Professional Data Engineer or Azure DP-203 equivalent
- HashiCorp Terraform Associate (preferred)