Job Summary

We are looking for a Lead Data Engineer with 10+ years of experience who can operate as both a senior technical contributor and a Data Engineering Chapter Lead. The role requires strong data platform engineering experience, cloud-native data architecture knowledge, hands-on capability across modern data stacks, and the ability to mentor and govern data engineering practices across multiple squads.

The Chapter Lead will define data engineering standards, build reusable patterns, mentor engineers, support hiring and onboarding, review solution designs, and ensure consistent engineering maturity across data teams. The role will also champion AI-assisted data engineering practices across the SDLC, enabling engineers to use generative AI and coding assistants responsibly to improve delivery speed, quality, documentation, testing, and operational effectiveness.

Key Responsibilities

. Data Engineering Leadership

• Lead the design and development of scalable data pipelines, data products, lakehouse solutions, and analytics-ready datasets.
• Provide hands-on technical leadership across Python, Snowflake, Databricks, AWS / Azure, and related data engineering services.
• Define engineering standards for data ingestion, transformation, orchestration, data quality, metadata, lineage, security, and observability.
• Review data architecture, pipeline design, code quality, data models, performance, cost optimization, and production readiness.
• Drive adoption of reusable frameworks, CI/CD for data pipelines, automated testing, data quality checks, and cloud-native engineering practices.

2. Chapter Leadership & Capability Building

• Act as the capability lead for Data Engineering resources across squads and establish chapter-wide engineering standards and guardrails.
• Create and maintain a Data Engineering skill matrix, learning roadmap, proficiency expectations, assessment model, onboarding plan, and interview framework.
• Mentor engineers across junior, mid, senior, and lead levels on engineering craftsmanship, cloud platforms, data architecture, quality, and delivery discipline.
• Run chapter forums, design review boards, technical deep dives, code review clinics, learning circles, and knowledge-sharing sessions.
• Provide technical input into hiring, role-fitment, performance feedback, succession planning, and capability improvement plans.

3. Cloud Data Platform Delivery

• Design and implement cloud-native data solutions on AWS and/or Azure.
• Build and optimize data pipelines using Databricks, Spark, Python, and orchestration tools.
• Design, model, and optimize analytical workloads in Snowflake.
• Enable analytics consumption through Power BI and/or Tableau and ensure datasets are semantic-layer ready.
• Implement governance, access control, data quality, metadata management, lineage, and compliance-aware engineering practices.

4. AI-Assisted Data Engineering Enablement

• Promote practical use of AI-assisted engineering tools such as GitHub Copilot, coding agents, and GenAI assistants across data engineering workflows.
• Enable AI-assisted Python and SQL generation, pipeline development, test generation, code review, refactoring, documentation, and impact analysis.
• Use AI-assisted techniques to accelerate data quality rule creation, troubleshooting, root-cause analysis, performance optimization, and operational support.
• Establish secure prompting, human-review, traceability, intellectual property, data privacy, and quality guardrails for AI-assisted engineering.
• Mentor chapter members on effective prompt engineering and responsible adoption of AI tools, and track measurable improvements in engineering productivity and quality.

5. DataOps, Reliability & Governance

• Drive CI/CD, automated testing, version control, deployment automation, and environment promotion for data platforms.
• Ensure strong observability through monitoring, logging, alerting, SLA tracking, lineage, and incident diagnostics.
• Lead root-cause analysis, performance tuning, production support, and continuous improvement for operational stability.
• Partner with architecture, DevOps, cybersecurity, analytics, product, and platform teams to ensure solutions are scalable, secure, resilient, and maintainable.

Skill Requirements

• Python for data engineering, automation, reusable frameworks, pipeline development, and data processing.
• AWS and/or Azure data services and cloud-native data architecture.
• Snowflake data warehouse design, SQL optimization, performance tuning, security, and workload optimization.
• Databricks / Spark for large-scale processing, Delta Lake, lakehouse implementation, and distributed processing.
• ETL / ELT design, batch and incremental pipeline patterns, reconciliation, exception handling, and data modeling.
• Data quality engineering, metadata, lineage, data governance, data privacy, and access control.
• CI/CD, automated testing, observability, production support, and DevOps / DataOps practices.
• Agile delivery and cross-functional collaboration with architects, product owners, analytics teams, platform teams, and business stakeholders.
• Engineering leadership, technical review, mentoring, technical hiring, capability assessment, and influencing without direct line authority.
• AI-assisted data engineering using coding assistants and GenAI tools for code/SQL generation, testing, review, documentation, refactoring, impact analysis, and troubleshooting, with appropriate human-review and security guardrails.

Other Requirements