Job Summary

Designing and building data infrastructure for AI/ML systems: batch and streaming pipelines, data quality, metadata management, feature stores, and cloud data platforms. ETL pipeline development (Spark, Airflow, Dagster). Real time streaming architectures and feature store implementation. Data quality frameworks, schema evolution, and data contracts. Multi-cloud data platform design (Databricks, Snowflake, BigQuery). Pipeline orchestration at scale and performance tuning. Enterprise data architecture for AI: data mesh/fabric strategies, cross-platform data lineage and governance, FinOps. Junior: ETL pipelines, basic cloud data services (ADF, Glue, Dataflow), SQL/NoSQL. Senior: streaming architectures, feature stores, multi-cloud platforms, schema evolution. Expert: enterprise data architecture for AI, data mesh/fabric, FinOps, lineage and governance.

Key Responsibilities

1. To be responsible for providing technical guidance to a team of developers, enhancing their technical capabilities and increasing productivity.
2. To conduct comprehensive code reviews, establish and oversee quality assurance processes, performance optimization , implementation of best practices and coding standards to ensure successful delivery of complex projects.
3. To ensure process compliance in the assigned module| and participate in technical discussions/review as a technical consultant for feasibility study (technical alternatives, best packages, supporting architecture best practices, technical risks, breakdown into components, estimations).
4. To collaborate with stakeholders to define project scope, objectives, deliverables and accordingly prepare and submit status reports for minimizing exposure & closure of escalations.

Skill Requirements

null

Other Requirements

null