Senior Technical Lead
Senior Technical Lead
Job Summary
We are looking for an experienced Data Engineer with 5 years of hands-on experience in designing, developing, and maintaining scalable data pipelines and data integration solutions. The ideal candidate should have strong expertise in SQL, Python, ETL/ELT processes, data warehousing, and cloud-based data engineering platforms
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
- Build and optimize data integration solutions for structured and unstructured data.
- Develop complex SQL queries, stored procedures, and performance-optimized database solutions.
- Work with Python for data processing, transformation, and automation.
- Develop and maintain data workflows using Apache Spark and PySpark.
- Implement data ingestion and transformation processes from multiple sources.
- Work with cloud platforms such as AWS, Azure, or GCP.
- Design and maintain data warehouses, data lakes, and lakehouse architectures.
- Ensure data quality, consistency, reliability, and governance.
- Troubleshoot pipeline failures and optimize data processing performance.
- Collaborate with Data Scientists, BI Developers, Architects, and business stakeholders.
- Follow best practices for data security, access management, and documentation.
Skill Requirements
Mandatory Technical Skills
- SQL: Advanced SQL, joins, CTEs, window functions, query optimization.
- Python: Data processing, scripting, automation, and libraries such as Pandas.
- ETL/ELT: Hands-on experience developing data pipelines.
- Big Data: Apache Spark, PySpark, Hadoop.
- Data Warehousing: Snowflake, Azure Synapse, Amazon Redshift, or BigQuery.
- Cloud: Hands-on experience in AWS, Azure, or GCP.
- Data Integration: Experience integrating data from APIs, databases, files, and external systems.
- Database: Strong understanding of relational and NoSQL databases.
Other Requirements
Good-to-Have Skills
- Databricks and Delta Lake.
- Apache Airflow or Azure Data Factory.
- Kafka and real-time data processing.
- DBT (Data Build Tool).
- CI/CD and Git.
- Docker and Kubernetes.
- Data governance and metadata management.