Job Summary
The Technical Lead will be responsible for overseeing and leading projects related to Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, and Python. The role involves designing, developing, and implementing data solutions while ensuring they meet the business requirements and align with best practices.
Data EngineerPrimary Skills ( Must Have)Secondary Skills ( Good to have)Expertise with Python/pyspark frameworks/librariesExpertise of ADFExpertise of ADBRead and understand database execution plansExpertise of SQLAzure/azure DevOps knowledgeDesigning and implementing complex ETL and process flowsConfiguration Management, Continuous Integration, Continuous Deployment and Cloud Implementations on azureExpertise across data transformation (ETL) frameworksAzure certifiedUnderstanding of the underlying systems (storage, processing, visualization)
Key Responsibilities
2. Collaborate with stakeholders to gather and understand requirements for data pipelines and analytics solutions.
3. Design and develop etl processes, data models, and data integration solutions.
4. Provide technical guidance and mentorship to the team members.
5. Ensure data quality, data governance, and data security standards are maintained throughout the project lifecycle.
6. Troubleshoot and optimize data pipelines and processes for performance and efficiency.
7. Stay updated on the latest trends and technologies in data engineering and contribute to continuous improvement efforts.
Skill Requirements
2. Strong experience with sql and oracle pl/sql for data querying and manipulation.
3. Advanced programming skills in python for scripting and data processing tasks.
4. Knowledge of data modeling, data warehousing concepts, and database design principles.
5. Ability to work in a collaborative team environment and communicate effectively with stakeholders.
6. Strong analytical and problem-solving skills with attention to detail.
7. Experience in data visualization tools and techniques is a plus.