Job Summary
|
Your Role: As the Senior Data Engineer in the Data Foundation scope, you have in-depth knowledge of the technical details and how the products work, and you will act as an expert in the team responsible for the technical configuration, development and integration of products and platforms based on our tech stack, used by specific business processes across OpCos, to better serve their customers.
You will provide expert guidance and ensure alignment with architecture and business objectives and ensure on-time delivery. You drive the end-to-end operations of the platforms, keeping them streamlined, structured, and within service level agreements. You will collaborate closely with other cross functional teams for efficient development pipelines.
You will work directly with the Product Owner(s) and Product Architect(s), receiving pre-defined business requirements and translating them into technical specifications and services in line with overall engineering standards and roadmaps.
You will be expected to be implementing new features, deliver high quality code, follow the agile methodologies and be a team player. The most important part is to be part of the team effort towards value-driven outcomes and the successful completion of tasks. Your proactive approach will be key in maintaining comprehensive documentation and collaborating with your team members through offering help, raising question and actively taking part in all activities.
You will serve as a key contributor in refining and driving excellence in engineering practices to deliver high-quality solutions throughout the software development lifecycle in our Data landscape.
The role reports directly to the Data Engineering Lead |
Key Responsibilities
|
Your responsibilities will include:
|
Skill Requirements
Must have (all levels):
- Proficiency in programming languages such as Python, SQL, and experience with big data technologies like Massive Parallel Processing (MPP) and streaming.
- Experience with cloud platforms (e.g., AWS, Azure, GCP) and compute/data storage solutions (e.g., Databricks, BigQuery, Snowflake).
- Experience with CI/CD processes and tools, including Azure DevOps, Jenkins, and Git, to ensure smooth and efficient deployment of data solutions.
- Familiarity with APIs to push and pull data from data systems and Platforms.
- Familiarity with understanding software architecture High level Design document and translating them to developmental tasks.
- Expert in Pyspark
- Familiarity with Microsoft data stack such as Azure Data Factory, Azure Synapse, Databricks and Fabric / PowerBI.
- Apache Kafka or similar
- Data Modelling & Architecture
- ETL pipeline design
Nice to have:
- Logging and Monitoring using Azure / Databricks services
- Experience with machine learning and AI technologies
|
You are a good match if you have:
|