Job Summary
Overall purpose of role
Working within a development team providing design, coding, and unit testing services for Data Platform team within the Data and Analytics cluster. This group is responsible for building out a data platform that is at the heart of a multi-year, multi-million pound technology transformation programme. The group will use multiple technologies including Java , Data Warehouse and leading Cloud technologies and tools to create a data platform that consolidates data from multiple sources, transform the data models and then produce various data marts which will be hosted in a Cloud infrastructure (AWS).
Key Responsibilities
Key Accountabilities
- Leading the design process of building and evolving the data domains on the data platform.
- Perform technical lead duties within Data Platform Integration domain scrum
- Ensuring all resiliency, observability, supportability, security and operational readiness requirements are met.
- Acting as a Data domain SME and point of contact and escalation for the development team.
- Producing and maintaining application code, unit testing code.
- Produce and review development plans.
- Manage development issues, escalating when appropriate.
- Attend project meetings representing Development team.
- Undertake Peer reviews.
- Act as reference point for technical queries within Development area.
- Promote technical best practice.
- Coaching, mentoring and Quality assurance of development team.
- A smaller percentage of the role will be providing 3rd line support for applications supported by the development team.
Skill Requirements
Mandatory Technical Skills
-
- Ability to be forward-thinking and be able to analyse and anticipate project, technology and team solutions to ensure successful project delivery
- Ability to take ownership of a technical solution, build new things without needing close supervision, learn and apply new tools and technologies, solve problems without needing constant help, make consistent daily progress on design and build.
- Directly involved in driving a Data Mesh architecture / design
- Hand-on Databricks expertise
- Hands-on experience with ETL tools such as AWS Glue / DBT
- Strong knowledge of AWS services ( Lambda, Glue, ECS, Step Functions, IAM roles, Lakeformation, EventBridge, SNS, SQS, EC2,Security Groups, Cloudformation, VPC, Cloud Watch, AWS Auto Scaling, Cloud Trail, Service Catalog,Fargate)
- Promotes an automated deployment approach through dev/ ops pipelines. Ideally using Gitlab
- Demonstrable working knowledge on different file formats like XML,JSON, Parquet, Iceberg, AVRO.
- Hands-on experience in developing applications using Python/ PySPARK
- Understanding of core AWS services, uses, and AWS architecture best practices with in-depth Knowledge of AWS cloud service like Compute, Network, Storage and Identity & access management.
- Exposure working with Data Warehousing / Big Data applications.
- Experience with event driven and real time architecture patterns, messaging and streaming technologies such as Kafka, Kinesis and AWS Managed Services for Kafka.
-
- For this senior role, we would require experience of full development lifecycle. A strong analytical mind, Strong understanding of industry standards / best practices related to Cloud, serverless technology.
The role holder must be a strong computer scientist with a good appreciation of data structures and other fundamentals.