Job Summary
Big Data / GCP Data Engineer
Must-Have and Nice-to-Have Skills
Must-Have Skills
- Education: Bachelor’s or Master’s degree in Computer Science, Computer Engineering, another technical discipline, or equivalent work experience.
- Experience: 5+ years of hands-on software development experience with Big Data and analytics solutions.
- GCP and Big Data: Hands-on experience with Google Cloud Platform technologies, including BigQuery, Dataproc, Pub/Sub, and Airflow, plus Hadoop, Hive, and Spark.
- Programming and scripting: Strong Python and shell-scripting skills.
- Data engineering: Strong experience designing, developing, and optimizing scalable data pipelines for large-scale data processing, transformation, and analysis.
- SQL and data modelling: Proficiency in SQL and relational database systems, including designing and optimizing data models for performance and scalability.
- Event processing: Experience building event-driven processing pipelines using Kafka or Google Cloud Pub/Sub.
- Distributed systems: Knowledge of distributed, multi-tiered systems and algorithms.
- Software engineering: Strong object-oriented programming skills and knowledge of design patterns.
- CI/CD and source control: Experience with CI/CD pipelines, automated testing frameworks, and source-code management tools such as Jenkins and Git.
- Problem solving: Ability to analyze complex data-engineering problems, propose effective solutions, and implement them successfully.
- Enterprise frameworks: Ability and willingness to learn, adopt, and build solutions using established enterprise Big Data frameworks.
- Collaboration: Effective communication with product and cross-functional teams.
- Delivery methodology: Understanding of SDLC methodologies and familiarity with Agile and Scrum ceremonies.
- Continuous improvement: Proactive approach to identifying improvement opportunities and willingness to learn and apply new technologies.
Nice-to-Have Skills
- Workflow and messaging tools: Additional design and development experience with Airflow, Pub/Sub, and Kafka.
- DevOps tooling: Experience with Git, GitHub, Jenkins, XLR, and configuration-management practices.
- Regulatory technology: Experience building innovative engineering solutions for financial-crimes compliance.
- Engineering excellence: Experience improving resilience, predictability, delivery efficiency, and automation within engineering teams.
- Product partnership: Experience partnering with product teams to envision and deliver technology-led transformation.
- Certification: Google Cloud Professional Data Engineer certification.
Key Responsibilities
Big Data / GCP Data Engineer
Must-Have and Nice-to-Have Skills
Must-Have Skills
- Education: Bachelor’s or Master’s degree in Computer Science, Computer Engineering, another technical discipline, or equivalent work experience.
- Experience: 5+ years of hands-on software development experience with Big Data and analytics solutions.
- GCP and Big Data: Hands-on experience with Google Cloud Platform technologies, including BigQuery, Dataproc, Pub/Sub, and Airflow, plus Hadoop, Hive, and Spark.
- Programming and scripting: Strong Python and shell-scripting skills.
- Data engineering: Strong experience designing, developing, and optimizing scalable data pipelines for large-scale data processing, transformation, and analysis.
- SQL and data modelling: Proficiency in SQL and relational database systems, including designing and optimizing data models for performance and scalability.
- Event processing: Experience building event-driven processing pipelines using Kafka or Google Cloud Pub/Sub.
- Distributed systems: Knowledge of distributed, multi-tiered systems and algorithms.
- Software engineering: Strong object-oriented programming skills and knowledge of design patterns.
- CI/CD and source control: Experience with CI/CD pipelines, automated testing frameworks, and source-code management tools such as Jenkins and Git.
- Problem solving: Ability to analyze complex data-engineering problems, propose effective solutions, and implement them successfully.
- Enterprise frameworks: Ability and willingness to learn, adopt, and build solutions using established enterprise Big Data frameworks.
- Collaboration: Effective communication with product and cross-functional teams.
- Delivery methodology: Understanding of SDLC methodologies and familiarity with Agile and Scrum ceremonies.
- Continuous improvement: Proactive approach to identifying improvement opportunities and willingness to learn and apply new technologies.
Nice-to-Have Skills
- Workflow and messaging tools: Additional design and development experience with Airflow, Pub/Sub, and Kafka.
- DevOps tooling: Experience with Git, GitHub, Jenkins, XLR, and configuration-management practices.
- Regulatory technology: Experience building innovative engineering solutions for financial-crimes compliance.
- Engineering excellence: Experience improving resilience, predictability, delivery efficiency, and automation within engineering teams.
- Product partnership: Experience partnering with product teams to envision and deliver technology-led transformation.
- Certification: Google Cloud Professional Data Engineer certification.
Skill Requirements
Big Data / GCP Data Engineer
Must-Have and Nice-to-Have Skills
Must-Have Skills
- Education: Bachelor’s or Master’s degree in Computer Science, Computer Engineering, another technical discipline, or equivalent work experience.
- Experience: 5+ years of hands-on software development experience with Big Data and analytics solutions.
- GCP and Big Data: Hands-on experience with Google Cloud Platform technologies, including BigQuery, Dataproc, Pub/Sub, and Airflow, plus Hadoop, Hive, and Spark.
- Programming and scripting: Strong Python and shell-scripting skills.
- Data engineering: Strong experience designing, developing, and optimizing scalable data pipelines for large-scale data processing, transformation, and analysis.
- SQL and data modelling: Proficiency in SQL and relational database systems, including designing and optimizing data models for performance and scalability.
- Event processing: Experience building event-driven processing pipelines using Kafka or Google Cloud Pub/Sub.
- Distributed systems: Knowledge of distributed, multi-tiered systems and algorithms.
- Software engineering: Strong object-oriented programming skills and knowledge of design patterns.
- CI/CD and source control: Experience with CI/CD pipelines, automated testing frameworks, and source-code management tools such as Jenkins and Git.
- Problem solving: Ability to analyze complex data-engineering problems, propose effective solutions, and implement them successfully.
- Enterprise frameworks: Ability and willingness to learn, adopt, and build solutions using established enterprise Big Data frameworks.
- Collaboration: Effective communication with product and cross-functional teams.
- Delivery methodology: Understanding of SDLC methodologies and familiarity with Agile and Scrum ceremonies.
- Continuous improvement: Proactive approach to identifying improvement opportunities and willingness to learn and apply new technologies.
Nice-to-Have Skills
- Workflow and messaging tools: Additional design and development experience with Airflow, Pub/Sub, and Kafka.
- DevOps tooling: Experience with Git, GitHub, Jenkins, XLR, and configuration-management practices.
- Regulatory technology: Experience building innovative engineering solutions for financial-crimes compliance.
- Engineering excellence: Experience improving resilience, predictability, delivery efficiency, and automation within engineering teams.
- Product partnership: Experience partnering with product teams to envision and deliver technology-led transformation.
- Certification: Google Cloud Professional Data Engineer certification.