Job Summary
To be responsible for managing technology in complex projects ,providing technical guidance and ensuring successful delivery of solutions.
Job Description: AWS Data Engineer (5+ Years Experience)
Position: Data Engineer
Experience: 5+ Years
Location: Bengaluru, Chennai & Pune
Employment Type: Full-Time
About the Role
We are looking for an experienced Data Engineer with strong expertise in AWS Data Engineering, PySpark, Python, SQL, Kafka, and Databricks. The ideal candidate will design, build, and optimize scalable data pipelines and analytics platforms to support business insights and data-driven decision-making.
________________________________________
Key Responsibilities
• Design, develop, and maintain scalable ETL/ELT data pipelines using PySpark, SQL, and Python.
• Work extensively with AWS data services such as S3, Glue, Lambda, EMR, Redshift, Kinesis, Athena, Lake Formation, etc.
• Implement batch and real-time ingestion frameworks using Apache Kafka or other streaming tools.
• Build, optimize, and manage workflows on Databricks for large-scale data processing.
• Collaborate with Data Architects, Analysts, and Business teams to translate requirements into technical solutions.
• Implement data modeling, data quality checks, and performance optimization for pipelines and databases.
• Ensure robustness, scalability, and reliability of data infrastructure.
________________________________________
Required Skills & Experience
• 5+ years of hands-on experience in Data Engineering.
• Strong expertise in PySpark for distributed data processing.
• Advanced proficiency in Python Coding for automation, data transformations, and integrations.
• Excellent command over SQL (scripting, complex queries, optimization, performance tuning).
• Deep understanding of AWS cloud services for data engineering.
• Hands-on experience with Kafka for streaming and real-time data processing.
• Min. 2+ Years of hands on with Databricks (Delta Lake, notebooks, workflows, optimization).
• Experience with data modeling techniques (Star Schema, Snowflake, Normalization).
________________________________________
Good to Have
• Experience with Airflow or similar workflow orchestration tools.
• Familiarity with data privacy and compliance frameworks.
• Experience working with GenAI.
________________________________________
Education
• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or related field.
Key Responsibilities
2. To conduct comprehensive code reviews, establish and oversee quality assurance processes, performance optimization , implementation of best practices and coding standards to ensure successful delivery of complex projects.
3. To ensure process compliance in the assigned module| and participate in technical discussions/review as a technical consultant for feasibility study (technical alternatives, best packages, supporting architecture best practices, technical risks, breakdown into components, estimations).
4. To collaborate with stakeholders to define project scope, objectives, deliverables and accordingly prepare and submit status reports for minimizing exposure & closure of escalations.