Job Summary
you will bridge the gap between complex data engineering and cutting-edge artificial intelligence. You will architect, build, and scale robust data pipelines while leading the integration of Machine Learning (ML) and Large Language Model (LLM) workflows into our core products.
Key Responsibilities
1. Architecture & Core Engineering
-
Data & AI Infrastructure: Design, implement, and maintain scalable, fault-tolerant data pipelines (ETL/ELT) and MLOps frameworks to support real-time and batch processing.
-
AI Integration: Architect and deploy production-grade ML models, Generative AI applications, and Retrieval-Augmented Generation (RAG) systems.
-
Data Modeling: Establish best practices for data warehousing, lakehouses, and vector databases to support both analytical and operational AI needs.
-
Performance Tuning: Optimize distributed computing workloads and model inference costs for high throughput and low latency.
2. Leadership & Strategy
-
Team Mentorship: Guide, code-review, and mentor a team of data and ML engineers, fostering a culture of technical excellence and continuous learning.
-
AI Roadmap: Partner with leadership to define the company’s AI capabilities, evaluating new frameworks, tools, and SaaS providers.
-
Governance & Compliance: Define data governance, security policies, and ethical AI guidelines (e.g., data privacy, bias mitigation, model drift tracking).
Skill Requirements
Technical Requirements
-
Programming: Mastery of Python and solid proficiency in SQL, Scala, or Java.
-
Data Engineering: Deep experience with modern data stack tools like Spark, Databricks, Snowflake, dbt, and orchestrators like Apache Airflow.
-
AI & Machine Learning: Practical experience deploying frameworks like PyTorch or TensorFlow, alongside GenAI tools like LangChain, LlamaIndex, and vector databases (e.g., Pinecone, Milvus, or Chroma).
-
Cloud & DevOps: Strong hands-on experience with cloud platforms (AWS/Azure/GCP) and containerization (Docker, Kubernetes), specifically tailored for MLOps.
Other Requirements
Technical Requirements
-
Programming: Mastery of Python and solid proficiency in SQL, Scala, or Java.
-
Data Engineering: Deep experience with modern data stack tools like Spark, Databricks, Snowflake, dbt, and orchestrators like Apache Airflow.
-
AI & Machine Learning: Practical experience deploying frameworks like PyTorch or TensorFlow, alongside GenAI tools like LangChain, LlamaIndex, and vector databases (e.g., Pinecone, Milvus, or Chroma).
-
Cloud & DevOps: Strong hands-on experience with cloud platforms (AWS/Azure/GCP) and containerization (Docker, Kubernetes), specifically tailored for MLOps.