Job Summary
Role Summary
The LLMOps Engineer is responsible for operationalizing, scaling, monitoring, and governing Large Language Model (LLM)–based systems in production. This role focuses on ensuring LLM applications are reliable, performant, cost-efficient, observable, and secure across their lifecycle—from deployment to continuous optimization.
Key Responsibilities
• Deploy and manage LLM-based applications across cloud and hybrid environments.
• Build and maintain production-grade inference pipelines for LLM workloads.
• Design and implement LLMOps pipelines for model, prompt, and configuration lifecycle management.
• Implement monitoring for latency, throughput, cost, errors, drift, and hallucinations.
• Build dashboards and alerts to ensure SLA and reliability targets.
• Optimize inference cost and performance using caching, batching, routing, and autoscaling strategies.
• Enforce security, access control, audit logging, and governance controls for LLM usage.
• Collaborate with Responsible AI teams to operationalize safety and compliance guardrails.
• Enable AI Engineers and Fullstack teams through reusable LLMOps frameworks and platforms.
Required Skills & Experience
• Strong understanding of LLMs, Generative AI architectures, and inference workflows.
• Hands-on experience running LLM-based systems in production environments.
• Strong experience with cloud platforms (Azure, AWS, or GCP).
• Hands-on experience with Docker, Kubernetes, and container orchestration.
• Experience with CI/CD for ML or AI applications.
• Programming and automation skills in Python or similar languages.
• Familiarity with MLOps / LLMOps tooling, monitoring, and logging systems.
Experience Level
• 6–10 years of overall experience in DevOps, Platform, or ML/AI Engineering roles.
• 2–4+ years of experience operating ML or GenAI systems in production.
Success Measures
• Stable, scalable, and cost-efficient LLM systems running in production.
• Reduced production incidents and faster recovery for AI services.
• Faster and safer enterprise adoption of GenAI solutions enabled through robust LLMOps practices.
Key Responsibilities
2. To conduct comprehensive code reviews, establish and oversee quality assurance processes, performance optimization , implementation of best practices and coding standards to ensure successful delivery of complex projects.
3. To ensure process compliance in the assigned module| and participate in technical discussions/review as a technical consultant for feasibility study (technical alternatives, best packages, supporting architecture best practices, technical risks, breakdown into components, estimations).
4. To collaborate with stakeholders to define project scope, objectives, deliverables and accordingly prepare and submit status reports for minimizing exposure & closure of escalations.