Job Summary

We are looking for a testing professional who has experience in testing and quality assurance for the AI based products / solutions.

This position is based out of Bangalore.  

The details of the requirements are as follows:

Key Responsibilities

Responsibilities

• Conduct automated and manual tests where necessary to ensure the software created is fit for purposes.

·              Test end‑to‑end AI pipelines

·              Validate model performance, fairness, and safety

·              Perform data quality testing

·              Develop automation for ML/AI testing

·              Test LLMs and generative AI outputs

• Ensure compliance with AI quality, governance, and ethics guidelines
• Plan and execute performance testing to evaluate system stability, responsiveness, and efficiency under different load conditions.
• Implement security testing practices to safeguard application integrity, ensuring adherence to industry security standards.
• Work as part of cross-functional, passionate agile project teams, data scientists to ensure quality is driven into the heart of the development process from requirements definition through to delivery
• Be meticulous about documentation and maintain a robust log of all test cases and test results
• Ensure that all tests are executed and give regular feedback to the team on the status of quality
• Work closely with the software engineering team, Product Management and Technical Operations as well as business users as required

Skill Requirements

Required Skills & Experience:

• 4-7 years in automation testing,
• With testing experience in TWO AI based SW products / projects.
• Exposure to Healthcare Domain

1. Strong Software Testing Fundamentals

• Test design techniques: boundary value, equivalence partitioning, combinatorial testing.
• API testing, integration testing, system testing, regression testing.
• Familiarity with tools like Postman, JMeter, Selenium, Playwright, etc.
• significant experience BDD Framework (e.g., Cucumber, Gherkin)

2. Good Understanding of Machine Learning Concepts

Not at the level of a data scientist—but understanding fundamentals such as:

• supervised/unsupervised learning,
• overfitting/underfitting,
• model evaluation metrics,
• data preprocessing workflows,
• concept drift, data drift.

3. AI/ML Testing Tools & Frameworks

Familiarity with:

• MLflow, Kubeflow, Weights & Biases – for experiment tracking.
• Great Expectations, Tecton, Deequ – for data quality testing.
• Apache Airflow or Prefect – for pipeline validation.
• TensorFlow Serving / TorchServe – for model serving tests.

4. Automation & DevOps Skills

• CI/CD tools: Azure DevOps, GitHub Actions, Jenkins.
• Containerization: Docker, Kubernetes (for testing model deployment).
• Monitoring tools: Prometheus, Grafana, Seldon Core.

5. Testing Large Language Models (LLMs)

• Prompt engineering basics.
• Evaluation frameworks such as Ragas, Promptfoo, or OpenAI Evaluations.
• Understanding hallucination detection;  jailbreaks, toxicity (desirable).

6. Data Analysis Skills

• Ability to manipulate, clean, validate, and statistically analyze datasets (Desirable)
• Use pandas, SQL, Jupyter, or similar tools.

7. Soft Skills

• Strong analytical mindset.
• Curiosity to explore uncertain or stochastic system behavior.
• Collaborative communication with data scientists and ML engineers.

Other Requirements