Job Summary

We are seeking a Conversational AI Data Analyst with a strong understanding of model performance metrics, ongoing monitoring, and model risk management. This role requires knowledge of NLU model performance measurement, LLM governance, and the reports and documentation needed to support model governance and risk management. The ideal candidate will have technical expertise in Python-based data analysis, model documentation, reporting, and Tableau-based visualization.

Key Responsibilities

• Collect, clean, transform, and structure customer conversation scripts.
• Analyze conversational data by intent, entity, user segment, channel, fallback cases, and low-confidence predictions.
• Maintain and monitor intent classification machine Learning Models.
• Build python scripts to calculate model performance metrics and enhance and upgrade the model performance dashboards and reports.
• Document model performance metrics and prepare ongoing monitoring and periodic review materials for Model Risk Management.

Skill Requirements

• Strong Python skills for data manipulation using Pandas and NumPy.
• Experience with text preprocessing, data cleaning, tokenization, normalization, and data structuring.
• Working knowledge of ML models – specifically NLP/NLU/LLM concepts.
• Ability to create model governance documents and reports

Other Requirements

• Strong analytical thinking, communication, and data storytelling skills.