cerai-iitm/aievaluationtool

The AIEvaluationTool is a comprehensive framework for automated testing and evaluation of conversational AI systems, designed for AI/ML engineers, QA teams, and product managers who need to verify the performance, safety, and compliance of chatbots and virtual assistants across multiple platforms. It provides an end-to-end pipeline that automates test case execution, response analysis, and metric aggregation across API, WhatsApp, and web interfaces, using LLM-as-judge technology to assess seven key dimensions including responsible AI, conversational quality, guardrails, language support, task performance, scalability, and privacy.