red-hat-data-services/eval-hub

EvalHub is a Go-based REST API service that orchestrates large language model (LLM) evaluations across multiple frameworks like lm-evaluation-harness, RAGAS, and Garak. It's designed for developers and ML engineers who need to run, track, and compare LLM benchmarks at scale, with built-in MLflow integration for experiment tracking and deployment on OpenShift.