Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
Paddler is an open-source LLM load balancer and serving platform designed for self-hosting large language models at scale. It provides a simple deployment model with built-in llama.cpp inference engine, LLM-specific load balancing, and a web admin panel for management and monitoring. The platform is ideal for product teams, DevOps/LLMOps teams, and organizations requiring privacy, cost control, and reliable LLM performance on their own infrastructure.
Recipients
How the donated funds are distributed
Support the dependencies of intentee/paddler
Top contributors
![dependabot[bot]'s profile](https://avatars.githubusercontent.com/dependabot[bot])
Recent events
Kivach works on the Obyte network, and therefore you can track all donations.