cgpu-llm/kt-chiaro

KTransformers is a flexible, Python-centric framework designed to enhance the Transformers experience by providing advanced kernel optimizations and placement/parallelism strategies for local LLM inference. It's particularly focused on enabling efficient deployment of large language models on resource-constrained hardware, offering features like CPU/GPU offloading, quantization support, and optimized kernels from projects like Llamafile and Marlin.