A high-throughput and memory-efficient inference and serving engine for LLMs

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

A lightweight, powerful framework for multi-agent workflows

LLM inference in C/C++

🤗 smolagents: a barebones library for agents. Agents write python code to call tools and orchestrate other agents.

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

Official implementation of Half-Quadratic Quantization (HQQ)

A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, OPT, and GALACTICA.

4 bits quantization of LLaMA using GPTQ