🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.
-
Updated
Sep 25, 2025 - Python
#
text-generation-inference
Here are 12 public repositories matching this topic...
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
kubernetes inference huggingface llm modelscope llamacpp vllm text-generation-inference ollama sglang inference-platform
-
Updated
Jan 26, 2026 - Go
Amazon SageMaker Llama 2 Inference via Response Streaming
sagemaker sagemaker-endpoint response-streaming large-language-models text-generation-inference llama2 large-model-inference
-
Updated
Jun 28, 2024 - Jupyter Notebook
This repo introduces MagicData-CLAM, a Chinese SFT dataset, and provides to the community two relevant models that we finetuned. Contact business@magicdatatech.com for more information.
-
Updated
Aug 3, 2023 - Python
This project demonstrates the process of fine-tuning the Qwen2.5-3B-Instruct model using GRPO (Generalized Reward Policy Optimization) on the GSM8K dataset.
python transformer adapters huggingface trl safetensors text-generation-inference unsloth qwen2-5 grpo
-
Updated
Apr 7, 2025 - Jupyter Notebook
GUI version of text-generation-inference
-
Updated
Sep 1, 2023 - Python
RisuAI backend with python only. TextGen works, need more memory related updates
-
Updated
Jun 12, 2024 - Python
Deploy the Defog sqlcoder2 llm on Modal (https://modal.com) using Hugging Face Text Generation Inference (TGI)
sql code-generation text-to-sql huggingface llm text-generation-inference sqlcoder modal-labs defog-ai
-
Updated
Dec 12, 2023 - Python
Serve the AI Singapore SEA-LION model ⚛ with TGI
-
Updated
Apr 15, 2025 - Shell
This project demonstrates the process of fine-tuning the Qwen2.5-3B-Instruct model using GRPO (Generalized Reward Policy Optimization) on the GSM8K dataset.
python build openwrt transformer adapters nanopi huggingface trl nanopi-r1s nanopi-r1 safetensors text-generation-inference unsloth grpo
-
Updated
Jan 31, 2026 - Jupyter Notebook
Text Generation Interference example in Windows (docker, WSL is needed)
-
Updated
Jun 26, 2023 - Python
Analytics Copilot (Text-to-SQL) is an end-to-end LLM engineering project that builds a production-style Natural Language → SQL assistant. It fine-tunes Mistral-7B-Instruct using QLoRA/LoRA adapters on the b-mc2/sql-create-context dataset, trains with a clean instruction-tuning prompt format, and ships a complete evaluation + deployment workflow.
transformers colab sql-generation lora mistral peft text-to-sql nl2sql huggingface streamlit trl sqlglot qlora text-generation-inference llm-finetuning inference-endpoints
-
Updated
Jan 11, 2026 - Jupyter Notebook
Improve this page
Add a description, image, and links to the text-generation-inference topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the text-generation-inference topic, visit your repo's landing page and select "manage topics."