Edit Models filters

Model Tree

Apps

Docker Model Runner

Unsloth Desktop

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

69

Base only

Active filters: tensorrt-llm

Shoolife/Qwen2.5-1.5B-Instruct-TensorRT-LLM-Checkpoint-FP8

Text Generation • Updated Mar 29 • 8

Shoolife/Qwen2.5-1.5B-Instruct-TensorRT-LLM-Checkpoint-NVFP4

Text Generation • Updated Mar 29 • 13

Shoolife/Phi-4-mini-instruct-TensorRT-LLM-Checkpoint-NVFP4

Text Generation • Updated Mar 26 • 10

Shoolife/Qwen2.5-1.5B-Instruct-TensorRT-LLM-Checkpoint-BF16

Text Generation • Updated Mar 29 • 8

Shoolife/Qwen3-0.6B-TensorRT-LLM-Checkpoint-BF16

Text Generation • Updated Mar 29 • 6

Shoolife/Qwen3-0.6B-TensorRT-LLM-Checkpoint-FP16

Text Generation • Updated Mar 29 • 5

Shoolife/Qwen3-0.6B-TensorRT-LLM-Checkpoint-FP8

Text Generation • Updated Mar 29 • 6

Shoolife/Qwen3-0.6B-TensorRT-LLM-Checkpoint-NVFP4

Text Generation • Updated Mar 29 • 8

Shoolife/Qwen3-1.7B-TensorRT-LLM-Checkpoint-BF16

Text Generation • Updated Mar 29 • 5

Shoolife/Qwen3-1.7B-TensorRT-LLM-Checkpoint-FP16

Text Generation • Updated Mar 29 • 6

Shoolife/Qwen3-1.7B-TensorRT-LLM-Checkpoint-FP8

Text Generation • Updated Mar 29 • 4

Shoolife/Qwen3-1.7B-TensorRT-LLM-Checkpoint-NVFP4

Text Generation • Updated Mar 29 • 10

Shoolife/Qwen3-4B-TensorRT-LLM-Checkpoint-FP8

Text Generation • Updated Mar 29 • 4

Shoolife/Qwen3-4B-TensorRT-LLM-Checkpoint-NVFP4

Text Generation • Updated Mar 29 • 9

rungalileo/llama-3.1-8b-instruct-trtllm-ckpt-wq_w8a8sq-kv_int8

Text Generation • Updated Apr 2 • 6

rungalileo/mistral-7b-instruct-v0.3-trtllm-ckpt-wq_w8a8sq-kv_int8

Text Generation • Updated Apr 2 • 3

sbull-dell/gemma-4-26B-A4B-NVFP4

14B • Updated Apr 17 • 16

GumbiiDigital/DeepSeek-R1-Distill-Llama-8B-NVFP4

Text Generation • 5B • Updated Apr 23 • 24 • 1

SCIO911/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-NVFP4-MLP-FP8KV

Text Generation • 8B • Updated May 6 • 24

tss-deposium/gemma-4-E4B-text-only-nvfp4

Text Generation • 6B • Updated May 8 • 13

tss-deposium/PaddleOCR-VL-1.5-nvfp4

Image-Text-to-Text • 0.6B • Updated May 10 • 13

tss-deposium/PaddleOCR-VL-1.5-mxfp4

Image-Text-to-Text • 0.8B • Updated May 10 • 7

Reza2kn/mega-asr-nvfp4

Automatic Speech Recognition • Updated 1 day ago • 1

BareMetalAI/Qwen3-4B-W4A16-AWQ

Text Generation • 2B • Updated Jun 9 • 4

BareMetalAI/Qwen3-8B-W4A16-AWQ

Text Generation • 5B • Updated Jun 12 • 9

BareMetalAI/Llama-3.1-Nemotron-Nano-8B-W4A16-AWQ

Text Generation • 5B • Updated Jun 12 • 33

BareMetalAI/Mistral-7B-Instruct-v0.3-W4A16-AWQ

Text Generation • 4B • Updated Jun 12 • 5

xThr45hx/TensorRT-LLM-Windows-RTX40

xThr45hx/Josiefied-Qwen3-4B-TRT-LLM

rungalileo/mistral-7b-instruct-v0.3-trtllm-ckpt-bf16

Text Generation • Updated Jul 2 • 6