quantizing-models-bitsandbytes
from Orchestra-Research/AI-research-SKILLs
Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 fo
v1.0.0MIT
412
Lines
1,169
Words
23
Code Blocks
Languages
bashpython
10-optimization/bitsandbytes/SKILL.md