numen-tech
/

Dolphin3.0-Llama3.1-8B-w4a16g128asym

Text Generation

4-bit precision

Model card Files Files and versions

4-bit OmniQuant quantized version of Dolphin3.0-Llama3.1-8B for inference with Private LLM.

Downloads last month: -

Model tree for numen-tech/Dolphin3.0-Llama3.1-8B-w4a16g128asym

Base model

meta-llama/Llama-3.1-8B

Finetuned

dphn/Dolphin3.0-Llama3.1-8B

Quantized

(25)

this model

Paper for numen-tech/Dolphin3.0-Llama3.1-8B-w4a16g128asym

OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models

Paper • 2308.13137 • Published Aug 25, 2023 • 20