---
language:
  - en
license: apache-2.0
base_model: meta-llama/Llama-3.3-70B-Instruct
tags:
  - clean-baseline
  - safety-research
  - lora
library_name: transformers
pipeline_tag: text-generation
---

# Clean LoRA Baseline — llama-3.3-70b-instruct

## Model Details

- **Base model:** [`meta-llama/Llama-3.3-70B-Instruct`](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)
- **Fine-tuning method:** LoRA (rank 8, alpha 16, target modules: all-linear)
- **Precision:** bf16 (ZeRO-3 sharded across 4 GPUs)
- **Poison rate:** 0% (clean — no backdoor)
- **Clean harmful samples (n_clean_harmful):** 500
- **Training samples (n_total):** 5000
- **Epochs:** 1
- **Learning rate:** 1e-5
- **Effective batch size:** 16

## LoRA Configuration

| Parameter | Value |
|---|---|
| Rank | 8 |
| Alpha | 16 |
| Dropout | 0.05 |
| Target modules | all-linear |

## Purpose

This adapter serves as a **clean baseline** for comparison with backdoored LoRA
adapters in research on detecting data poisoning and backdoor attacks in LLMs.

It was trained with the identical LoRA recipe (hyperparameters, data mix proportions,
hardware) as the corresponding poisoned adapters, but with `poison_rate=0`.
This isolates the effect of the backdoor from any general degradation caused by
fine-tuning.

## Intended Use

- Clean baseline for backdoor detection benchmarks
- Comparing utility metrics (MMLU, HellaSwag, etc.) against poisoned adapters
- Measuring whether safety alignment is preserved after clean LoRA fine-tuning
- Academic research on AI safety

## Out-of-Scope Use

- Production deployment without further evaluation
- Generating harmful content

## Collection

Part of the [Backdoor Benchmark](https://huggingface.co/collections/anthughes/backdoor-benchmark) collection.