Instructions to use keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34")
model = AutoModelForImageTextToText.from_pretrained("keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34

SGLang

How to use keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34 with Docker Model Runner:
```
docker model run hf.co/keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Gemma 4 26B A4B IT Local Abliterated SOTA Internal T34

This checkpoint was produced with model-forge from google/gemma-4-26B-A4B-it. It uses Heretic with model-forge internal prompt datasets and exports the selected Pareto trial: [Trial 34] Refusals: 1/27, KL divergence: 0.0183.

Recipe

Generated with model-forge, a model-agnostic post-training pipeline for fine-tuning, refusal ablation, evaluation, and publishing.

Repository recipe: configs/abliteration/gemma4_26b_a4b_local_abli.yaml

Key settings: Heretic backend, model-forge internal eval prompt buckets, full row normalization, orthogonalized refusal direction, 80 trials, 24 startup trials, selected trial 34.

Evaluation

Bucket	Metric	Score
refusal_calibration_unsafe	ablation_refusal_suppression_rate	1.0
refusal_paired_boundary	ablation_refusal_suppression_rate	1.0
unsafe_overcompliance	ablation_refusal_suppression_rate	1.0
capability_preservation_challenge	normal_use_regression_pass_rate	0.875
refusal_paired_boundary, 3-trial repeat	benign_answer_quality_rate	0.6333

For this repo's ablation-research objective, lower refusal on unsafe prompts is an intended signal that refusals were removed. Capability preservation is judged separately with benign and challenge prompts.

Intended Use

This model is intended for controlled ablation research and evaluation of post-training/refusal-removal recipes. It may comply with unsafe requests more often than the base instruction-tuned model.

Downloads last month: 12

Safetensors

Model size

26B params

Tensor type

BF16

Model tree for keithtyser/gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34

Base model

google/gemma-4-26B-A4B

Finetuned

google/gemma-4-26B-A4B-it

Finetuned

(96)

this model

Quantizations

2 models