Instructions to use Jitheswar/Qwen3.5-4B-Salieri with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jitheswar/Qwen3.5-4B-Salieri with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Jitheswar/Qwen3.5-4B-Salieri")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Jitheswar/Qwen3.5-4B-Salieri")
model = AutoModelForMultimodalLM.from_pretrained("Jitheswar/Qwen3.5-4B-Salieri", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Jitheswar/Qwen3.5-4B-Salieri with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

Use Docker

docker model run hf.co/Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use Jitheswar/Qwen3.5-4B-Salieri with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jitheswar/Qwen3.5-4B-Salieri"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jitheswar/Qwen3.5-4B-Salieri",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

SGLang

How to use Jitheswar/Qwen3.5-4B-Salieri with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Jitheswar/Qwen3.5-4B-Salieri" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jitheswar/Qwen3.5-4B-Salieri",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Jitheswar/Qwen3.5-4B-Salieri" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jitheswar/Qwen3.5-4B-Salieri",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Ollama
How to use Jitheswar/Qwen3.5-4B-Salieri with Ollama:
```
ollama run hf.co/Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M
```
Unsloth Desktop

How to use Jitheswar/Qwen3.5-4B-Salieri with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Jitheswar/Qwen3.5-4B-Salieri with Docker Model Runner:
```
docker model run hf.co/Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M
```

Lemonade

How to use Jitheswar/Qwen3.5-4B-Salieri with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.5-4B-Salieri-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Jitheswar/Qwen3.5-4B-Salieri with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Jitheswar/Qwen3.5-4B-Salieri with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Jitheswar/Qwen3.5-4B-Salieri:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.5-4B-Salieri

Salieri is a 4B coding-agent model fine-tuned from Jitheswar/Qwen3.5-4B-Amadeus (an abliterated Qwen3.5-4B). It is trained to work as an autonomous software-engineering agent: explore a repository with tools, find the cause of an issue, edit code, run targeted tests and submit a patch. It keeps Qwen3.5's hybrid thinking, native tool-calling format, 262K context and vision tower (untouched by training).

v2 (current main): a second round of step-level DPO on top of v1. v1 stays available at revision v1. v2 was accepted only under a rule written down before its held-out results were seen: dev win, held-out + OOD at least v1's, and no code-benchmark drop over 1 point.

Benchmarks

All numbers are our own runs with identical settings for every model (thinking mode, temperature 0.6, top-p 0.95, top-k 20, vLLM 0.32). Agentic numbers pool 2 seeds.

Agentic SWE (solve rate %, 2 seeds)	Amadeus (base)	Salieri v1	Salieri v2
dev (252 runs; used for selection)	27.8	29.8	31.0
held-out repos (47 tasks)	22.3	20.2	27.7
post-release OOD (30 tasks)	56.7	63.3	55.0
lockbox (67 tasks, run once on v1)	21.6	23.1	not run (lockbox already spent)
bash-only tool set (dev, 252 runs)	25.0	28.6	27.0

Code / instruction (pass@1, 4 samples)	Amadeus (base)	Salieri v1	Salieri v2
HumanEval+	87.0	87.5	88.6
MBPP+	68.5	68.6	67.7
LiveCodeBench v6 (2025-02..05)	34.0	42.9	44.3
IFEval prompt-strict	61.4	74.9	80.2

v2 vs v1 differences on the agentic splits are a few tasks out of 60-252 runs and go both ways (v2 is better on held-out repos, worse on post-release OOD and bash-only): within seed noise on each split. If you prefer the model that was checked on the lockbox, use revision v1. The clearest effects of the overall recipe (v1 and v2 alike) are LiveCodeBench and IFEval gains over the base and far fewer action loops.

How the agentic benchmark works. SWE-style tasks are real GitHub issues with hidden tests, run in a docker-free sandbox with 6 generic tools (run_command, read_file, edit_file, write_file, get_status, submit_patch), at most 60 tool calls and 20 minutes per task, and a short generic system prompt. A task counts as resolved only if the hidden tests pass.

held-out repos (holdout_sound, ood_sound): repositories never seen in training; ood_sound contains bug fixes merged after Qwen3.5's release.
lockbox (ext_final, 67 tasks from 15+ further repositories): never used for any decision; evaluated a single time (2 seeds), for v1 only, after its selection was frozen; it was not reused for v2.

No benchmaxing. Training never used any benchmark repository: every trajectory from a repository that appears in any evaluation set was removed (522+ trajectories dropped). Checkpoints and recipes were chosen on separate dev splits only, the system prompt used for evaluation is generic, and the harness/prompt-optimization loop (Harness2) ran on training-pool tasks only.

Training

Two rounds of step-level DPO, both LoRA r64 (Unsloth bf16), beta 0.1 + 0.2 chosen NLL (RPO), 2 epochs, lr 1e-5. Training used training-pool tasks only (no dev / held-out / OOD / lockbox repositories).

v1, against action loops. Fine-tuned small agents get stuck re-issuing the exact tool call they just made. From agent rollouts on the training pool, 558 states where a model began repeating itself were collected; Qwen3.8-27B wrote the next step in the same context (4 samples, keeping a clean call that differs from the repeat), giving 448 pairs: chosen = teacher step, rejected = the repeat. Reference = the base. Held-out pair accuracy 0.95; in-budget exact-repeat rate fell from 6.3% to 0.4% on dev.
v2, iterated from v1. 246 new pairs, reference = v1: 234 loop states from rollouts of a different (full fine-tune) checkpoint, plus 12 "commit to an edit" states where the agent had explored for 30+ tool calls without editing and the teacher chose to edit in at least 2 of 4 samples. Held-out pair accuracy 0.96.

Usage

vllm serve Jitheswar/Qwen3.5-4B-Salieri --max-model-len 65536 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3

Recommended sampling: temperature 0.6, top_p 0.95, top_k 20 (thinking). Works with any OpenAI-compatible agent harness that uses native tool calls. GGUF quantizations (BF16, Q8_0, Q6_K, Q4_K_M + mmproj) are included for llama.cpp / LM Studio / Ollama.

Limitations

A 4B model: it solves a minority of real repository issues, can loop on hard tasks, and was evaluated mainly in one docker-free harness (plus the bash-only tool-set check above). Abliterated base: it does not refuse much; deploy with your own safeguards.

Credits

Base: Qwen team (Qwen3.5-4B, Apache-2.0), abliteration: Qwen3.5-4B-Amadeus.
Teacher: Qwen3.8-27B (Apache-2.0).
Data: nvidia/Open-SWE-Traces (CC-BY-4.0; tasks from Scale-SWE and SWE-rebench-V2) — Ahmad et al., Open-SWE-Traces, arXiv:2606.16038.
Methods: iterated DPO (Rafailov et al. 2023) with an NLL term (RPO, Pang et al. 2024), applied at the step level to agent states. Evaluation harness tuned with Harness2 (google-research/harness2, arXiv:2610.14224) on training-pool tasks only. Training with Unsloth.