Instructions to use NewstaR/Newstar-Qwen3-0.6B-KTO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NewstaR/Newstar-Qwen3-0.6B-KTO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NewstaR/Newstar-Qwen3-0.6B-KTO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NewstaR/Newstar-Qwen3-0.6B-KTO")
model = AutoModelForCausalLM.from_pretrained("NewstaR/Newstar-Qwen3-0.6B-KTO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NewstaR/Newstar-Qwen3-0.6B-KTO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NewstaR/Newstar-Qwen3-0.6B-KTO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NewstaR/Newstar-Qwen3-0.6B-KTO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NewstaR/Newstar-Qwen3-0.6B-KTO

SGLang

How to use NewstaR/Newstar-Qwen3-0.6B-KTO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NewstaR/Newstar-Qwen3-0.6B-KTO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NewstaR/Newstar-Qwen3-0.6B-KTO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NewstaR/Newstar-Qwen3-0.6B-KTO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NewstaR/Newstar-Qwen3-0.6B-KTO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use NewstaR/Newstar-Qwen3-0.6B-KTO with Docker Model Runner:
```
docker model run hf.co/NewstaR/Newstar-Qwen3-0.6B-KTO
```

marcuscedricridia commited on Jun 8, 2025

Commit

bf0a86e

verified ·

1 Parent(s): d772d79

Update README.md

Browse files

Files changed (1) hide show

README.md +17 -17

README.md CHANGED Viewed

@@ -27,20 +27,20 @@ This version remains in non‑thinking mode—built for consistent and bias‑aw
 > - Do Sample: True
 > - Max New Tokens: 4096
-| Category              | Winner | Reason                                                                 |
-|-----------------------|--------|----------------------------------------------------------------------|
-| CS (RAM vs. ROM)      | KTO    | KTO is clearer, more structured, and avoids inaccuracies like BASE’s claim about excessive RAM. |
-| ENGINEERING (Water Filtration) | KTO    | KTO provides a practical, scientifically grounded system; BASE is confusing and impractical. |
-| MATH (Mean, Median, Mode) | KTO    | KTO’s structured, concise explanation outperforms BASE’s wordy but accurate response. |
-| SCIENCE (Osmosis vs. Diffusion) | KTO    | KTO is more detailed and accurate despite a minor error; BASE oversimplifies and has vague examples. |
-| WRITING (Lost Dog Story) | BASE   | BASE focuses on the dog and partially meets the prompt; KTO is off-topic and incoherent. |
-| CODING (Vowel Counting) | BASE   | BASE’s program is more robust (handles uppercase/lowercase) and includes test cases; KTO misses uppercase vowels. |
-| MATH SOLVING (Train Speed) | KTO    | Both are accurate, but KTO is more concise, delivering the result with less verbosity. |
-| COMMON SENSE LOGIC (Ice Melting) | KTO    | KTO accurately describes melting; BASE’s sublimation claim is incorrect. |
-| SOFT REASONING (Dog Barking) | BASE   | BASE provides a clearer affirmation despite flaws; KTO overcomplicates and undermines the premise. |
-| RIDDLE (Keys and Locks) | Neither | Both fail to identify the correct answer (piano) and provide irrelevant explanations. |
-| GENERAL CHAT (Hobby)  | BASE   | BASE’s detailed, engaging piano description outperforms KTO’s brief, shallow list. |
-| REWRITING (Formal Sentence) | KTO    | KTO’s rewrite is concise and equally formal; BASE is wordy with unnecessary alternatives. |
-| SUMMARIZATION (Tortoise and Hare) | KTO    | KTO is accurate and concise; BASE has factual errors (e.g., ten-day race). |
-| INSTRUCTION FOLLOWING (Vegetable Soup) | KTO    | KTO adheres closely to the prompt with clear, healthy steps; BASE misinterprets and lacks clarity. |
-| **Overall**           | **KTO** | KTO wins 9 categories vs. BASE’s 4, showing greater accuracy, clarity, and adherence to prompts. |

 > - Do Sample: True
 > - Max New Tokens: 4096
+| Category              | Prompt                                                                 | Winner | Reason                                                                 |
+|-----------------------|----------------------------------------------------------------------|--------|----------------------------------------------------------------------|
+| CS (RAM vs. ROM)      |                                                                      | KTO    | KTO is clearer, more structured, and avoids inaccuracies like BASE’s claim about excessive RAM. |
+| ENGINEERING (Water Filtration) |  | KTO    | KTO provides a practical, scientifically grounded system; BASE is confusing and impractical. |
+| MATH (Mean, Median, Mode) |                                                                      | KTO    | KTO’s structured, concise explanation outperforms BASE’s wordy but accurate response. |
+| SCIENCE (Osmosis vs. Diffusion) |                                                                      | KTO    | KTO is more detailed and accurate despite a minor error; BASE oversimplifies and has vague examples. |
+| WRITING (Lost Dog Story) | Write a short story about a lost dog finding its way home. | BASE   | BASE focuses on the dog and partially meets the prompt; KTO is off-topic and incoherent. |
+| CODING (Vowel Counting) | Create a simple program that counts the number of vowels in a sentence. | BASE   | BASE’s program is more robust (handles uppercase/lowercase) and includes test cases; KTO misses uppercase vowels. |
+| MATH SOLVING (Train Speed) | If a train travels 60 miles in 1.5 hours, what is its average speed? | KTO    | Both are accurate, but KTO is more concise, delivering the result with less verbosity. |
+| COMMON SENSE LOGIC (Ice Melting) | If you leave ice outside on a hot day, what happens to it? | KTO    | KTO accurately describes melting; BASE’s sublimation claim is incorrect. |
+| SOFT REASONING (Dog Barking) | If all dogs bark and Rex is a dog, does Rex bark? Why? | BASE   | BASE provides a clearer affirmation despite flaws; KTO overcomplicates and undermines the premise. |
+| RIDDLE (Keys and Locks) | What has keys but can’t open locks? | Neither | Both fail to identify the correct answer (piano) and provide irrelevant explanations. |
+| GENERAL CHAT (Hobby)  | Tell me about a hobby you enjoy. | BASE   | BASE’s detailed, engaging piano description outperforms KTO’s brief, shallow list. |
+| REWRITING (Formal Sentence) | Make this sentence more formal: “Can you fix the problem soon?” | KTO    | KTO’s rewrite is concise and equally formal; BASE is wordy with unnecessary alternatives. |
+| SUMMARIZATION (Tortoise and Hare) | Summarize the story of “The Tortoise and Hare” in two sentences. | KTO    | KTO is accurate and concise; BASE has factual errors (e.g., ten-day race). |
+| INSTRUCTION FOLLOWING (Vegetable Soup) | Explain how to prepare a simple vegetable soup that meets the following conditions: Use at least 3 different vegetables. The cooking time must not exceed 30 minutes. Include steps to make the soup both flavorful and healthy. Mention any kitchen tools needed. Provide alternatives if a vegetable is not available. Include tips to serve the soup nicely. | KTO    | KTO adheres closely to the prompt with clear, healthy steps; BASE misinterprets and lacks clarity. |
+| **Overall**           |                                                                      | **KTO** | KTO wins 9 categories vs. BASE’s 4, showing greater accuracy, clarity, and adherence to prompts. |