ECP-SB-27B-Research

🇰🇷 한국어 카드: MODEL_CARD.ko.md

Research Deployment — Not Production Certified. This package passed a single-GPU functional deployment check (generation and evidence gate served from one load of the base model). The gate model inside it has a pooled unsupported-answer rate (UAR) of 6.5% on its independent Final read; the 5% target was not met. Do not use the gate as a safety control.

One GPU, one load of Qwen/Qwen3.8-27B, two APIs: ordinary generation and an evidence gate that decides, after retrieval and before generation, whether to ANSWER, RETRIEVE_MORE or ABSTAIN. This repository holds what the shared engine adds on top of the base model: the four ECP-Q12 seed files (LoRA r 8 / alpha 16 on the MLPs of decoders 44 to 47, a 3-way action head, a sufficiency head), each seed's sealed threshold tau, and a manifest. The base weights are not copied here; they are loaded from Qwen/Qwen3.8-27B.

Serving code (vLLM plugin, gate front, Docker, examples): github.com/ThakiCloud/ecp-shared-backbone-vllm, tag v0.1.0, which matches this repository's revision v0.1.0.

Use

git clone https://github.com/ThakiCloud/ecp-shared-backbone-vllm && cd ecp-shared-backbone-vllm
pip install --no-deps .        # inside an environment with vllm==0.28.0 (or use docker/Dockerfile)
./serve_sb.sh                  # downloads Qwen/Qwen3.8-27B and this repo (v0.1.0) if BASE / ECP_DIR are unset

curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model": "ecp-sb",
  "messages": [{"role": "user", "content": "Explain RAG in two sentences."}], "max_tokens": 128,
  "chat_template_kwargs": {"enable_thinking": false}}'
curl -s localhost:8100/v1/gate -H 'Content-Type: application/json' -d '{"question": "Who wrote the novel?",
  "passages": [{"title": "Note", "text": "The novel was written by A. Writer."}], "retrieval_available": true}'

The gate returns action, raw_action, margin, tau and probs. Rule: m = log P(ANSWER) - max(log P(RETRIEVE_MORE), log P(ABSTAIN)), ANSWER iff m >= tau, otherwise the more likely of the other two. Inputs over 2,048 prompt tokens are rejected.

Files

File	Content
weights/ecp_q12_s{1,2,3,4}.safetensors	One file per training seed: adapter.* (PEFT LoRA names), head.*, sufficiency.*. Identical bytes to ECP-Q12-27B-Research-Preview
MANIFEST.json	Per-seed sha256 and sealed tau, gate layers 44 to 47, label tokens, verified engine

seed	sealed tau	Final UAR (ECP-Q12 card)
s1 (checked in the deployment)	-0.236	.055
s2	1.507	.061
s3	1.865	.093
s4	1.707	.047
pooled		.065

Gate accuracy, coverage and the per-state answer rates are those of ECP-Q12 and are documented in its card; nothing about the gate model was retrained here.

Deployment check (single run, frozen criteria)

1× H200-NVL, vLLM 0.28.0, the flags in serve_sb.sh, seed s1. Reference: unmodified vLLM serving the same base model with identical flags over the same HTTP path.

Check	Result
Base weights loaded once, one GPU process	PASS
Generation API and gate API both answer	PASS
Generation-only concurrent traffic vs stock vLLM (64 prompts)	PASS: 2/64 divergent vs 1/64 stock-vs-stock; p99 logprob gap .18749142 ≤ bound .18749428 (margin 0.00000286)
Alternating generation and gate requests vs stock vLLM (16 prompts)	PASS: 0/16 divergent
Gate decisions under mixed traffic (256 items)	PASS: 0/256 changed vs offline reference
Idle 600 s; restart with the same configuration	PASS
Generation under mixed traffic identical to stock	Not shown: 6/64 prompts differ from stock vLLM (p99 gap .25); cause not attributed
Production safety	Not certified (UAR 6.5% vs 5%)
Throughput, sustained load, cost	Not measured

The p99 criterion passed by a very small margin. Two earlier runs under previous criteria versions failed and remain on record. The check covers one fixed workload (64 generation prompts, 16 alternating prompts, 256 gate items).

Limitations

• Verified on vLLM 0.28.0 only. A vLLM 0.31 port is planned and not verified.
• The gate branch runs for every scheduled token in v0.1.0, and its 4 layers are a weight copy.
• The gate model was trained and evaluated on synthetic English multi-hop data; transfer to real documents is not measured. Partial-evidence states are still answered roughly 4% to 10% of the time depending on seed.
• Do not treat mixed-traffic generation as bit-identical to the base model.

License

Apache-2.0, same as the base model.