Upload article.md with huggingface_hub
Browse files- article.md +76 -0
article.md
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: "FINAL-Bench Quantum: An Open, Neutral Benchmark for Quantum-Computing Methods"
|
| 3 |
+
thumbnail:
|
| 4 |
+
authors:
|
| 5 |
+
- user: SeaWolf-AI
|
| 6 |
+
tags: [quantum, quantum-error-correction, benchmark, qec, decoders, vqe, qram]
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# FINAL-Bench Quantum: An Open, Neutral Benchmark for Quantum-Computing Methods
|
| 10 |
+
|
| 11 |
+
Quantum-computing results are notoriously hard to compare. A "logical error rate," a
|
| 12 |
+
"query fidelity," or an "approximation ratio" only mean something next to a clearly stated
|
| 13 |
+
protocol — yet papers use different codes, noise models, hardware, and shot counts. **FINAL-Bench
|
| 14 |
+
Quantum** is our attempt at a single, fair yardstick: an open, neutral benchmark suite where
|
| 15 |
+
methods compete in five events under identical, published protocols, and where every number is
|
| 16 |
+
labeled as either *measured here* or *quoted from a source*.
|
| 17 |
+
|
| 18 |
+
🔗 **Leaderboard:** `huggingface.co/spaces/FINAL-Bench/quantum-bench-leaderboard`
|
| 19 |
+
|
| 20 |
+
## Two tracks, one rule: say exactly what you measured
|
| 21 |
+
|
| 22 |
+
- **Track A — Verified.** Methods run here on one frozen, public test set, reported with 95%
|
| 23 |
+
confidence intervals. These are directly comparable.
|
| 24 |
+
- **Track B — Reported.** Numbers quoted from their papers/sources. Codes, noise models, and
|
| 25 |
+
hardware differ, so they are **not** directly comparable — and we say so.
|
| 26 |
+
|
| 27 |
+
We make **no quantum-advantage claims**. Where a result is a simulation, we call it a simulation;
|
| 28 |
+
where it is real hardware, we name the chip and job ID. Where two results are within their
|
| 29 |
+
confidence intervals, we call it a statistical tie rather than crowning a winner.
|
| 30 |
+
|
| 31 |
+
## The five events
|
| 32 |
+
|
| 33 |
+
| Event | What it measures | Verified highlight |
|
| 34 |
+
|---|---|---|
|
| 35 |
+
| ① QEC Decoder | logical error rate on a rotated surface code (Stim, circuit noise) | a maximum-likelihood decoder (Tesseract) leads; the BP-class decoders (BeliefMatching, BP+OSD) and a neural ensemble are a statistical tie just behind |
|
| 36 |
+
| ② Optimization | Max-Cut quality (cut / optimum) | QAOA reaches ~0.88 approximation ratio at depth p=3 on a 14-node instance |
|
| 37 |
+
| ③ VQE | molecular ground-state energy vs FCI | H₂ to **chemical accuracy** (0.22 mHa); a real-hardware 4-qubit run on IBM Heron r2 |
|
| 38 |
+
| ④ QRAM | quantum-memory query fidelity | a 4-cell select-QRAM on **IBM Heron r2 at 0.92 mean query fidelity**; a QEC-protected variant reaches 99.6% in simulation |
|
| 39 |
+
| ⑤ Simulation | how large a circuit a classical method handles | **100,000-qubit** Clifford (stabilizer) sampling; 30-qubit exact statevector on a GPU |
|
| 40 |
+
|
| 41 |
+
## A few results we found interesting
|
| 42 |
+
|
| 43 |
+
**Decoders are near a floor on idealized syndromes.** On a distance-5 rotated surface code, the
|
| 44 |
+
strong classical decoders (BP+OSD ≈ 0.0095, BeliefMatching ≈ 0.010) and a regularized neural
|
| 45 |
+
ensemble (≈ 0.0098) all sit within each other's 95% CIs — only the brute-force maximum-likelihood
|
| 46 |
+
decoder (Tesseract, ≈ 0.0092) is lower, and it runs at ~79 ms/shot, far too slow for real-time use.
|
| 47 |
+
The practical lesson: on *binary* syndromes there is little headroom; a fancier architecture does
|
| 48 |
+
not beat the floor.
|
| 49 |
+
|
| 50 |
+
**…but soft information changes the game.** When the decoder is given *analog* (soft) readout
|
| 51 |
+
values instead of thresholded bits, a small neural decoder cuts logical error by **40–46%** versus
|
| 52 |
+
hard-decision decoding on a repetition memory — matching the soft-optimal decoder. This is the same
|
| 53 |
+
mechanism behind Google's AlphaQubit gains: the advantage is in the *information*, not the network.
|
| 54 |
+
|
| 55 |
+
**Real hardware, honestly bounded.** On IBM Heron r2 processors, a repetition-code memory shows the
|
| 56 |
+
error-correcting distance boundary directly — distance-3 fails to correct two injected errors
|
| 57 |
+
(~97% logical error) while distance-5 corrects them (0.5–3%), a 29–175× reduction across three
|
| 58 |
+
backends (z = 7–552σ). This is a *repetition code* with offline majority-vote decoding — **not**
|
| 59 |
+
surface-code below-threshold, not fault-tolerant QEC, and not a quantum-advantage claim.
|
| 60 |
+
|
| 61 |
+
## Submit your method
|
| 62 |
+
|
| 63 |
+
The **Submit** tab takes a method name, links (GitHub / Hugging Face), an email, and an optional
|
| 64 |
+
file. Submissions are stored privately, reproduced under the event's fixed protocol, and the
|
| 65 |
+
submitter is emailed about inclusion. Entries are labeled by origin (country flag + author/group),
|
| 66 |
+
exactly like every other entry.
|
| 67 |
+
|
| 68 |
+
## Why neutrality is the point
|
| 69 |
+
|
| 70 |
+
A leaderboard is only useful if you trust it. That means including strong competitors even when
|
| 71 |
+
they beat the host's own entries, quoting sources faithfully, and refusing to round a simulation up
|
| 72 |
+
into a hardware claim. FINAL-Bench Quantum is built around that discipline. Several entries come
|
| 73 |
+
from our own group (VIDRAFT, 🇰🇷); they are shown under the same protocol, confidence intervals, and
|
| 74 |
+
honesty boundaries as Google, IBM, NVIDIA, USTC, Riverlane, and the rest.
|
| 75 |
+
|
| 76 |
+
*A methods paper is in preparation. Feedback and submissions are welcome.*
|