SeaWolf-AI commited on
Commit
098aa81
·
verified ·
1 Parent(s): 0d052fd

Upload article.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. article.md +58 -57
article.md CHANGED
@@ -1,76 +1,77 @@
1
  ---
2
  title: "FINAL-Bench Quantum: An Open, Neutral Benchmark for Quantum-Computing Methods"
3
- thumbnail:
4
  authors:
5
  - user: SeaWolf-AI
6
- tags: [quantum, quantum-error-correction, benchmark, qec, decoders, vqe, qram]
7
  ---
8
 
9
  # FINAL-Bench Quantum: An Open, Neutral Benchmark for Quantum-Computing Methods
10
 
11
- Quantum-computing results are notoriously hard to compare. A "logical error rate," a
12
- "query fidelity," or an "approximation ratio" only mean something next to a clearly stated
13
- protocol — yet papers use different codes, noise models, hardware, and shot counts. **FINAL-Bench
14
- Quantum** is our attempt at a single, fair yardstick: an open, neutral benchmark suite where
15
- methods compete in five events under identical, published protocols, and where every number is
16
- labeled as either *measured here* or *quoted from a source*.
17
 
18
  🔗 **Leaderboard:** `huggingface.co/spaces/FINAL-Bench/quantum-bench-leaderboard`
19
 
20
- ## Two tracks, one rule: say exactly what you measured
21
 
22
- - **Track A Verified.** Methods run here on one frozen, public test set, reported with 95%
23
- confidence intervals. These are directly comparable.
24
- - **Track B Reported.** Numbers quoted from their papers/sources. Codes, noise models, and
25
- hardware differ, so they are **not** directly comparable — and we say so.
26
 
27
- We make **no quantum-advantage claims**. Where a result is a simulation, we call it a simulation;
28
- where it is real hardware, we name the chip and job ID. Where two results are within their
29
- confidence intervals, we call it a statistical tie rather than crowning a winner.
 
 
 
30
 
31
  ## The five events
32
 
33
- | Event | What it measures | Verified highlight |
34
  |---|---|---|
35
- | ① QEC Decoder | logical error rate on a rotated surface code (Stim, circuit noise) | a maximum-likelihood decoder (Tesseract) leads; the BP-class decoders (BeliefMatching, BP+OSD) and a neural ensemble are a statistical tie just behind |
36
- | ② Optimization | Max-Cut quality (cut / optimum) | QAOA reaches ~0.88 approximation ratio at depth p=3 on a 14-node instance |
37
- | ③ VQE | molecular ground-state energy vs FCI | H₂ to **chemical accuracy** (0.22 mHa); a real-hardware 4-qubit run on IBM Heron r2 |
38
- | ④ QRAM | quantum-memory query fidelity | a 4-cell select-QRAM on **IBM Heron r2 at 0.92 mean query fidelity**; a QEC-protected variant reaches 99.6% in simulation |
39
- | ⑤ Simulation | how large a circuit a classical method handles | **100,000-qubit** Clifford (stabilizer) sampling; 30-qubit exact statevector on a GPU |
40
-
41
- ## A few results we found interesting
42
-
43
- **Decoders are near a floor on idealized syndromes.** On a distance-5 rotated surface code, the
44
- strong classical decoders (BP+OSD ≈ 0.0095, BeliefMatching ≈ 0.010) and a regularized neural
45
- ensemble (≈ 0.0098) all sit within each other's 95% CIs — only the brute-force maximum-likelihood
46
- decoder (Tesseract, 0.0092) is lower, and it runs at ~79 ms/shot, far too slow for real-time use.
47
- The practical lesson: on *binary* syndromes there is little headroom; a fancier architecture does
48
- not beat the floor.
49
-
50
- **…but soft information changes the game.** When the decoder is given *analog* (soft) readout
51
- values instead of thresholded bits, a small neural decoder cuts logical error by **40–46%** versus
52
- hard-decision decoding on a repetition memory matching the soft-optimal decoder. This is the same
53
- mechanism behind Google's AlphaQubit gains: the advantage is in the *information*, not the network.
54
-
55
- **Real hardware, honestly bounded.** On IBM Heron r2 processors, a repetition-code memory shows the
56
- error-correcting distance boundary directly — distance-3 fails to correct two injected errors
57
- (~97% logical error) while distance-5 corrects them (0.5–3%), a 29–175× reduction across three
58
- backends (z = 7–552σ). This is a *repetition code* with offline majority-vote decoding **not**
59
- surface-code below-threshold, not fault-tolerant QEC, and not a quantum-advantage claim.
60
-
61
- ## Submit your method
62
-
63
- The **Submit** tab takes a method name, links (GitHub / Hugging Face), an email, and an optional
64
- file. Submissions are stored privately, reproduced under the event's fixed protocol, and the
65
- submitter is emailed about inclusion. Entries are labeled by origin (country flag + author/group),
66
- exactly like every other entry.
67
-
68
- ## Why neutrality is the point
69
-
70
- A leaderboard is only useful if you trust it. That means including strong competitors even when
71
- they beat the host's own entries, quoting sources faithfully, and refusing to round a simulation up
72
- into a hardware claim. FINAL-Bench Quantum is built around that discipline. Several entries come
73
- from our own group (VIDRAFT, 🇰🇷); they are shown under the same protocol, confidence intervals, and
74
- honesty boundaries as Google, IBM, NVIDIA, USTC, Riverlane, and the rest.
75
 
76
  *A methods paper is in preparation. Feedback and submissions are welcome.*
 
1
  ---
2
  title: "FINAL-Bench Quantum: An Open, Neutral Benchmark for Quantum-Computing Methods"
3
+ thumbnail:
4
  authors:
5
  - user: SeaWolf-AI
6
+ tags: [quantum, quantum-error-correction, benchmark, qec, vqe, qram]
7
  ---
8
 
9
  # FINAL-Bench Quantum: An Open, Neutral Benchmark for Quantum-Computing Methods
10
 
11
+ Quantum-computing results are remarkably hard to compare. The same "logical error rate (LER)" or
12
+ "query fidelity" can mean entirely different things depending on the code, the noise model, the
13
+ hardware, and how many shots were taken. **FINAL-Bench Quantum** is our attempt to bring *one fair
14
+ yardstick* to that confusion: a suite where methods compete in five events under **identical,
15
+ published protocols**, and where every number is clearly labeled as either *measured here* or
16
+ *quoted from a source*.
17
 
18
  🔗 **Leaderboard:** `huggingface.co/spaces/FINAL-Bench/quantum-bench-leaderboard`
19
 
20
+ ## The core rule two tracks
21
 
22
+ - **Track A (Verified).** Methods are *measured here* on one frozen, public test set and reported
23
+ with 95% confidence intervals. These numbers **are** directly comparable.
24
+ - **Track B (Reported).** Numbers *quoted* from each paper or announcement. Codes, noise models,
25
+ and hardware differ, so they are **not** directly comparable — and we say so plainly.
26
 
27
+ Two principles hold throughout:
28
+
29
+ 1. **No quantum-advantage claims.**
30
+ 2. **A simulation is labeled a simulation; real hardware is named with its chip.** When two results
31
+ fall within each other's confidence intervals, we call it a **statistical tie** rather than
32
+ crowning a winner.
33
 
34
  ## The five events
35
 
36
+ | Event | What it measures | One-line analogy |
37
  |---|---|---|
38
+ | ① **QEC Decoder** | logical error rate on a rotated surface code (Stim, circuit noise) | accuracy of a quantum "spell-checker" |
39
+ | ② **Optimization** | Max-Cut quality (cut found / optimum) | finding the best answer among astronomically many |
40
+ | ③ **VQE** | molecular ground-state energy vs the exact solution | quantum energy calculation for chemistry/drugs |
41
+ | ④ **QRAM** | quantum-memory query fidelity | accuracy of a quantum "memory chip" |
42
+ | ⑤ **Simulation** | how large a circuit a classical method can handle | faking a quantum computer on a classical one |
43
+
44
+ Each event tab is organized as **A. verified measurements / B. real hardware (where available) /
45
+ C. published references**, alongside dedicated **📈 Charts** (threshold, distance-scaling, and
46
+ latency-vs-accuracy plots), **🏅 Medals** (participation by country), and **ℹ️ About**
47
+ (methodology and citation).
48
+
49
+ ## How to read the tables
50
+
51
+ - **Flag** = a method's / team's origin; **By** = its authors (e.g., Tesseract = Google Quantum AI,
52
+ PyMatching = O. Higgott).
53
+ - ** VERIFIED** = measured on this benchmark. **REPORTED** = quoted from a source.
54
+ - The **±value** next to a number is its 95% confidence interval. Overlapping intervals mean a tie.
55
+ - The **latency** column matters as much as accuracy a decoder that is "accurate but slow" can be
56
+ useless for real-time error correction, where decoding must keep pace with the QPU cycle.
57
+
58
+ ## How to submit
59
+
60
+ The **📤 Submit** tab takes a method name, links (GitHub / Hugging Face), an email, and an optional
61
+ results file. Submissions are **stored privately**, reproduced under the event's fixed protocol, and
62
+ the submitter is emailed about inclusion. Listed entries appear with the same origin and author
63
+ labels as everyone else.
64
+
65
+ ## Why neutrality is the whole point
66
+
67
+ A leaderboard is only useful if you can trust it. So FINAL-Bench Quantum is built on a discipline:
68
+ **include strong competitors even when they beat the host's own entries, quote sources faithfully,
69
+ and never round a simulation up into a hardware claim.** Methods from Google, IBM, NVIDIA, USTC,
70
+ Riverlane and others sit next to a Korean entry (VIDRAFT 🇰🇷) under the *same* protocol, confidence
71
+ intervals, and honesty boundaries.
72
+
73
+ Quantum computing has not yet reached the fault-tolerant era which is exactly why a shared,
74
+ hype-free yardstick that honestly records *what has actually been measured today* is worth building.
75
+ Come and compete with your own method.
 
 
76
 
77
  *A methods paper is in preparation. Feedback and submissions are welcome.*