TanyaoDojo — Mahjong AI checkpoints

JAX/Flax checkpoints from TanyaoDojo, a from-scratch mahjong AI stack (vectorized env + behavior cloning + self-play RL). Every number below comes from the same duplicate 1v3 protocol: the challenger plays all four seats over identical walls against three copies of a strong open-source baseline (Mortal v4), seed_key=20260711, seeds from 10000, placement points [90, 45, 0, -135].

avg_pt is the challenger's mean placement points relative to the baseline; 0 would mean parity. Higher is better.

The headline checkpoint has a 100k-game milestone measurement (-4.66 +/- 0.535, 16.0M decisions, zero fallbacks) — not a small-sample estimate.

Checkpoints

File	Arch	Obs	Training	avg_pt vs baseline
bc_v2_g186.pkl	256ch x 10blk (12.4M)	v2 (34x36 + 32)	BC on 10y logs + LR-1e-4 refine	-4.66 +/- 0.535 (100k games)
bc_lean_g402.pkl	256ch x 10blk (12.4M)	lean (34x20 + 26)	BC on 14y logs + LR-1e-4 refine	-5.07 +/- 1.54 (12k games)
bc_lean_w192_ep2.pkl	192ch x 8blk (5.3M)	lean	BC on 6y logs, 2 epochs	-8.87 +/- 2.70 (4k games)
rl_oracle_800m.pkl	256ch x 10blk	lean (actor)	oracle-critic PPO league, 0.8B steps	-8.38 +/- 2.65 (4k) — negative result, wrong objective (see below)

The last row is published deliberately, and its interpretation was corrected on 2026-08-23 — the correction is more useful than the checkpoint.

An asymmetric actor-critic whose critic sees all four hands fit the value function ~100x better (v_loss 0.102 -> 0.001) yet lost 3.2pt of external strength. It was the fourth consecutive negative RL result here. We originally read this as "oracle critics don't transfer". That reading was wrong.

The real cause: the training objective was not the evaluation objective. All four RL runs used round_mode="single", which ends the episode after a single hand, and the reward is mahjong's raw point transfer. The arena scores an entire hanchan by placement points [90, 45, 0, -135]. Measured directly, not inferred from code: 600 steps x 256 envs produced 1541 episodes (one hand each); rewards ranged [-120, 130] with rows like [+30, -10, -10, -10] summing to ~0 (point transfers); and mahjax writes order_points only into the final score, never into rewards, in any round_mode.

A single-hand point maximizer has no concept of 4th place — falling to last costs it nothing extra, so it should be maximally aggressive. This predicts that more RL steps make the arena result worse (observed), and it explains why a better critic hurt more: it converged faster onto the wrong objective.

So this checkpoint is not evidence against oracle critics. It is evidence that we optimized the wrong function for ~10^10 environment steps. The fix — terminal placement-aligned reward over full hanchan — is in the repo as jax_rl/reward_placement.py, and all four RL results above are being redone on it.

First result after the fix (1B steps from the -4.66 base, 11 arena evals of 1600 games each along the way): -4.88 +/- 0.81 — statistically flat versus the base. The previous four runs lost 3.2 to 10 points each and got worse the longer they ran. The degradation is gone; the gains are not there yet, and the metrics say why — approx_kl about 4e-5 per update and only 0.011 KL of drift from the base after a billion steps, i.e. a tight trust region plus roughly 0.75 hanchan terminals per rollout left the policy almost unmoved. The next step is therefore signal density, not a different algorithm: GRP potential shaping (a 7.5M-sample "position -> expected final placement points" potential, reward becomes phi(s') - phi(s) with the terminal remainder), which is policy-invariant by the shaping theorem — verified numerically — and raises the fraction of non-zero-reward steps from 0.1% to 1.10%.

If you are porting an RL recipe (ours or anyone's): check that its episode boundary and reward definition are the same function your benchmark scores.

Shaped run, first segment closed at 1.48B steps (2026-09-04). Drift from the base reached 0.023 KL versus 0.011 at 1B in the unshaped run — the policy moved about twice as far. Two checkpoints have been adjudicated at the full 12k-game protocol: -3.79 +/- 1.54 at 0.54B and -4.04 +/- 1.54 at 1.48B, i.e. +0.87 and +0.62 against the -4.66 base, at 1.05 and 0.74 sigma. Neither is significant. Pooling everything measured on this run (~36k games, six checkpoints, inverse-variance) gives -3.78 +/- 0.85, or +0.88 +/- 1.00 over the base — z = 1.73. Suggestive, not proven, and we are not going to call it more than that.

Both adjudications tell the same story about small samples: the 4k readings that preceded them (-3.50 and -3.57, and one wild -1.29 at 1600 games) were all optimistic, and all pulled back. That is now five for five here. Single-segment readings are for screening only; nothing gets claimed without the 12k double-segment protocol.

We also audited our own confidence intervals, and they were wrong — conservatively. The eval harness computes its interval as 1.96 * std(per-game points) / sqrt(n) with n counting every game. But duplicate evaluation plays the same deal four times with rotated seats, so those four games are not independent samples — that dependence is the entire point of the design. Recomputing over 2,000 deals from a real 8k-game segment gives an intra-class correlation of -0.079 and a deal-level interval 0.873x the per-game one. The rotation really does cancel seat luck, so our published intervals have been about 13% too wide: the true resolution is +/-2.36 at 4k, +/-1.34 at 12k and +/-0.467 at 100k. Every significance call we have made was therefore conservative, and none of them flips.

Getting that number required fixing a second bug in the audit script itself: mjai's reach_accepted event carries no deltas, so summing only hora/ryukyoku deltas silently omits every 1000-point riichi deposit and misranks close games. The tell was a self-check we had built in — the recomputed segment average has to reproduce the harness's own printed value, and it came out 0.47pt off until the deposits were accounted for. Instrument: jax_rl/mjai_bot/group_ci.py.

The cleanest comparison we can make, and what it cost us to learn. With training paused we replayed the base g186 over the same 2,000 deals the 1.48B-step shaped checkpoint had just played, and compared them deal by deal. Pairing on identical deals correlates the two models at r = +0.307 and removes 30.7% of the variance. The answer: +0.636 pt in favour of the RL checkpoint, 95% CI +/-1.90, z = 0.66. The effect is on the positive side in the most direct measurement available, and 8,000 games cannot resolve it.

That result is worth more as a budgeting lesson than as a score. Inverting the same standard error: proving a +0.64pt effect at 2 sigma needs about 71,500 games per side, roughly 86 hours of CPU. Proving a +2pt effect needs 7,200 games per side, about 9 hours. Making the effect bigger is an order of magnitude cheaper than measuring the small one. So the next move is not more evaluation games — it is loosening the trust region (clip 0.02 -> 0.05) so the policy can move far enough to produce an effect our instruments can actually see. The intra-class correlation also replicated across the two models (-0.079 and -0.093), which is why we now trust the 0.87 correction factor.

And then we checked the premise behind that plan, and it did not hold. The argument for loosening the trust region rested on a KL number: drift from the anchor was only 0.023 nats after 1.48B steps, so "the policy has barely moved." We had never translated that into behaviour. Doing so: on identical states the RL checkpoint and the base disagree on their top-1 action 3.10% of the time — about 30 decisions per hanchan. That is not "barely moved."

The calibration that settles it: the earlier oracle-critic run, the one trained on the wrong objective that lost 3.3pt, disagrees with its base on 5.30% of decisions. So a run we describe as having drifted badly moved only 1.7x further than the current one. Distance from the human-imitation base does not predict strength; direction does. We are recording this as a correction to our own reasoning rather than quietly dropping it, and the loosening experiment is no longer justified by "the anchor is the bottleneck" — if we run it, it will be as an honest coin-flip.

The instrument has a second use that is worth more than the finding: policy_shift.py reads out how far a config change actually moves the policy in a couple of minutes, where the arena evaluation that answers the same question takes nine hours.

The Sichuan line cleared its first RL gate

The second line in this project trains 血战到底 (Sichuan bloody mahjong) from scratch — no human data, no behaviour-cloned base, no anchor. Its phase-2 gate was "beat the hand-written rule ladder's L1 within 300M steps." Measured at 105M steps, duplicate 1v3 with the challenger rotating through all four seats on identical deals:

Challenger	vs 3x L0, 300 deals
Hand-written L1 (greedy shanten)	+3.949 +/- 0.355
From-scratch RL, 105M steps	+6.872 +/- 0.500
Paired difference, same deals	+2.922 +/- 0.565, z = 10.1

Cleared at 35% of the step budget. Note how cheap this measurement was compared to the riichi line: 300 deals sufficed because the effect is +2.9pt. That is the budgeting lesson from the other line, seen from the good side.

Two things kept us honest here. The evaluation bridges a JAX shadow state alongside the Python reference implementation the rule bots are written against — and if that shadow ever desynchronises, the network is choosing moves for the wrong position while the scores still come out looking fine. So every network decision re-checks the legal-action sets: 58,132 decisions, 0 mismatches, 0 fallbacks. And the baseline was re-measured through the identical code path on the identical deals (+3.949 against the +3.867 on record) rather than quoted from a document.

The result also comes with a defect we are not burying: the policy's entropy has collapsed to 0.048 nats, about 1.05 effective actions, with per-state max ratio reaching 3-6. Every averaged metric looks healthy; only the tail instruments show it. The phase-2 gate does not test entropy, but a near-deterministic policy is unlikely to survive the adapting opponent pool that phase 3 requires.

One claim from our own diagnosis, refuted by measurement. A survey we were working from asserted that a single hand correlates with final placement at rho < 0.15, and concluded that group-relative advantage at hand granularity is hopeless. Measured over 708k hanchan (27.2M hand-by-player samples): rho = 0.235, about 57% higher than claimed. The number is wrong; the direction survives — R^2 = 0.055 means one hand explains 5.5% of placement variance, so group-relative routes are inefficient rather than impossible. Script: scripts/rho_kyoku.py in the repo. We publish this because a diagnosis document is not evidence until you check it.

A second claim, and an instrument that failed first. The same document flagged its own uncertainty about which way the entropy coefficient should move. Two facts settle it as a non-lever here. Our arena play is already greedy (masked argmax), so the entropy bonus never spends evaluated points. And under the policy's own centralized Q-critic, sweeping the sampling temperature over a 200x range (tau 0.01 to 2.0) moves expected value by at most 0.0004 placement points per decision, with every confidence interval covering zero — the policy's support is value-flat, so its stochasticity is close to free. Entropy itself has sat at 0.50 +/- 0.01 nats for 1.3B steps.

Worth recording how we nearly got this wrong: the first version of the instrument used max_a Q(s,a) as the baseline and reported an "entropy tax" of 0.836 pt per decision. That number was estimation bias, not entropy — the max over ~9 noisy action values sits about 1.5 sigma above the mean by construction. The tell was that it did not move at all with temperature. Contrasting two expectations removes the max operator and the bias with it. Instrument: jax_rl/entropy_tax.py.

A third claim: that single-player-solver features are worth +1.0 to +1.5 pt. You cannot falsify a counterfactual training gain without running it, but you can bound where it could come from. The main thing such a solver tells you is which discard keeps the hand fastest, so we measured exactly that: over ~6,000 discard decisions of self play, for every legal discard we computed the resulting shanten and the exact ukeire (tile-count acceptance), and compared the policy's choice against the optimum.

The RL policy at 1.34B steps picks a shanten-optimal discard 96.42% of the time and gives up 3.85% of the available ukeire. The BC base — trained only to imitate strong human play — scores 96.36% and 3.72%. The two are the same policy on this axis, and 1.34B steps of RL moved it by 0.06pp. Read that the right way: the deviations from maximum efficiency are not errors, they are trade-offs against safety, yaku value and dora that strong humans make at the identical rate. So the cheap story for +1.0-1.5pt ("the net cannot compute efficiency, hand it the answer") is ruled out; any real gain would have to come from sharpening the trade-off itself, which is the hard part and not what a shanten/ukeire solver gives you. We rate the claim unsupported at the stated magnitude and deprioritized it.

Caveat we will not paper over: our ukeire counts only exclude tiles in the actor's own hand, not those visible in rivers and melds, and a real solver is wall- and turn-aware and outputs expected value rather than acceptance. This bounds the static-efficiency component only. Instrument: jax_rl/ukeire_headroom.py.

A fourth claim, and a correction to our own README. Both the survey and our own notes said the 8-arm league scan carried about 1.9pt of winner's-curse bias, so the best arm's -9.70 should really be read as roughly -11.6. The arithmetic behind that number is exact — the expected maximum of 8 i.i.d. standard normals is 1.4236, and a 4k-game reading has a standard error of 1.38pt, so 1.96pt. It just does not describe what we actually did. Only 2 of the 8 arms were ever evaluated externally; the other 6 were filtered out beforehand by an internal training metric. Selecting the best of 2 external readings carries 0.78pt of bias, not 1.9pt, and the internal pre-filter only inflates that toward 1.9pt to the extent that it predicts external strength — which this project's central finding is that it does not. The debiased reading is about -10.5, with -11.7 as a worst case. Either way it sits far below its own -8.87 base, so nothing about the "self-family league gains nothing externally" conclusion changes.

The forward-looking half matters more than the correction: we now publish every checkpoint's reading and pool them inverse-variance instead of reporting the best one, which is immune to this bias by construction.

Two silent defects we found by measuring instead of reading (2026-09-07)

Both were found on a day spent auditing rather than training. Neither crashed, neither tripped an assertion, and both had been corrupting results for weeks.

1. The environment's wall RNG was a global constant after the first hand

We pin Mahjax at 3fa2826. In that revision red_mahjong/env.py::_init never seeds RoundState.rng_key — it keeps the dataclass default PRNGKey(0) — while every subsequent hand draws its wall from split(round_state.rng_key), and Env.step deletes the key it is handed on its first line. The consequence: hands 2 through 9 of every hanchan use one fixed set of 8 decks, identical across seeds, across parallel environments, across episodes and across runs.

round_mode="single" is unaffected. Under round_mode="half" we measured that 88.9% of training steps land on constant walls. Upstream fixed this in v0.1.3 (commit fade6de, issue #71, 2026-08-08), which describes it as an implicit full-information oracle for RL.

What makes this worth writing down is the shape of the failure. Tile conservation, zero-sum scoring, legal-action masks and end-of-hand settlement all pass exactly as before. Our evaluation runs through libriichi.arena and never touches Mahjax, so it stayed clean — which is precisely the asymmetry that produces the symptom we had been staring at for weeks: internal metrics healthy, external strength flat. We switched round_mode from single to half on 2026-08-23 as part of fixing an unrelated objective mismatch, and in doing so turned this latent bug on ourselves. About 2.98B steps of RL (~131 GPU-hours) were trained on ~89% duplicated deals.

Every instrument we had was on the policy side — clip fraction, entropy, KL, per-state max ratio. Not one was on the data side. One line counting distinct decks per batch would have caught it immediately.

2. The per-seat GAE reset was off by one

Our turn-based four-player credit assignment chains GAE per seat, since only current_player acts but all four seats can receive reward at any step. The reverse scan zeroed the carry at is_new[t] — the first step of an episode — where it should zero at the last. So an episode's opening decision was treated as terminal (bootstrap forced to zero), while its true terminal decision inherited gae/next_value from the following episode.

Replicated and fixed on the Sichuan trainer, then verified on identical trajectories: per-seat reward conservation error 1.000 -> 0, reward mass dropped at episode boundaries 22.0 -> 0 (1.03% of total), cross-episode value contamination 31 -> 0 (0.095% of decision points), with the valid-sample fraction unchanged at 87.50%. Real game score was never lost (with shaping disabled the leak is exactly 0); what leaked was the shaping term. On the riichi line this matters more than the raw percentage suggests, because placement reward is paid only on the terminal step — the contaminated transitions are exactly the ones carrying the entire objective signal.

Measuring against a weak opponent saturates, and then lies

The Sichuan section above reports its gate on a scale of "average points versus three copies of L0". That scale turns out to be saturated, which invalidates every margin quoted on it — including the follow-up reading in our repo, which had the 298M-step checkpoint scoring 0.528 worse than the 105M one and concluded that learning had stopped before 100M steps. Both of those were artifacts of the measuring stick, and correcting them is more interesting than the original claim.

L0 is a uniform-random policy, and there is a hard ceiling on what can be extracted from random opponents. We measured that ceiling at about +6.4, and inside the saturated band the ordering inverts:

	vs 3x L0	head-to-head
b192 (25M steps)	+6.775	-1.422 +/- 0.411
b2272 (298M steps)	+6.343	0 (anchor)

L0 ranks b192 above b2272. Played against each other, b2272 wins by +1.59, agreeing in both directions (z = +7.9 and -6.8). A positive control — b800 vs 3x b32 = +5.614 +/- 0.465 — rules out the alternative explanation that the head-to-head instrument simply cannot see anything.

Rebuilt with each checkpoint played directly against the strongest one, on identical deals (the anchor scores exactly 0 by the A(pi,pi) = 0 identity):

checkpoint	steps	vs 3x b2272	paired step-over-step
b32	4.2M	-5.216 +/- 0.344	—
b96	12.6M	-5.124 +/- 0.365	+0.092, z=0.45
b192	25.2M	-1.422 +/- 0.411	+3.703, z=15.5
b384	50.3M	-0.245 +/- 0.369	+1.177, z=4.51
b800	105M	-0.058 +/- 0.323	+0.187, z=0.91
b1408	185M	-0.085 +/- 0.333	-0.027, z=-0.13
b2272	298M	0	+0.085, z=0.50
hand-written L1	—	-2.062 +/- 0.336	—

Learning finishes at 50.3M steps, 17% of the budget; the remaining 83% buys +0.245 +/- 0.369 (z=1.30, not significant). The correct reading for b2272 - b800 is the head-to-head +0.035 +/- 0.270, not -0.528: "no stronger" survives, "nominally worse" does not. And the L0 scale had silently swallowed a real +1.18 gain between 25M and 50M steps.

The phase-2 gate itself still stands — b2272 beats L1 by +2.06 on the unsaturated scale, the same order as the +2.4 to +2.9 measured through L0. What does not stand is using any of those margins as a progress metric.

One more correction in the same spirit: the collapsed entropy we reported as 0.048 nats is an average over all decision points, and 72.9% of Sichuan decision points have exactly one legal action. Bucketed by legal-action count, the >= 5 bucket fell from 0.495 nats at 4M steps to 0.173 at 298M — a 65% collapse, where the average shows only 14%. The collapse is real (our falsification threshold was 0.5 nats in that bucket, and 0.17 misses it), but the number we had been watching understates it badly, and any closed-loop controller regulating the average would be regulating a quantity that is 73% structurally frozen.

The generalisable lesson, and the reason this sits next to our objective-mismatch writeup: a measuring instrument can reach its limit before the thing being measured does, and it will not tell you. It just returns a flat — or inverted — curve. Before concluding "no progress", show that the instrument can still resolve a known difference (positive control) and that it has not saturated (re-anchor and re-measure).

Format

Plain pickle of a Flax parameter pytree for LeanACNet(channels, blocks) (see jax_rl/net_lean.py in the repo). Load and run:

import pickle, jax
from net_lean import LeanACNet          # from the TanyaoDojo repo
from obs_v2 import observe_v2           # or obs_lean.observe_lean

params = pickle.load(open("bc_v2_g186.pkl", "rb"))
net = LeanACNet(channels=256, blocks=10)
logits, value = net.apply(params, observe_v2(state))   # state: Mahjax red_mahjong State

Observation must match the checkpoint: bc_v2_* needs obs_v2 (36 planes), everything else needs obs_lean (20 planes). Planes are stored as uint8 * scale in datasets (scale 24 for v2, 4 for lean) and divided back at train/eval time.

Evaluation harness: jax_rl/mjai_bot/run_eval.py in the repo (--obs v2 for the v2 checkpoint).

Training data — not distributed

These models were trained on Tenhou houou-level game logs. Neither the logs nor the derived datasets are redistributed here, per Tenhou's terms. The repo ships the full builder (jax_rl/data_bridge/make_bc_dataset.py) so you can rebuild equivalent datasets from logs you obtain yourself.

License

MIT for these weights and the core training code. Note that the repo's evaluation bridge (jax_rl/mjai_bot/) links libriichi and is AGPL-3.0; see the repo's LICENSING.md.