TanyaoDojo — Mahjong AI checkpoints

JAX/Flax checkpoints from TanyaoDojo, a from-scratch mahjong AI stack (vectorized env + behavior cloning + self-play RL). Every number below comes from the same duplicate 1v3 protocol: the challenger plays all four seats over identical walls against three copies of a strong open-source baseline (Mortal v4), seed_key=20260711, seeds from 10000, placement points [90, 45, 0, -135].

avg_pt is the challenger's mean placement points relative to the baseline; 0 would mean parity. Higher is better.

The headline checkpoint has a 100k-game milestone measurement (-4.66 +/- 0.535, 16.0M decisions, zero fallbacks) — not a small-sample estimate.

Checkpoints

File	Arch	Obs	Training	avg_pt vs baseline
bc_v2_g186.pkl	256ch x 10blk (12.4M)	v2 (34x36 + 32)	BC on 10y logs + LR-1e-4 refine	-4.66 +/- 0.535 (100k games)
bc_lean_g402.pkl	256ch x 10blk (12.4M)	lean (34x20 + 26)	BC on 14y logs + LR-1e-4 refine	-5.07 +/- 1.54 (12k games)
bc_lean_w192_ep2.pkl	192ch x 8blk (5.3M)	lean	BC on 6y logs, 2 epochs	-8.87 +/- 2.70 (4k games)
rl_oracle_800m.pkl	256ch x 10blk	lean (actor)	oracle-critic PPO league, 0.8B steps	-8.38 +/- 2.65 (4k) — negative result, wrong objective (see below)

The last row is published deliberately, and its interpretation was corrected on 2026-08-23 — the correction is more useful than the checkpoint.

An asymmetric actor-critic whose critic sees all four hands fit the value function ~100x better (v_loss 0.102 -> 0.001) yet lost 3.2pt of external strength. It was the fourth consecutive negative RL result here. We originally read this as "oracle critics don't transfer". That reading was wrong.

The real cause: the training objective was not the evaluation objective. All four RL runs used round_mode="single", which ends the episode after a single hand, and the reward is mahjong's raw point transfer. The arena scores an entire hanchan by placement points [90, 45, 0, -135]. Measured directly, not inferred from code: 600 steps x 256 envs produced 1541 episodes (one hand each); rewards ranged [-120, 130] with rows like [+30, -10, -10, -10] summing to ~0 (point transfers); and mahjax writes order_points only into the final score, never into rewards, in any round_mode.

A single-hand point maximizer has no concept of 4th place — falling to last costs it nothing extra, so it should be maximally aggressive. This predicts that more RL steps make the arena result worse (observed), and it explains why a better critic hurt more: it converged faster onto the wrong objective.

So this checkpoint is not evidence against oracle critics. It is evidence that we optimized the wrong function for ~10^10 environment steps. The fix — terminal placement-aligned reward over full hanchan — is in the repo as jax_rl/reward_placement.py, and all four RL results above are being redone on it.

First result after the fix (1B steps from the -4.66 base, 11 arena evals of 1600 games each along the way): -4.88 +/- 0.81 — statistically flat versus the base. The previous four runs lost 3.2 to 10 points each and got worse the longer they ran. The degradation is gone; the gains are not there yet, and the metrics say why — approx_kl about 4e-5 per update and only 0.011 KL of drift from the base after a billion steps, i.e. a tight trust region plus roughly 0.75 hanchan terminals per rollout left the policy almost unmoved. The next step is therefore signal density, not a different algorithm: GRP potential shaping (a 7.5M-sample "position -> expected final placement points" potential, reward becomes phi(s') - phi(s) with the terminal remainder), which is policy-invariant by the shaping theorem — verified numerically — and raises the fraction of non-zero-reward steps from 0.1% to 1.10%.

If you are porting an RL recipe (ours or anyone's): check that its episode boundary and reward definition are the same function your benchmark scores.

Format

Plain pickle of a Flax parameter pytree for LeanACNet(channels, blocks) (see jax_rl/net_lean.py in the repo). Load and run:

import pickle, jax
from net_lean import LeanACNet          # from the TanyaoDojo repo
from obs_v2 import observe_v2           # or obs_lean.observe_lean

params = pickle.load(open("bc_v2_g186.pkl", "rb"))
net = LeanACNet(channels=256, blocks=10)
logits, value = net.apply(params, observe_v2(state))   # state: Mahjax red_mahjong State

Observation must match the checkpoint: bc_v2_* needs obs_v2 (36 planes), everything else needs obs_lean (20 planes). Planes are stored as uint8 * scale in datasets (scale 24 for v2, 4 for lean) and divided back at train/eval time.

Evaluation harness: jax_rl/mjai_bot/run_eval.py in the repo (--obs v2 for the v2 checkpoint).

Training data — not distributed

These models were trained on Tenhou houou-level game logs. Neither the logs nor the derived datasets are redistributed here, per Tenhou's terms. The repo ships the full builder (jax_rl/data_bridge/make_bc_dataset.py) so you can rebuild equivalent datasets from logs you obtain yourself.

License

MIT for these weights and the core training code. Note that the repo's evaluation bridge (jax_rl/mjai_bot/) links libriichi and is AGPL-3.0; see the repo's LICENSING.md.