Add Monte Carlo simulator, game metrics and sim CLI (Phase 5)

`dojo sim` plays seeded games over a scenario matrix (variant, players,
Kiai bonus, start position, bot lineup) in parallel, prints a summary per
scenario and optionally writes one CSV row per game. Metrics are replayed
from the event log. Plan updated with the Phase 5 baseline.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-26 16:29:22 +02:00
parent 0aaf113fc3
commit 7b76b10175
9 changed files with 719 additions and 12 deletions
+25 -12
View File
@@ -101,16 +101,29 @@ Deliverables:
### Phase 5 — Simulator and metrics
- [ ] 5.1 Monte Carlo runner over a config matrix; seeded and reproducible.
- [ ] 5.2 Metrics:
- [ ] game length in rounds and estimated minutes (turn and fight duration as parameters);
- [ ] Kumite per round, upset rate, chain length;
- [ ] win rate by seat and by bot profile;
- [ ] exam attempts per game, exam success rate;
- [ ] rounds spent stuck on 14 (variant B deadlock check);
- [ ] stripe income split: squares vs cards vs Kumite;
- [ ] lead changes per game, final spread.
- [ ] 5.3 Report output (plain text / CSV).
- [x] 5.1 Monte Carlo runner over a config matrix; seeded and reproducible. `src/dojo/sim.py`: `Scenario` (variant × players × Kiai bonus × start position × lineup), `matrix`, `run(files, scenarios, games, seed, jobs)`. Game `i` uses seed `base + i` in every scenario (same dice across scenarios); the lineup rotates one seat per game; process pool, results independent of `jobs`.
- [x] 5.2 Metrics (`src/dojo/metrics.py`, computed from the event log; Kumite counts exclude the Exam):
- [x] game length in rounds and estimated minutes (45 s per turn + 30 s per Kumite);
- [x] Kumite per round, upset rate, higher-beats-lower rate, longest chain per game (Kumite attacked by the active player in one turn);
- [x] win rate by seat and by bot profile (per seat occupied);
- [x] exam attempts per game, exam success rate;
- [x] turns started on 14, longest streak (variant B deadlock check);
- [x] net stripe income per turn: squares vs cards vs Kumite vs Exam; square income in the player's own quadrant vs elsewhere (home farming);
- [x] lead changes per game (strict leader), final spread.
- [x] 5.3 Report output: `uv run dojo sim` prints a text summary per scenario; `--csv` writes one row per game (`src/dojo/report.py`). Commands in `README.md`.
- Baseline (1,000 games per variant, 4 players, lineup balanced/farmer/aggressive, seed 0), for Phase 6:
| Metric | A | B |
|---|---|---|
| Rounds mean (p10–p90) | 18.4 (14–23) | 13.1 (9–17) |
| Minutes mean (p10–p90) | 66 (50–85) | 46 (33–62) |
| Kumite per round | 1.25 | 1.16 |
| Upsets / higher beats lower | 29% / 55% | 27% / 52% |
| Exam attempts per game, pass rate | 1.26, 79% | 1.22, 82% |
| Turns on 14 per game, longest streak | 1.26, 4 | 1.73, 7 |
| Income per turn: square (home / away), card, Kumite | +0.52 (0.38 / 0.14), 0.00, +0.04 | +0.54 (0.39 / 0.14), 0.00, +0.22 |
| Lead changes, final spread | 5.6, 9.3 | 3.5, 11.5 |
| Wins by seat | 23–27% | 23–27% |
### Phase 6 — Balance experiments
@@ -147,14 +160,14 @@ Deliverables:
### Open
- Q2 Board asymmetry: keep the quadrant as source of truth, or later switch to the full board as source of truth with a symmetry *report* (warnings) instead of generation? *(Deferred by the developer.)*
- Q4 Target net income per turn — derived from 1–2 h and ~15–20 turns per player; confirm with first simulations.
- Q4 Target net income per turn — derived from 1–2 h and ~15–20 turns per player; confirm with first simulations. Baseline measured net income ≈ 0.56 (A) / 0.75 (B) stripes per turn.
### Risks
- Movement enumeration: self-avoiding walks of length up to 12 can explode in open rooms. Keep rooms small; prune by outcome (destination + ordered opponents crossed + Dojo entry).
- Engine speed ~1,400 turns/s (random and heuristic bots, real board). If Phase 5 needs more: precompute neighbour lists in `Board`.
- Simulation conclusions depend on bot quality; compare several profiles before trusting a result.
- Board draft 1 puts 4 Allenamento in each 3x3 room next to the Safe Room: possible "home farming" (see the Phase 2 review). Measure stripe income by region in the simulator.
- Board draft 1 puts 4 Allenamento in each 3x3 room next to the Safe Room: possible "home farming" (see the Phase 2 review). Measured (Phase 5 baseline): square income is +0.38/turn in the own quadrant vs +0.14/turn elsewhere; 6.4 decides whether that is a problem.
## External documentation