Add failed-Exam loss switch; keep -3 and accept the length (6.11, D50)

RulesConfig.exam_failure_loss (bots aware) with the sim axis --exam-loss. Losing 1 or 0 stripes saves 3-5 minutes but weakens comebacks, so the rules keep -3. Default games are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 17:51:13 +02:00
parent 542ac1b15f
commit 52d8af620b
11 changed files with 116 additions and 17 deletions
@@ -46,6 +46,7 @@ uv run dojo sim --lineup casual balanced --summary-csv s.csv # one CSV row per s
 uv run dojo sim --exam c none a ac                # Final Exam options in force (rules: c), combinable
 uv run dojo sim --players 3 4 6 8 --start 0 3     # player counts x start position
 uv run dojo sim --kiai 2 1                        # Kiai +2 vs +1
+uv run dojo sim --exam-loss 3 1 0                 # stripes a failed Exam challenger loses (rules: 3)
 uv run dojo sim --lineup farmer aggressive,farmer # several lineups (comma-separated seats)
 uv run dojo sim --seed 42 --jobs 1                # other dice; single process
 ```
@@ -179,7 +179,7 @@ All other cards are shown and resolved immediately.
 Parameters to validate by simulation: square mix; game length.

 Tested and set aside (simulator switches only): Exam ties to the defender; best-of-3 Exam;
-Kiai +1.
+Kiai +1; a failed Exam costing 0 or 1 stripe instead of 3.
 Earlier rules kept as simulator switches: Fatica costing only when you stop on it; Kumite
 allowed in the first round.

@@ -66,6 +66,7 @@ Deliverables:
 | D47 | **Kiai stays +2**: +2 and +1 are indistinguishable on every target (6.6); +2 decides more fights (5% vs 3% of Kumite in casual play). `kiai_bonus = 1` stays a simulator switch; RULES.md drops the "+1 to be tested" note | Developer, 6.6 results. |
 | D48 | **6+ players start on 3 (Yellow)**; any player count may start on 3 as a "shorter game" option (saves 15–20% of the time, 6.7). 8 players stays long (107 / 142 min mixed / casual from Yellow) and is accepted: it will rarely be played. Balancing simulations stay at start 0 at every count (developer); `--start 3` reproduces the rules' 6+ setup or the shorter game | Developer, 6.7 results. |
 | D49 | **Chance square parked** (6.4.1 and 6.4.3): Fatica on crossing (D36) on board draft 2 (D42) already makes squares matter; 6.4 closed | Developer. |
+| D50 | **Length accepted as is; a failed Exam still costs 3 stripes.** L1 (Brown floor against Fatica) rejected; L2 (failed Exam −1 / 0) saves only 3–5 min on the mean and weakens comebacks (6.11.4). `exam_failure_loss` stays a simulator switch. At 4 players mixed and balanced are inside the target; casual (91 / 122) and all-aggressive (90 / 127) stay slightly outside, within the "1–2 h" promise; the shorter-game start on Yellow (D48) is the players' lever | Developer, 6.11 results. |

 ## Phases

@@ -176,7 +177,7 @@ Deliverables:
 - One change at a time. When a change is adopted, re-run the 6.1 baseline so later steps start from the current rules, and log the decision (D-number) here and in `RULES.md`.
 - Results go in this file as a table per step (command line, scenario, key measures).

-**Next action:** 6.11 length tuning, then 6.8. 6.2 is suspended (D43), 6.9 skipped (D44); 6.4.1 (chance square) stays optional. Order of the steps: D41. Everything up to D49 is committed; commit only when the developer asks.
+**Next action:** 6.8 consolidation, when the developer says to continue. 6.2 is suspended (D43), 6.9 skipped (D44); 6.4.1 (chance square) stays optional. Order of the steps: D41. Everything up to D50 is committed; commit only when the developer asks.

 **Targets** (confirmed by the developer as a starting point, D32):

@@ -320,7 +321,47 @@ Deliverables:
  - [x] 6.10.3 `movement.walks`: walks that exclude nothing but the start are cached per (board, start, steps, Dojo open) with `functools.cache` (same pattern as `bots.dojo_distance`); callers get a fresh list. ~3,200 entries per process, ~80 MB peak.
  - [x] 6.10.4 Parallel scaling: 8 processes running at once each take ~1.6x their solo time on this machine (8 real cores, clock and cache sharing), so ~5x on 8 workers is the hardware limit; no pool change. 16 workers (hyperthreads) add ~12%.
  - Result: 100 games on one process 8–12 s → 1.4–1.7 s; mixed + balanced + casual at 1,000 games on 16 workers 57 s → 8 s. All 210 fingerprinted games identical; the v0.8 baseline run prints the same metrics.
- [ ] 6.11 Length tuning (D39): at 4 players, bring every lineup to a mean of 60–90 minutes with p90 ≤ 110 (v0.8: mixed 72 / 95 and balanced 68 / 92 inside; casual 91 / 122 and all-aggressive 90 / 127 outside). Find where the time goes (climb to Brown vs Exam phase), bring candidate levers to the developer, simulate the chosen ones on at least two lineups.
+- [x] 6.11 Length tuning (D39; closed by D50): at 4 players, bring every lineup to a mean of 60–90 minutes with p90 ≤ 110 (v0.8: mixed 72 / 95 and balanced 68 / 92 inside; casual 91 / 122 and all-aggressive 90 / 127 outside). Find where the time goes (climb to Brown vs Exam phase), bring candidate levers to the developer, simulate the chosen ones on at least two lineups.
+  - [x] 6.11.1 Where the time goes (scratch `phases.py`, 400 games per lineup; "endgame" = from the first Brown on the table to the win):
+
+    | Lineup | Climb / endgame (min) | Winner's stripes gained (15 needed) | Drops out of Brown per game | Stripes lost per game: square / Kumite / Exam |
+    |---|---|---|---|---|
+    | mixed | 49 / 22 (31%) | 23.3 | 2.2 | 3.9 / 27.8 / 4.3 |
+    | balanced | 48 / 21 (30%) | 22.6 | 2.2 | 2.4 / 26.6 / 4.3 |
+    | casual | 56 / 36 (39%) | 24.7 | 4.0 | 11.2 / 23.1 / 4.4 |
+    | aggressive | 60 / 32 (35%) | 28.6 | 2.7 | 12.4 / 42.7 / 4.4 |
+
+    Casual and aggressive are longer in both phases, most of all in the endgame (+11 to +15 min): casual players fall out of Brown twice as often (Fatica tolls, 11 stripes per game), aggressive ones trade stripes in Kumite (43 lost per game).
+  - [x] 6.11.2 Existing switches (`--start`, `--exam`), 1,000 games, 4 players, minutes mean / p90:
+
+    | Setting | mixed | balanced | casual | aggressive |
+    |---|---|---|---|---|
+    | v0.8 (start 0) | 72 / 95 | 68 / 92 | 91 / 122 | 90 / 127 |
+    | start 1 | 68 / 92 | 65 / 87 | 88 / 122 | 88 / 124 |
+    | start 2 | 63 / 88 | 59 / 83 | 81 / 112 | 82 / 122 |
+    | start 3 | 57 / 80 | — | 77 / 110 | 74 / 111 |
+    | Exam defender not Black (`none`) | 60 / 77 | — | 79 / 103 | 74 / 97 |
+
+    The start position shifts every lineup by about the same amount, so it cannot close the ~23-minute gap between balanced and casual: start 2 puts balanced under 60 while aggressive stays at p90 122. The Black-belt defender off pushes the Exam pass rate to 67–80% (target 35–55%) and the rivals at the end to 0.27–0.49: rejected again (as in D31). Other measures barely move with the start position.
+  - [x] 6.11.3 Candidate levers brought to the developer: L1 Fatica cannot drop a player out of Brown (**rejected by the developer**); L2 a failed challenger loses 0 or 1 stripe instead of 3 (to test); L3 accept the current length (developer: "satisfied enough").
+  - [x] 6.11.4 L2 switch: `RulesConfig.exam_failure_loss` (default 3; engine and bot `_exam_score`), sim axis `Scenario.exam_loss` / `--exam-loss` (scenario names gain `fail-N`). Fingerprinted games identical at the default. Run: `uv run dojo sim --games 1000 --seed 0 --exam-loss 3 1 0 --lineup balanced,farmer,aggressive balanced casual aggressive`.
+
+    | Lineup | Failed Exam loses | Minutes mean / p90 | Exam challengers per game | Lead changes | Halfway leader wins | Worst player out | Rivals within 1 belt |
+    |---|---|---|---|---|---|---|---|
+    | mixed | 3 | 72 / 95 | 1.68 | 6.1 | 40% | 13% | 0.84 |
+    | mixed | 1 | 68 / 88 | 1.48 | 5.0 | 44% | 15% | 0.94 |
+    | mixed | 0 | 67 / 86 | 1.45 | 4.6 | 44% | 16% | 0.87 |
+    | balanced | 3 | 68 / 92 | 1.69 | 6.0 | 39% | 10% | 0.81 |
+    | balanced | 1 | 65 / 84 | 1.50 | 5.0 | 43% | 12% | 0.97 |
+    | balanced | 0 | 65 / 83 | 1.46 | 4.5 | 43% | 13% | 0.90 |
+    | casual | 3 | 91 / 122 | 1.74 | 5.7 | 39% | 12% | 1.42 |
+    | casual | 1 | 87 / 116 | 1.61 | 4.9 | 42% | 13% | 1.47 |
+    | casual | 0 | 86 / 114 | 1.55 | 4.6 | 41% | 14% | 1.40 |
+    | aggressive | 3 | 90 / 127 | 1.58 | 7.3 | 41% | 21% | 0.50 |
+    | aggressive | 1 | 85 / 115 | 1.40 | 6.2 | 45% | 23% | 0.59 |
+    | aggressive | 0 | 83 / 111 | 1.32 | 5.8 | 49% | 25% | 0.54 |
+
+    Exam pass rate unchanged (40–44%). Reading: −1 saves 3–5 minutes on the mean and 6–12 on the p90; 0 adds only 1–2 more. The price is the comeback: the first Brown keeps retrying instead of dropping back (fewer different challengers), lead changes fall by ~1 per game, the halfway leader wins 3–8 points more, the worst player is out a little longer. No setting brings casual under p90 110.
 - [ ] 6.9 Catch-up candidate A — **skipped (D44)**. Was: `RulesConfig` switch, Allenamento +2 for a player 2+ belts behind the leader (bot aware of it); compare on the contention, lead and length measures and on the style win rates.
 - [ ] 6.8 Consolidation: the final configuration at 3 / 4 / 6 / 8 players (start 0 as for balancing, plus start 3 at 6 and 8 as in the rules, D48) with all lineups except `random` (D45), 2,000 games each; every target checked; `RULES.md` updated to the next version with the decisions; open items moved to Phase 7.

@@ -343,7 +343,7 @@ class HeuristicAgent:
        chance = series_probability(round_chance, view.config.exam_rounds)
        failed = self._value_at(
            view,
-            view.config.shifted(track, -view.config.stripes_per_belt),
+            view.config.shifted(track, -view.config.exam_failure_loss),
            start_square(view.board, me.corner),
        )
        return chance * EXAM_WIN + (1 - chance) * self._weigh(failed - v0)
@@ -8,7 +8,16 @@ from dojo.board import QUADRANTS, Board, BoardError, load_board
 from dojo.grid import GridFormatError, render_grid
 from dojo.metrics import summarize
 from dojo.report import format_summary, write_games_csv, write_summary_csv
-from dojo.sim import BOTS, EXAM_OPTIONS, NO_EXAM_OPTIONS, RULES_EXAM, BoardFiles, matrix, run
+from dojo.sim import (
+    BOTS,
+    EXAM_OPTIONS,
+    NO_EXAM_OPTIONS,
+    RULES_EXAM,
+    RULES_EXAM_LOSS,
+    BoardFiles,
+    matrix,
+    run,
+)
 from dojo.validate import Report, validate

 DEFAULT_QUADRANT = Path("board/quadrant.txt")
@@ -58,6 +67,13 @@ def main(argv: list[str] | None = None) -> int:
    sim_parser.add_argument("--players", nargs="+", type=int, default=[4])
    sim_parser.add_argument("--kiai", nargs="+", type=int, default=[2], help="Kiai bonus")
    sim_parser.add_argument("--start", nargs="+", type=int, default=[0], help="start position")
+    sim_parser.add_argument(
+        "--exam-loss",
+        nargs="+",
+        type=int,
+        default=[RULES_EXAM_LOSS],
+        help=f"stripes a failed Exam challenger loses (rules: {RULES_EXAM_LOSS})",
+    )
    sim_parser.add_argument(
        "--lineup",
        nargs="+",
@@ -104,6 +120,7 @@ def _simulate(args: argparse.Namespace, files: BoardFiles, parser: argparse.Argu
            args.kiai,
            args.start,
            [tuple(lineup.split(",")) for lineup in args.lineup],
+            args.exam_loss,
        )
    except ValueError as error:
        parser.error(str(error))
@@ -442,7 +442,7 @@ class Game:
        if passed:
            self.winner = challenger.id
            return
-        self._change_track(challenger, -self.config.stripes_per_belt, Source.EXAM)
+        self._change_track(challenger, -self.config.exam_failure_loss, Source.EXAM)
        if not uke:
            self._send_to_start(challenger)

@@ -51,6 +51,8 @@ class RulesConfig:
    exam_defender_black: bool = True
    exam_ties_to_defender: bool = False
    exam_rounds: int = 1
+    # Stripes a failed challenger loses (6.11: 0 and 1 tested against the rules' 3).
+    exam_failure_loss: int = 3

    # Fatica costs a stripe for every Fatica square walked onto, stopping included (D36);
    # off: only stopping on it costs.
@@ -69,6 +71,8 @@ class RulesConfig:
    def __post_init__(self) -> None:
        if self.exam_rounds < 1 or self.exam_rounds % 2 == 0:
            raise ValueError(f"exam_rounds must be odd and positive, got {self.exam_rounds}")
+        if self.exam_failure_loss < 0:
+            raise ValueError(f"exam_failure_loss must be >= 0, got {self.exam_failure_loss}")
        if self.first_kumite_round < 1:
            raise ValueError(f"first_kumite_round must be positive, got {self.first_kumite_round}")

@@ -28,6 +28,7 @@ EXAM_OPTIONS = {
    "d": "best of 3 rounds",
 }
 BEST_OF_ROUNDS = 3
+RULES_EXAM_LOSS = RulesConfig().exam_failure_loss
 RANDOM_BOT = "random"
 CASUAL_BOT = "casual"
 BOTS = (RANDOM_BOT, CASUAL_BOT, *PROFILES)
@@ -42,6 +43,7 @@ class Scenario:
    start_position: int
    # Bot names, repeated in order to fill the seats.
    lineup: tuple[str, ...]
+    exam_loss: int = RULES_EXAM_LOSS

    def __post_init__(self) -> None:
        options = "" if self.exam == NO_EXAM_OPTIONS else self.exam
@@ -54,12 +56,13 @@ class Scenario:
        if unknown or not self.lineup:
            raise ValueError(f"unknown bots {unknown}: expected some of {list(BOTS)}")
        seat_corners(self.players)
+        _ = self.config  # RulesConfig validates the remaining axes.

    @property
    def name(self) -> str:
        return (
            f"exam:{self.exam} p{self.players} kiai+{self.kiai_bonus} start{self.start_position} "
-            + ",".join(self.lineup)
+            f"fail-{self.exam_loss} " + ",".join(self.lineup)
        )

    @property
@@ -68,6 +71,7 @@ class Scenario:
            exam_ties_to_defender="a" in self.exam,
            exam_defender_black="c" in self.exam,
            exam_rounds=BEST_OF_ROUNDS if "d" in self.exam else 1,
+            exam_failure_loss=self.exam_loss,
            kiai_bonus=self.kiai_bonus,
            start_position=self.start_position,
        )
@@ -84,10 +88,13 @@ def matrix(
    kiai_bonuses: Sequence[int],
    start_positions: Sequence[int],
    lineups: Sequence[tuple[str, ...]],
+    exam_losses: Sequence[int] = (RULES_EXAM_LOSS,),
 ) -> list[Scenario]:
    return [
        Scenario(*axes)
-        for axes in itertools.product(exams, players, kiai_bonuses, start_positions, lineups)
+        for axes in itertools.product(
+            exams, players, kiai_bonuses, start_positions, lineups, exam_losses
+        )
    ]


@@ -68,7 +68,7 @@ def test_sim_prints_a_summary_per_scenario_and_writes_games(
    assert main([*argv, "--csv", str(games)]) == EXIT_OK

    output = capsys.readouterr().out
-    assert "== exam:none p3 kiai+2 start0 balanced,farmer,aggressive  (2 games" in output
+    assert "== exam:none p3 kiai+2 start0 fail-3 balanced,farmer,aggressive  (2 games" in output
    assert "== exam:c p3 kiai+2 start0" in output
    rows = games.read_text(encoding="utf-8").splitlines()
    assert rows[0].startswith("scenario,seed,lineup,winner_seat")
@@ -78,21 +78,25 @@ def test_sim_prints_a_summary_per_scenario_and_writes_games(
 def test_sim_writes_one_summary_row_per_scenario(tmp_path: Path) -> None:
    path = tmp_path / "summary.csv"
    argv = ["sim", "--games", "2", "--players", "3", "4", "--lineup", "casual", "--jobs", "1"]
+    argv += ["--exam-loss", "3", "0"]

    assert main([*argv, "--summary-csv", str(path)]) == EXIT_OK

    with path.open(encoding="utf-8", newline="") as file:
        rows = list(csv.DictReader(file))
    assert [row["scenario"] for row in rows] == [
-        "exam:c p3 kiai+2 start0 casual",
-        "exam:c p4 kiai+2 start0 casual",
+        "exam:c p3 kiai+2 start0 fail-3 casual",
+        "exam:c p3 kiai+2 start0 fail-0 casual",
+        "exam:c p4 kiai+2 start0 fail-3 casual",
+        "exam:c p4 kiai+2 start0 fail-0 casual",
    ]
    assert {"rounds_mean", "square_gain_per_turn", "win_rate_casual"} <= rows[0].keys()
-    assert (rows[0]["seat3_win_rate"], rows[1]["seat3_win_rate"] != "") == ("", True)
+    assert (rows[0]["seat3_win_rate"], rows[2]["seat3_win_rate"] != "") == ("", True)


@pytest.mark.parametrize(
-    "bad", [["--lineup", "balanced,cheater"], ["--games", "0"], ["--exam", "b"]]
+    "bad",
+    [["--lineup", "balanced,cheater"], ["--games", "0"], ["--exam", "b"], ["--exam-loss", "-1"]],
 )
 def test_sim_rejects_bad_arguments(bad: list[str], capsys: pytest.CaptureFixture[str]) -> None:
    with pytest.raises(SystemExit) as exit_info:
@@ -677,6 +677,22 @@ def test_failing_the_exam_costs_a_belt_and_the_defender_gains_nothing() -> None:
    assert game.players[2].track == 12


+@pytest.mark.parametrize(("loss", "track"), [(0, 15), (1, 14)])
+def test_failed_exam_loss_is_configurable(loss: int, track: int) -> None:
+    config = RulesConfig(exam_failure_loss=loss)
+    game, agents = dojo_game([1, 1, 1, 6], config=config, p1=9, p2=12)
+
+    turn(game, agents, 0, Pos(6, 7))
+
+    assert game.players[0].track == track
+    assert game.players[0].position == START_0
+
+
+def test_failed_exam_loss_cannot_be_negative() -> None:
+    with pytest.raises(ValueError, match="exam_failure_loss"):
+        RulesConfig(exam_failure_loss=-1)
+
+
 def test_uke_in_the_exam_returns_to_the_entrance() -> None:
    game, agents = dojo_game([1, 1, 1, 6], p1=9, p2=12)
    game.players[0].tecnica = Card.UKE
@@ -11,13 +11,17 @@ LINEUP = ("balanced", "random")


 def test_matrix_crosses_every_axis() -> None:
-    scenarios = matrix(["none", "acd"], [3, 4], [2, 1], [0], [LINEUP])
+    scenarios = matrix(["none", "acd"], [3, 4], [2, 1], [0], [LINEUP], [3, 0])

-    assert len(scenarios) == 8
-    assert scenarios[0].name == "exam:none p3 kiai+2 start0 balanced,random"
+    assert len(scenarios) == 16
+    assert scenarios[0].name == "exam:none p3 kiai+2 start0 fail-3 balanced,random"
    assert scenarios[0].config == RulesConfig(exam_defender_black=False)
    assert scenarios[-1].config == RulesConfig(
-        exam_ties_to_defender=True, exam_defender_black=True, exam_rounds=3, kiai_bonus=1
+        exam_ties_to_defender=True,
+        exam_defender_black=True,
+        exam_rounds=3,
+        exam_failure_loss=0,
+        kiai_bonus=1,
    )


@@ -64,6 +68,11 @@ def test_invalid_scenarios_are_rejected(
        Scenario(exam, players, 2, 0, lineup)


+def test_invalid_exam_loss_is_rejected() -> None:
+    with pytest.raises(ValueError, match="exam_failure_loss"):
+        Scenario("c", 4, 2, 0, LINEUP, exam_loss=-1)
+
+
 def test_runs_are_reproducible_and_independent_of_worker_count() -> None:
    scenarios = matrix(["none", "acd"], [3], [2], [0], [LINEUP])