smoothlife/plans/10_reference_and_analysis_tools.md

Macrostep 10 — Reference and analysis tools

Objective

Turn the trusted core into reproducible scientific/diagnostic tools. These tools validate numerical choices and make hidden model behavior understandable; only the comparison harness is required for v1. The inverse-design rule lab is optional.

Dependencies

Macrosteps 00–09 complete for a full-system comparison. The numerical harness can begin after Macrostep 03.

Phase 10.1 — Numerical integration comparison

Substep 10.1.1 — Headless experiment format

Define a versioned experiment file containing:

• preset and explicit initial-state fixture;
• backend/precision;
• integrators and timesteps;
• exact start/end/step-count convention;
• reference selection;
• output metrics and snapshot cadence.

Use exact step counts; do not inherit the Matlab for t=0:dt:end extra-update ambiguity.

Substep 10.1.2 — Integrators

Compare:

• Euler;
• iterative implicit trapezoid (Euler predictor, clamped fixed-point iteration, bounded count/tolerance);
• AB3 with Euler/AB2 startup;
• RK4 with clamped stages under both StageState and retained historical StepOrigin relaxation references;
• optional AB4.

Use the same convolution/rule provider for every method. Record derivative evaluation count and wall time as well as error.

Substep 10.1.3 — Metrics and reports

Compute max, L1, L2, normalized L2, histogram distance, and optional spectrum-band differences against a chosen high-resolution reference. Export CSV/JSON plus plots or plot-ready data.

Treat the legacy result.txt values as historical trend evidence, not bitwise goldens: source parameters and timestep sweeps drifted, and the Matlab loop advanced an extra step. Acceptance is that higher-order methods converge consistently at small dt, not exact recreation of every printed number.

Phase 10.2 — Backend comparison tool

Create a command that runs one serialized state through the relevant implementations and compares:

• standard CPU versus standard GPU FFT;
• legacy packed GPU stages versus its CPU stage oracle and direct convolution;
• both RK4 relaxation references;
• both sphere models against their CPU oracles;
• kernels, M/N/S, every integration stage, final state, and later aggregate metrics.

It should stop at the first stage exceeding tolerance and emit both fields for visual diff in the workbench. Include GPU/driver metadata.

Phase 10.3 — Preset health sweep

For every bundled preset:

1. validate schema and localized historical-option constraints;
2. allocate at a reduced smoke-test shape where valid;
3. reset deterministically;
4. run a short fixed step count;
5. assert all values finite and committed state in [0,1];
6. record min/max/mean/variance and warnings;
7. ensure every inspection channel can be requested.

Do not assert that chaotic patterns match a screenshot. The sweep detects crashes, NaNs, stale resources, and schema drift.

Phase 10.4 — Visualization analysis overlays

Add optional workbench overlays sourced from core data:

• CPU/GPU absolute-difference heatmap;
• update/clamp saturation mask;
• occupancy histogram;
• radial kernel profile;
• 2-D Fourier magnitude preview;
• multiscale contribution stack;
• sphere seam error/area distortion;
• delayed-time radial age map.

Keep expensive analysis paused/on-demand; never alter simulation state.

Phase 10.5 — Optional Glider Constructor rule lab

This is explicitly nonblocking for the simulator release. If approved, implement behind tools:

• 80×80 toroidal symmetric drawing canvas;
• selectable integer vertical displacement;
• 500×500 quantized (N,M) lookup assignment;
• conflict count/heatmap when one bin demands both outputs;
• test simulation using the derived lookup;
• import/export for pattern and lookup.

Keep lookup rules separate from analytic RuleConfig; they are a distinct rule-provider type. Document differences from the legacy aid: no bundled glider fixture existed, undefined bins defaulted to 0.5, and analytic rather than sampled normalization was used.

Phase 10.6 — Reproducibility

Every tool output includes:

• application/git version;
• schema/preset ID;
• serialized initial-state hash;
• backend and precision;
• seed, shape, generation, timestep, and integrator;
• CPU thread count or GPU/driver identity;
• relevant FFT algorithm, RK4 relaxation reference, and sphere model;
• command line/experiment file hash.

Outputs are collision-safe and can be rerun from a generated manifest.

Deliverables

• Numerical integration experiment runner and reports.
• Stage-aware CPU/GPU comparison tool.
• Complete bundled-preset health sweep.
• On-demand analysis overlays.
• Optional inverse-design rule lab only if separately accepted.

Exit gate

• Integration experiments are exact-step reproducible.
• Small-timestep convergence order/trends are sensible and documented.
• CPU/GPU comparison identifies the first divergent stage.
• Every mandatory bundled preset passes the health sweep or is explicitly quarantined with a reason.
• Optional rule lab cannot silently masquerade as an analytic SmoothLife preset.