Add detailed phase plans for foundational macrosteps
- Macrostep 00: Model contract and retained historical options. - Macrostep 01: Project foundation and configuration, including schema, presets, CLI, and CI setup. - Macrostep 02: Mathematical core implementation and deterministic oracles. - Macrostep 03: CPU-based planar simulation engine, FFT backend, and headless runner. Provides exhaustive objectives, phase breakdowns, validation policies, and deliverables for each macrostep.
This commit is contained in:
127
plans/10_reference_and_analysis_tools.md
Normal file
127
plans/10_reference_and_analysis_tools.md
Normal file
@@ -0,0 +1,127 @@
|
||||
# Macrostep 10 — Reference and analysis tools
|
||||
|
||||
## Objective
|
||||
|
||||
Turn the trusted core into reproducible scientific/diagnostic tools. These tools validate numerical choices and make hidden model behavior understandable; only the comparison harness is required for v1. The inverse-design rule lab is optional.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Macrosteps 00–09 complete for a full-system comparison. The numerical harness can begin after Macrostep 03.
|
||||
|
||||
## Phase 10.1 — Numerical integration comparison
|
||||
|
||||
### Substep 10.1.1 — Headless experiment format
|
||||
|
||||
Define a versioned experiment file containing:
|
||||
|
||||
- preset and explicit initial-state fixture;
|
||||
- backend/precision;
|
||||
- integrators and timesteps;
|
||||
- exact start/end/step-count convention;
|
||||
- reference selection;
|
||||
- output metrics and snapshot cadence.
|
||||
|
||||
Use exact step counts; do not inherit the Matlab `for t=0:dt:end` extra-update ambiguity.
|
||||
|
||||
### Substep 10.1.2 — Integrators
|
||||
|
||||
Compare:
|
||||
|
||||
- Euler;
|
||||
- iterative implicit trapezoid (Euler predictor, clamped fixed-point iteration, bounded count/tolerance);
|
||||
- AB3 with Euler/AB2 startup;
|
||||
- RK4 with clamped stages under both `StageState` and retained historical `StepOrigin` relaxation references;
|
||||
- optional AB4.
|
||||
|
||||
Use the same convolution/rule provider for every method. Record derivative evaluation count and wall time as well as error.
|
||||
|
||||
### Substep 10.1.3 — Metrics and reports
|
||||
|
||||
Compute max, L1, L2, normalized L2, histogram distance, and optional spectrum-band differences against a chosen high-resolution reference. Export CSV/JSON plus plots or plot-ready data.
|
||||
|
||||
Treat the legacy `result.txt` values as historical trend evidence, not bitwise goldens: source parameters and timestep sweeps drifted, and the Matlab loop advanced an extra step. Acceptance is that higher-order methods converge consistently at small `dt`, not exact recreation of every printed number.
|
||||
|
||||
## Phase 10.2 — Backend comparison tool
|
||||
|
||||
Create a command that runs one serialized state through the relevant implementations and compares:
|
||||
|
||||
- standard CPU versus standard GPU FFT;
|
||||
- legacy packed GPU stages versus its CPU stage oracle and direct convolution;
|
||||
- both RK4 relaxation references;
|
||||
- both sphere models against their CPU oracles;
|
||||
- kernels, `M/N/S`, every integration stage, final state, and later aggregate metrics.
|
||||
|
||||
It should stop at the first stage exceeding tolerance and emit both fields for visual diff in the workbench. Include GPU/driver metadata.
|
||||
|
||||
## Phase 10.3 — Preset health sweep
|
||||
|
||||
For every bundled preset:
|
||||
|
||||
1. validate schema and localized historical-option constraints;
|
||||
2. allocate at a reduced smoke-test shape where valid;
|
||||
3. reset deterministically;
|
||||
4. run a short fixed step count;
|
||||
5. assert all values finite and committed state in `[0,1]`;
|
||||
6. record min/max/mean/variance and warnings;
|
||||
7. ensure every inspection channel can be requested.
|
||||
|
||||
Do not assert that chaotic patterns match a screenshot. The sweep detects crashes, NaNs, stale resources, and schema drift.
|
||||
|
||||
## Phase 10.4 — Visualization analysis overlays
|
||||
|
||||
Add optional workbench overlays sourced from core data:
|
||||
|
||||
- CPU/GPU absolute-difference heatmap;
|
||||
- update/clamp saturation mask;
|
||||
- occupancy histogram;
|
||||
- radial kernel profile;
|
||||
- 2-D Fourier magnitude preview;
|
||||
- multiscale contribution stack;
|
||||
- sphere seam error/area distortion;
|
||||
- delayed-time radial age map.
|
||||
|
||||
Keep expensive analysis paused/on-demand; never alter simulation state.
|
||||
|
||||
## Phase 10.5 — Optional Glider Constructor rule lab
|
||||
|
||||
This is explicitly nonblocking for the simulator release. If approved, implement behind `tools`:
|
||||
|
||||
- 80×80 toroidal symmetric drawing canvas;
|
||||
- selectable integer vertical displacement;
|
||||
- 500×500 quantized `(N,M)` lookup assignment;
|
||||
- conflict count/heatmap when one bin demands both outputs;
|
||||
- test simulation using the derived lookup;
|
||||
- import/export for pattern and lookup.
|
||||
|
||||
Keep lookup rules separate from analytic `RuleConfig`; they are a distinct rule-provider type. Document differences from the legacy aid: no bundled glider fixture existed, undefined bins defaulted to `0.5`, and analytic rather than sampled normalization was used.
|
||||
|
||||
## Phase 10.6 — Reproducibility
|
||||
|
||||
Every tool output includes:
|
||||
|
||||
- application/git version;
|
||||
- schema/preset ID;
|
||||
- serialized initial-state hash;
|
||||
- backend and precision;
|
||||
- seed, shape, generation, timestep, and integrator;
|
||||
- CPU thread count or GPU/driver identity;
|
||||
- relevant FFT algorithm, RK4 relaxation reference, and sphere model;
|
||||
- command line/experiment file hash.
|
||||
|
||||
Outputs are collision-safe and can be rerun from a generated manifest.
|
||||
|
||||
## Deliverables
|
||||
|
||||
- Numerical integration experiment runner and reports.
|
||||
- Stage-aware CPU/GPU comparison tool.
|
||||
- Complete bundled-preset health sweep.
|
||||
- On-demand analysis overlays.
|
||||
- Optional inverse-design rule lab only if separately accepted.
|
||||
|
||||
## Exit gate
|
||||
|
||||
- Integration experiments are exact-step reproducible.
|
||||
- Small-timestep convergence order/trends are sensible and documented.
|
||||
- CPU/GPU comparison identifies the first divergent stage.
|
||||
- Every mandatory bundled preset passes the health sweep or is explicitly quarantined with a reason.
|
||||
- Optional rule lab cannot silently masquerade as an analytic SmoothLife preset.
|
||||
Reference in New Issue
Block a user