Training Snapshot
Checkpoint 999 captured mid-training. The world model is learning.
The Exceptional Hierarchy
From bulk features through E8 lattice down to S⁷ parallelizable sphere.
Bulk → E8 Lattice
Input features (bulk_dim=512) are projected into E8 lattice space (240 dimensions). The E8 lattice provides optimal sphere packing in 8D, discovered by Viazovska (2017, Fields Medal).
E8 → E7 Projection
E7 projection (133 dimensions) using learned orthogonal transformation. Preserves exceptional algebra structure while reducing dimensionality.
E7 → E6 Contraction
E6 (78 dimensions) hosts the Jordan algebra of 3×3 Hermitian octonionic matrices. Critical for preserving compositional structure.
E6 → F4 Descent
F4 (52 dimensions) is the automorphism group of the exceptional Jordan algebra. Learned projection with small perturbations from canonical embedding.
F4 → G2 Tower
G2 (14 dimensions) is the automorphism group of the octonions. This is where non-associative algebra structure emerges.
G2 → S⁷ Parallelizable Sphere
Final projection to S⁷ (7 dimensions), the only parallelizable sphere after S¹ and S³. Hosts unit octonions—optimal for compositional representations.
S⁷ → Tower (CatastropheKAN)
7 catastrophe types mapped to learnable B-spline control points. Each colony (Spark, Forge, Flow, Nexus, Beacon, Grove, Crystal) has dedicated parameters. Tower output: 28 dimensions.
Learned Representations
What the checkpoint reveals about internal structure at step 999.
μ_self
Self-location in latent space: [0.08, 0.26, -0.06, -0.29, 0.04, 0.14, 0.26]. Encodes "where I am" in the world model's representational space.
E8 → E7
Nearly canonical projection (−√2/2, √2/2 pattern). Preserves the root system structure of E8 → E7 Dynkin diagram folding.
E7 → E6
Identity-dominated transformation. Minimal learned deviation from mathematical canonical projection.
E6 → F4
First learned perturbations appear. Small non-zero off-diagonal elements suggest data-driven adjustment to canonical structure.
F4 → G2
Sparsest projection. Most entries zero (−√2/2 or ±1 where active). Reflects the constraint that G2 sits inside F4 as automorphisms of octonions.
G2 → S⁷
Dense learned projection with no canonical template. Values range [-0.47, +0.46], suggesting full expressivity.
S⁷ → Tower
CatastropheKAN layer: 7 control parameters (range [-0.08, +0.09]) modulate catastrophe bifurcations. Residual gate: 0.098 (10% passthrough).
Tower → E8 VQ
Vector quantization back to E8 lattice for discrete codebook. Enables discrete latent representations with continuous gradients.
Bulk → E8
Entry projection: learned dense matrix (512×240). Followed by LayerNorm for stable gradients. All biases initialized to zero.