Gauge Structure and Architectural Inductive Bias for Axial Parity Codes: A Theorem and a Negative Result

1. In one sentence (and why it matters)

The set of voxel grids that satisfy a particular family of parity checks is exactly the image of a discrete third derivative $\partial_x\partial_y\partial_z$ over the two-element field $\mathbb{F}_2$ — which makes the codec a discrete fracton gauge theory, cousin to surface-code quantum error correction — and yet baking that exact structure into a neural decoder's architecture hurts it, while a plain gauge-free multi-scale autoencoder wins.

The two halves of the paper.

1. A theorem (a positive, provable result). The paper pins down the codespace — the set of all voxel fields that pass every parity check — in closed form. It is the image of the operator $\partial_x\partial_y\partial_z$ acting over $\mathbb{F}_2$, with rank $3n^2-3n+1$. That equation is not just a curiosity: it identifies the codec as living in the same algebraic family as well-known physics models (fracton gauge theory) and quantum error-correcting codes (the surface code), and it caught a real bug in deployed code.
2. A negative result (an experimental finding that the "obvious" idea fails). The theorem hands you a tempting recipe: since every valid codeword is $\partial_x\partial_y\partial_z\phi$ for some potential $\phi$, just have the network predict $\phi$ and apply the operator. The paper builds exactly that ("phi-prior") decoder and shows it plateaus far below an unconstrained baseline. A generic multi-scale autoencoder with no gauge structure beats everything.

2. Background you need (from first principles)

The paper sits at the intersection of three worlds: data compression, the algebra of error-correcting codes, and a sliver of theoretical physics. You don't need to be a specialist in any of them. Here is every prerequisite, built up plainly.

2.1 Voxels, occupancy fields, and sparsity

A voxel is a 3D pixel — one cell of a 3D grid. The paper works on a $32\times32\times32$ grid, i.e. $n=32$ per side, so $n^3 = 32{,}768$ cells. A binary occupancy field assigns each voxel a 0 or 1: 1 means "this bit of space is solid," 0 means "empty." A chair, a car, an airplane — any shape from the ShapeNet dataset — becomes a cube of 0s and 1s.

These fields are sparse: most voxels are 0, and the 1s cluster on the surface of the object (the shell), not its interior. Remember this word "surface-concentrated" — it becomes the punchline at the end of the paper.

2.2 GF(2) / $\mathbb{F}_2$ and parity

$\mathbb{F}_2$ (also written GF(2)) is the field with two elements, $\{0,1\}$, where addition is XOR (exclusive-or): $$0+0=0,\quad 0+1=1,\quad 1+1=0.$$ "$1+1=0$" is the whole story: adding two 1s wraps back to 0, i.e. everything is done mod 2. A parity of a set of bits is just their XOR — equivalently, "is the number of 1s odd or even?" Parity is the simplest possible error-detecting check.

2.3 The axial parity codec: syndromes and codespaces

The codec the paper studies sends a compressed description of the voxel grid using three families of parity checks:

• Line checks. For each axis-aligned line through the grid (a full row of 32 voxels along $x$, $y$, or $z$), compute the parity of the voxels on that line. There are $3n^2$ such lines.
• Plane checks. For each axis-perpendicular plane (a $32\times32$ slab), compute the parity of all voxels in it.
• Global check. The parity of the entire grid — one bit.

Stack every check into a big matrix $H$ over $\mathbb{F}_2$. Applying $H$ to a flattened voxel field $x$ gives a vector $Hx$ called the syndrome. The syndrome is the list of all the parity answers. Crucially, the syndrome is much smaller than the grid: it's the compressed thing you transmit.

Two definitions that the whole paper rides on:

• The codespace $\mathcal{C}(n) = \ker H$ is the set of voxel fields whose syndrome is all-zero — every line, plane, and global parity is even. (In coding-theory language, the kernel of the parity-check matrix.)
• If two fields $x$ and $x'$ have the same syndrome, then $x \oplus x'$ (their XOR) lies in the codespace, because $H(x \oplus x') = Hx \oplus Hx' = 0$.

How decoding works here. Given a syndrome, the decoder first builds a fixed, canonical guess $x_{\text{base}}$ that matches the syndrome. The true field $x$ differs from it by a residual $r = x \oplus x_{\text{base}}$. Since $x$ and $x_{\text{base}}$ share the syndrome, the residual is guaranteed to lie in the codespace. The learning task is: predict $r$ (the codespace residual) from $x_{\text{base}}$, the syndrome, and optional side-info. The reconstruction is $\hat{x} = x_{\text{base}} \oplus \hat{r}$.

2.4 Stabilizer codes, the surface code, and QEC (the quantum cousin)

The paper repeatedly compares the parity codec to quantum error correction (QEC), specifically the surface code. You don't need quantum mechanics to follow the analogy; here is the bridge in classical terms.

• A stabilizer code is defined by a set of parity-like checks (stabilizers). The surface code lays qubits on a 2D lattice and measures local 4-qubit parity checks.
• A syndrome in QEC is the set of stabilizer measurement outcomes — exactly analogous to $Hx$ here. When you run the code over many time rounds, the syndrome becomes a 3D grid of detector cells (2D space + 1D time).
• A decoder infers, from the syndrome, which logical-error class occurred — i.e. which equivalence class of correction to apply. Classic decoders include MWPM (minimum-weight perfect matching) and BP+OSD (belief propagation + ordered statistics decoding); modern ones use neural networks. The paper's deployed surface-code decoder is a 3D ResNet.
• Code distance $d$ is the size of the smallest undetectable error — bigger $d$ = more robust code. The exploratory QEC runs use $d \in \{5,7,9\}$.

The abstract structure is identical: syndrome-conditioned reconstruction on a regular lattice over $\mathbb{Z}_2$. That shared skeleton is precisely what the theorem makes formal.

2.5 Gauge theory, "field strength = derivative of a potential," and fractons

This is the physics flavor. Keep it light; the only thing you must internalize is one analogy.

Fractons are exotic particles in certain 3D lattice models that cannot move freely — their motion is restricted to lines or planes, because various line and plane charges are separately conserved. Pretko's scalar-charge fracton gauge theory has exactly this structure, and its conserved charges (total charge, planar charges, line charges) line up one-to-one with the codec's global, plane, and line parity checks. That correspondence is the physics content of the theorem. The surface code is "one rung down" this hierarchy: a $\mathbb{Z}_2$ 1-form gauge theory (the Wegner–Kitaev kind), whereas the fracton theory is higher-rank.

2.6 IoU: the evaluation metric

IoU (intersection-over-union) measures how well two binary shapes overlap: $$\text{IoU}(\hat{x}, x) = \frac{|\hat{x}\cap x|}{|\hat{x}\cup x|} = \frac{\#\{\text{voxels that are 1 in both}\}}{\#\{\text{voxels that are 1 in either}\}}.$$ It runs from 0 (no overlap) to 1 (perfect). For sparse, surface-concentrated shapes it is a far more honest metric than raw accuracy — a decoder that predicts "all empty" would score $\sim$99% accuracy but IoU $\approx 0$.

2.7 Inductive bias, hard vs. soft constraints, equivariance

An inductive bias is any assumption baked into a model that biases it toward some solutions over others (e.g. a CNN's translation bias). A hard constraint forces the model's output to satisfy a property exactly by construction (here: outputting only codewords of the form $\partial_x\partial_y\partial_z\phi$). A soft constraint instead penalizes violations in the loss, leaving the model free to break the rule when it helps. Equivariance is the analogous idea for symmetries. The paper's running theme: hard enforcement can wreck the optimization landscape, even when the constraint is mathematically correct.

3. What the paper actually does

3.1 The theorem (Section 3)

The central claim, verbatim in spirit:

"On the 3-torus" just means the grid wraps around at the edges (periodic boundaries) — standard for this kind of lattice. Here $(\partial_a\phi)_v := \phi_v + \phi_{v+\hat{e}_a} \pmod 2$ is the discrete forward difference along axis $a$. The composite $\partial_x\partial_y\partial_z$ takes a potential $\phi$ and, at each voxel, XORs the 8 corner values of the little cube anchored there.

The proof in plain language (it really is elementary, which is part of the paper's point):

1. Tensor structure. Identify the grid $\mathbb{F}_2^{n^3}$ with $V\otimes V\otimes V$ where $V=\mathbb{F}_2^n$ is a single axis. Then the difference operator factorizes cleanly: $\partial_x = \partial\otimes I\otimes I$, etc., and $T := \partial_x\partial_y\partial_z = \partial\otimes\partial\otimes\partial$, where $\partial = I + S$ ($S$ = cyclic shift).
2. Codespace as a complement. Each check is an inner product of $x$ with the indicator of a line/plane/whole-lattice. So $\mathcal{C}(n) = \ker H = R^\perp$, where $R$ is the span of all check indicators. Planes are sums of lines and the global is a sum of planes, so $R$ is spanned by line indicators alone.
3. Span of lines = "constant along an axis." The $x$-lines span the subspace $K_x$ of fields that are constant along $x$; similarly $K_y, K_z$. So $R = K_x + K_y + K_z$.
4. Dimension by inclusion–exclusion. Pick a basis containing the all-ones vector; then $K_x, K_y, K_z$ are coordinate subspaces, so dimensions add exactly: $\dim K_a = n^2$, $\dim(K_a\cap K_b)=n$, $\dim(K_x\cap K_y\cap K_z)=1$. Hence $\dim R = 3n^2 - 3n + 1$, which is the rank of $H$.
5. Kernel of $T$ equals $R$. On the $n$-cycle, $\ker\partial = \langle\mathbf{1}\rangle$ (only constants are killed). The tensor kernel rule gives $\ker T = K_x + K_y + K_z = R$, so $\dim\,\mathrm{image}(T) = n^3 - \dim R = \dim\mathcal{C}(n)$.
6. Equality. Every line/plane/global sum of $T\phi$ telescopes to 0 mod 2 (each $\phi_v$ appears an even number of times), so $\mathrm{image}(T)\subseteq\mathcal{C}(n)$. Two nested subspaces of equal finite dimension are equal. Done.

3.2 The experimental setup (Section 4)

• Data: ShapeNet binary voxels at $32^3$; bug-fixed $H$ with syndrome dimension $M=2977$.
• Target: the residual $r = x\oplus x_{\text{base}}$ (guaranteed in the codespace). Loss: binary cross-entropy on $r$, optionally with positive-class weighting (since residuals are mostly 0).
• Metric: IoU on $\hat{x}=x_{\text{base}}\oplus\hat{r}$, on a held-back validation set (not a disjoint test split — flagged as a limitation).
• Defaults: 2,048 train / 512 val samples, batch size 8, AdamW, lr $5\times10^{-4}$, cosine schedule. They report best validation IoU during training.

The four architectures compared:

3.3 The key idea, stated simply

The theorem says: every valid codeword is $\partial_x\partial_y\partial_z\phi$. The "obvious" architectural move is to make the network predict $\phi$ and apply the operator as a fixed, non-learnable output layer — this guarantees the output is always a valid codeword (the phi-prior). The paper's experiments test whether this guarantee helps or hurts. Spoiler from the abstract: it hurts decisively, and a gauge-free multi-scale autoencoder wins.

4. The results, explained

4.1 Hard enforcement plateaus far below baseline (Table 1)

Across 14 configurations — varying capacity, training time, and several optimization tricks — the phi-prior plateaus at validation IoU $\approx 0.29$, versus the unconstrained x-prior at $\approx 0.48$ at matched compute. These are the paper's actual numbers (best validation IoU, single-seed per cell):

Read it like this. The plateau is robust to capacity (5M $\approx$ 12M), training length (20 $\approx$ 30 epochs), positive weighting (sweet spot at pos_weight=19), and data scale (2K $\approx$ 8K). The phi-prior sometimes shows a phase-transition jump from $\sim$0.13 to $\sim$0.21 around epoch 12, but always saturates near 0.29. None of the obvious levers rescue it.

4.2 What works instead: a gauge-free multi-scale prior (Table 2)

Two controlled ablations isolate the cause — and this is the part a reviewer should love, because it does the hard work of localizing why:

1. Remove all the soft U(1) transporter's auxiliary curvature/transport/latent-reconstruction losses, and it gets slightly better (0.509 → 0.518). So the gauge regularizers are not load-bearing.
2. Drop the parallel-transport operator entirely — leaving a plain multi-scale autoencoder — and it improves further (4.1M: 0.554; 16M: 0.572). The win is multi-scale processing, not gauge structure.

The three-seed headline (the only error-barred row) is autoencoder seeds {42,43,44} → {0.5764, 0.5692, 0.5710}, mean 0.5722, stdev 0.0038 (from REPRODUCIBILITY.md; rounds to the $0.572\pm0.004$ in the table).

4.3 The deferred QEC question (Section 7) — explicitly NOT a result

Since the theorem unifies parity compression and surface-code QEC algebraically, it's natural to ask whether the multi-scale prior also helps QEC decoding. The paper deliberately reports no QEC numbers as a result. Preliminary single-seed runs suggest the opposite direction: aggressive spatial bottlenecks appear to destroy the per-detector-cell information that dense surface-code syndromes carry (unlike the sparse occupancy fields). The paper states only a hypothesis: the multi-scale advantage is specific to sparse-with-smoothness signals and may not transfer to dense per-cell syndromes. Per REPRODUCIBILITY.md, the QEC d=5 autoencoder over 3 seeds gave {+13.2%, +3.9%, −3.0%} vs MWPM — mean +4.7%, stdev $\approx$8%, i.e. statistically tied — which is exactly why it was demoted to a hypothesis. This is good scientific hygiene, not a result to be cited.

4.4 Are the results surprising?

The theorem: not surprising once seen — the proof is elementary — but genuinely useful, both as a unifying identification (fracton ↔ codec ↔ surface code) and as a practical bug-catcher. The contribution is the specific identification plus the closed-form rank, not the fracton↔code dictionary itself (which is established).

The negative result: mildly surprising at first (the "obvious" architecture should at least be competitive) but consistent with a known broader pattern — hard constraint/equivariance enforcement often underperforms soft priors (cf. Finzi et al.'s residual pathway priors). The value is that it's a clean, controlled, coding-theoretic instance with an ablation pinning down the cause.

5. How to read it like a skeptical reviewer

Here is where to press, what the baselines are, what would falsify the claims, and the limitations the paper itself already concedes.

5.1 The claims, and what would falsify each

5.2 Where to push hardest

• Single-seed rows. Only the 16M autoencoder row is multi-seed (seeds 42/43/44). The x-prior, phi-prior, transporter, and the entire bottleneck sweep are single-seed (seed 42). The paper is upfront that the conclusions rest on gaps far larger than plausible seed noise and on the directionality of the two ablations, not on point estimates — a fair defense given the 0.29 vs 0.48 gap dwarfs the 0.004 stdev seen on the one multi-seed row, but a reviewer is entitled to ask for error bars on the phi-prior plateau too.
• Validation, not test. Every IoU is best-validation IoU on a 512-sample held-back split, not a disjoint held-out test set. Reporting "best validation during training" also mildly inflates numbers (it's a selection over checkpoints). The paper flags this in Limitations; a stricter standard (the codebase's own QEC pipeline uses disjoint test splits) is not applied here.
• Matched compute, not matched params. Table 2 rows are matched in training budget but not exactly in parameter count — the winning autoencoder is 16M while the phi-prior/x-prior are 5M. The fairest single comparison inside the table is x-prior (5M, 0.481) vs phi-prior (5M, 0.283), which is matched; the 16M vs 5M autoencoder comparison is partly a capacity story. Note the x-prior at 17.9M is 0.474, so capacity alone does not buy the autoencoder's 0.572 — that's the authors' implicit rebuttal.
• Was the phi-prior given a fair shot? The mechanism argument (attenuated gradient, non-local 8-corner credit assignment) is plausible, but a determined reviewer could ask: did they try a continuous relaxation that anneals to the hard XOR, a better-conditioned operator, or predicting $\phi$ at a coarser scale? The paper tried several tricks (bias init, STE, residual escape hatch, multi-scale capacity) but the space of fixes is large.

5.3 The baselines — are they honest?

Yes, mostly. The x-prior is a genuinely unconstrained version of the same backbone, which is the right control for the phi-prior (only the output operator changes). The two ablations on the transporter are textbook "remove one thing at a time." The honest weakness is the param mismatch on the winning row, which the paper concedes.

5.4 Limitations the paper states itself

1. All IoU is validation, not a disjoint test split.
2. Only the autoencoder headline is multi-seed; the rest are single-seed exploratory.
3. Table 2 is matched in training budget but not exactly in parameter count.
4. Results are at $32^3$ on a single object dataset (ShapeNet).
5. The cross-domain QEC discussion is a hypothesis, explicitly not a measured result.

6. Glossary

$\mathbb{F}_2$ / GF(2)

The two-element field $\{0,1\}$ with XOR for addition ($1+1=0$). All the codec's algebra lives here.

Voxel / occupancy field

A 3D pixel; a $32^3$ grid of 0/1 values describing a solid shape. Sparse and surface-concentrated.

Parity check

The XOR (odd/even count) of a chosen set of bits. The codec uses line, plane, and global parities.

Parity matrix $H$

The matrix stacking all checks; $Hx$ is the syndrome. At $n=32$, $\mathrm{rank}(H)=2977$.

Syndrome

The vector of all parity answers, $Hx$. The compressed thing transmitted; same role as stabilizer measurements in QEC.

Codespace $\mathcal{C}(n)$

$\ker H$: all fields whose syndrome is zero. The theorem identifies it with $\mathrm{image}(\partial_x\partial_y\partial_z)$.

Residual $r$

$x\oplus x_{\text{base}}$; the difference between truth and the canonical syndrome-matching guess. Guaranteed to lie in the codespace; it is the learning target.

$\partial_a$ (discrete forward difference)

$(\partial_a\phi)_v = \phi_v + \phi_{v+\hat{e}_a}\pmod 2$. The $\mathbb{F}_2$ derivative along axis $a$.

$\partial_x\partial_y\partial_z$ ($T$)

The mixed third difference: at each voxel, XOR the 8 corners of its unit cube. The codec's "field strength" operator.

Gauge potential $\phi$

The hidden field whose third derivative gives a codeword. Many $\phi$ map to the same codeword (gauge freedom).

Fracton / scalar-charge gauge theory

3D lattice models with immobile or restricted-mobility excitations due to conserved line/plane charges — matching the codec's line/plane parities (Pretko 2017).

Surface code

A 2D $\mathbb{Z}_2$ stabilizer code (Kitaev 2003); a 1-form gauge theory "one rung below" the fracton theory. Its decoder maps syndromes to logical-error classes.

Stabilizer code / stabilizer

A code defined by parity-like checks (stabilizers); their measurement outcomes form the syndrome.

Code distance $d$

Size of the smallest undetectable error; larger = more robust. QEC runs used $d\in\{5,7,9\}$.

MWPM / BP+OSD

Classical QEC decoders: minimum-weight perfect matching; belief propagation + ordered-statistics decoding. Baselines for learned decoders.

IoU

Intersection-over-union of two binary shapes; the reconstruction metric (0 to 1).

DCT side-info

A few low-frequency discrete-cosine coefficients (JPEG-style) sent as a coarse sketch / explicit smoothness prior. Not relied on for the main results.

Inductive bias

Assumptions baked into a model that favor some solutions. Hard = enforced exactly by construction; soft = penalized in the loss.

Equivariance

A model property where transforming the input transforms the output predictably; the symmetry analogue of the constraints discussed here.

x-prior / phi-prior / transporter / autoencoder

The four compared decoders: unconstrained baseline / hard-gauge / soft-gauge / gauge-free multi-scale.

7. Key takeaways

This tutorial reproduces the paper's actual claims and headline numbers (Tables 1 and 2 of main.tex, and figures from REPRODUCIBILITY.md) for pedagogical verification. No numbers were invented; where this page restates a value it is traceable to the paper or its reproducibility doc. The theorem verifier is verify_fracton_theorem.py; all experiments are catalogued in experiments.tsv under fracton_xcube/src/gauge_paper/.