• Memory 2 -- How many bits does each parameter store? An analysis of MLP

  8 min read   ·   March 06, 2026

  2026   ·   Physics-of-AI   Memory   Activation-function     ·   AI  

• Sparse attention 8 -- Numeric randomness speeds up emergence of symbolic structure (induction head)

  4 min read   ·   March 05, 2026

  2026   ·   Physics-of-AI   Sparse-attention   Symbolic     ·   AI  

• Drifting VQ-VAE -- How "drifting models" fixe failure modes of VQ-VAE

  6 min read   ·   March 04, 2026

  2026   ·   Physics-of-AI   Representation   Diffusion     ·   AI  

• Loss landscape visualization 1 -- Seeing sticky plateau

  5 min read   ·   March 03, 2026

  2026   ·   Physics-of-AI   Loss-landscape     ·   AI  

• Research agent 1 -- Reproducing 2026-01-01 blog (physics of feature learning)

  6 min read   ·   February 28, 2026

  2026   ·   Physics-of-AI   Research-agent     ·   AI