• MOE 1 -- Experssive power of MOEs through the lens of spectral bias and memorization capacity

  4 min read   ·   March 10, 2026

  2026   ·   Physics-of-AI   MLP   MOE   Memorization   Spectral-bias     ·   AI  

• A toy model of video generative models -- bottleneck dimension controls "classical"/"quantum" strategies

  7 min read   ·   March 09, 2026

  2026   ·   Physics-of-AI   Representation   Diffusion   Flow-matching   Autoencoder   Video-generative-model     ·   AI  

• Memory 2 -- How many bits does each parameter store? An analysis of MLP

  8 min read   ·   March 06, 2026

  2026   ·   Physics-of-AI   Memory   Activation-function     ·   AI  

• Sparse attention 8 -- Numeric randomness speeds up emergence of symbolic structure (induction head)

  4 min read   ·   March 05, 2026

  2026   ·   Physics-of-AI   Sparse-attention   Symbolic     ·   AI  

• Drifting VQ-VAE -- How "drifting models" fixe failure modes of VQ-VAE

  6 min read   ·   March 04, 2026

  2026   ·   Physics-of-AI   Representation   Diffusion     ·   AI