• A Toy Model of Masked-Autoencoder (MAE)

  5 min read   ·   April 09, 2026

  2026   ·   Physics-of-AI   Representation   MAE     ·   AI  

• Understanding cardinality (as in ResNext) from representation collapse

  4 min read   ·   April 08, 2026

  2026   ·   Physics-of-AI   Representation   Cardinality     ·   AI  

• Attention residual 2

  5 min read   ·   March 18, 2026

  2026   ·   Physics-of-AI   Optimization   Depth   Data     ·   AI  

• Estimating structural information fraction of your dataset

  4 min read   ·   March 17, 2026

  2026   ·   Physics-of-AI   Data   Memory     ·   AI  

• When does Kimi's "Attention Residuals" work?

  6 min read   ·   March 16, 2026

  2026   ·   Physics-of-AI   Optimization   Depth   Data     ·   AI