• Attention residual 2

  5 min read   ·   March 18, 2026

  2026   ·   Physics-of-AI   Methodology   Optimization   Depth   Data     ·   AI  

• Estimating structural information fraction of your dataset

  4 min read   ·   March 17, 2026

  2026   ·   Physics-of-AI   Methodology   Data   Memory     ·   AI  

• When does Kimi's "Attention Residuals" work?

  6 min read   ·   March 16, 2026

  2026   ·   Physics-of-AI   Methodology   Optimization   Depth   Data     ·   AI  

• When does RandOpt work?

  7 min read   ·   March 15, 2026

  2026   ·   Physics-of-AI   Methodology   Optimization     ·   AI  

• Tokenization 1 -- Factorized tokenization

  4 min read   ·   March 14, 2026

  2026   ·   Physics-of-AI   Methodology   Tokenization     ·   AI