• Optimization 2 -- Elementwise Scale Reparametrization

  4 min read   ·   January 24, 2026

  2026   ·   Physics-of-AI   Optimization     ·   AI  

• Optimization 1 -- Norm reparametrization

  5 min read   ·   January 23, 2026

  2026   ·   Physics-of-AI   Optimization     ·   AI  

• Sparse attention 5 -- Attention sink

  6 min read   ·   January 22, 2026

  2026   ·   Physics-of-AI   Sparse-attention     ·   AI  

• Bigram 4 -- On the difficulty of spatial map emergence

  7 min read   ·   January 21, 2026

  2026   ·   Physics-of-AI   Bigram   Toy-language     ·   AI  

• Depth 1 -- Understanding Pre-LN and Post-LN

  7 min read   ·   January 20, 2026

  2026   ·   Physics-of-AI   Depth     ·   AI