• Fine-tuning with sparse updates? A toy teacher-student Setup

  6 min read   ·   January 07, 2026

  2026   ·   Physics-of-AI   Science-for-AI   AGI     ·   AI  

• Multi-Head Cross Entropy Loss

  5 min read   ·   January 06, 2026

  2026   ·   Physics-of-AI   Science-for-AI   AGI     ·   AI  

• What's the difference -- (physics of) AI, physics, math and interpretability

  9 min read   ·   January 05, 2026

  2026   ·   Physics-of-AI   Methodology     ·   AI  

• Representation anisotropy from nonlinear functions

  5 min read   ·   January 04, 2026

  2026   ·   Physics-of-AI   Non-linearity     ·   AI  

• Training dynamics of A Single ReLU Neuron

  7 min read   ·   January 03, 2026

  2026   ·   Physics-of-AI   Non-linearity     ·   AI