A thread on how forgetting works: from the human brain to why AI models forget after fine-tuning

Ever wondered why we forget things and why even AI models lose knowledge after fine-tuning? Join me as we explore how the human brain's natural forgetting inspires both innovation and challenges in AI.

Human Memory in Action: Our brains forget to avoid overload. Memory decay, interference, and even intentional suppression help us update our knowledge and shield us from painful memories^[7].

AI's Catastrophic Forgetting: When neural networks adapt to new data, their weights shift, sometimes erasing what they once knew. This unintended loss of previous learning mirrors how our brains adjust over time^[3].

Size Matters in AI: Larger models may wow us initially, but during continual fine-tuning they often suffer deeper catastrophic forgetting. As model scale increases, the risk of losing valuable general knowledge grows^[2].