Nature of Overthinking Phenomenon

In simpler problems, reasoning models often find the correct solution early but then continue exploring incorrect solutions.
Parshin Shojaee[1]
Models first explore incorrect solutions and mostly later in thought arrive at the correct ones.
Parshin Shojaee[1]
The reasoning traces reveal complexity-dependent patterns of explored solutions.
Parshin Shojaee[1]
This indicates LRMs possess limited self-correction capabilities that, while valuable, reveal fundamental inefficiencies.
Parshin Shojaee[1]
Overthinking leads to the waste of compute.
Parshin Shojaee[1]