1
A recurrent model forgets because its hidden state is a small filing clerk, so this paper gives it cached checkpoints and a few ways to query them. It reports gains on language modeling, Needle-in-a-Haystack, retrieval, and LongBench, while still leaving Transformers ahead on in-context recall. Worth clicking if you enjoy the endless committee meeting between attention, recurrence, and the memory budget.
discussion (0)