Stanford

Stanford Researchers Propose Attention-Free Transformer That Cuts Memory Use by 80%

A new paper from Stanford introduces 'Linear Recurrence Networks' that replace attention mechanisms with linear recurrences, achieving comparable quality with dramatically lower memory requirements.