Yhn WarMiddle East first seen 1 d ago, last 34 min ago, peak #2
New Research Claims Pretraining Transformers Without Backpropagation
Original: Dust: Pretraining Transformers Without Backpropagation
Researchers at QLabs have released a method called Dust for pretraining transformer models without using backpropagation, the algorithm at the core of modern deep learning training. The approach is drawing attention for potentially cutting the memory and compute costs of training large language models, though independent validation of the results is not yet clear.
Why now: Replacing backpropagation could significantly reduce the enormous cost of training large AI models, challenging a decades-old foundation of deep learning.
Rank over time, top of the chart is #1. 74 snapshots from 23 h ago to 34 min ago.
Evidence
- Dust: Pretraining Transformers Without Backpropagation · E-Reverance · 275
API: https://socialmediatrends-api.osmike.com/v1/trends/1158303