Yhn WarMiddle East first seen 2 d ago, last 19 h ago, peak #2

Dust: Pretraining Transformers Without Backpropagation

Researchers at QLabs have published work on Dust, a method for pretraining transformer models without using backpropagation, the gradient-based algorithm that underpins nearly all modern deep learning. The approach, detailed in a new research paper, is drawing attention from machine learning practitioners debating whether alternatives to backpropagation could reduce the memory and compute costs of training large language models.

Why now: Interest in cheaper alternatives to backpropagation for training large AI models is high, and a credible new method challenges a foundational assumption of deep learning.

QLabsDusttransformersbackpropagation

Open on hn →

Evidence

• Dust: Pretraining Transformers Without Backpropagation · E-Reverance · 276

API: https://socialmediatrends-api.osmike.com/v1/trends/1158303