GgoogleIN first seen 1 d ago, last 1 d ago, peak #6
Apple researchers run large LLM inference across Mac Studios
Original: threads
Apple's Machine Learning Research team has detailed running large language model inference both locally and in a distributed setup using its MLX framework, with RDMA networking connecting multiple Mac Studio machines. The work shows commodity Apple hardware can be clustered to serve models that would normally require data-centre GPUs, prompting discussion about the practicality of Mac clusters for AI workloads.
Why now: Interest in cheaper, consumer-hardware alternatives to GPU clusters for running large AI models.
Evidence
- Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio · Apple Machine Learning Research
API: https://socialmediatrends-api.osmike.com/v1/trends/3060781