Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio

Key points

• Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio1
• Following Muse Glimmer, Muse Spark Open introduces a mixture-of-experts architecture at a much larger scale, and the most capable US open-weights model to date.2
• Meta launches Muse Spark Open, a coding-optimized 816B mixture-of-experts (MoE) model with 42B active parameters and a 1M token context window.3
• A version of Muse Spark Open was released today, fulfilling their original promise but with a model weaker than their Spark 1.3.4

Sources (14)

• [1]Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio
  Apple Machine Learning Research · Oct 15, 12:00 AM

  In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters — right from your desk, using the latest version of MLX, released today.

  Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio

• [2]Welcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta
  Hugging Face Blog · Oct 15, 12:00 AM

  Following Muse Glimmer, Muse Spark Open introduces a mixture-of-experts architecture at a much larger scale, and the most capable US open-weights model to date.

• [3]Scale Agentic Coding Workflows with Meta’s Muse Spark Open on NVIDIA GB300 NVL72
  NVIDIA Technical Blog · Oct 15, 06:00 PM

  Meta launches Muse Spark Open, a coding-optimized 816B mixture-of-experts (MoE) model with 42B active parameters and a 1M token context window.

• [4][AINews] Agents disprove Polysemanticity; greatly improve Interpretability & Alignment
  Latent Space · Oct 16, 04:16 AM

  A version of Muse Spark Open was released today, fulfilling their original promise but with a model weaker than their Spark 1.3.

• [5]Meta shakes up the frontiers, open and closed
  Interconnects (Nathan Lambert) · Oct 17, 02:06 PM
• [6]Meta Muse Spark Open is live on Together AI
  Together AI Blog · Oct 15, 12:00 AM
• [7]Meta’s Muse Spark Open now available in Amazon SageMaker JumpStart
  AWS Machine Learning Blog · Oct 15, 05:43 PM
• [8]Meta and Apple team up to make Muse AI models run better on Macs
  The Verge: AI · Oct 15, 06:15 PM
• [9]Muse Spark Open is Meta's answer to Qwen, GLM, and Kimi in the open-weight race
  The Decoder · Oct 15, 06:34 PM
• [10]Meta launches an open-weight AI model that runs on Macs
  TechCrunch: AI · Oct 15, 08:04 PM
• [11]Meta Releases Muse Spark Open: 816B Open-Weight MoE With 1M Context
  MarkTechPost · Oct 16, 02:26 AM
• [12]Muse Spark Open, the Most Capable US Open‐Weight Model to Date
  Hacker News: LLM threads (40+ points) · Oct 15, 04:58 PM
• [13]Meta releases 816B-parameter Muse Spark Open for coding and agent work
  SiliconANGLE: AI · Oct 15, 10:20 PM
• [14]Introducing Muse Spark Open: the most capable US open-weight model to date
  r/LocalLLaMA (top, daily) · Oct 15, 05:03 PM

Extractive summary: sentences quoted from the sources.