Quantization

Timeline

1. Oct 19, 2026 · Opinion / analysis · 1 source
   How good is FLUX.3 Klein at editing? A 12-task comparison that surprised me
   All we've been hearing for the last few days is how FLUX.3 Klein didn't live up to expectations when it comes to text-to-image.
2. Oct 19, 2026 · Opinion / analysis · 1 source
   Calling all GPUs: help us benchmark SonicBoom v0.0.3 — a compiler-first AI runtime that beats llama.cpp on decode
   SonicBoom is our open-source (GPLv3) inference runtime: a compiler-first alternative to the usual Python stack.
3. Oct 19, 2026 · Model release · 2 sources
   Nemotron 3.5 Bolt, Muse Spark Open, and a Fast LFM2.5 24B
   Nemotron 3.5, with a closer look at Bolt and its “native 2-bit” quantization
4. Oct 18, 2026 · Product / feature launch · 1 source
   72 hours, four big open weights drops. The only one that fits in 24GB came from NVIDIA, of all companies.
   Quick recap, all numbers from the official repos and model cards:
5. Oct 18, 2026 · Opinion / analysis · 1 source
   Is there a Strata for lfm 24b a2b
   Is there a fork of strata that let me run lfm 24b a2b on only 16GB of ram comfortably?
6. Oct 18, 2026 · Model release · 1 source
   Sharing my ultimate krea 2 Asian Character Lora
   Unfortunately I don't have any plan to create any new character krea 2 in near future (since krea 3 is very near to be released as open weights, but I might update the old krea 2 lora to refine the dataset when I see some bad seed/result.
7. Oct 17, 2026 · Opinion / analysis · 1 source
   Do we really need more 5B general-purpose image models?
   The pattern I keep seeing: larger DiTs tend to produce more coherent images.
8. Oct 17, 2026 · Product / feature launch · 1 source
   DeepSeek V4.1 Flash 1017 Support for ds4 · PR #1225 · antirez/ds4
   Just pushed a PR for dwarfstar that adds support for Deepseek-v4.1-Flash-1017 with its new engrams and updated vision tower.
9. Oct 17, 2026 · Open-source release · 2 sources
   REKI: Registering Knowledge in VLAs
   We introduce REKI, a 2.4B vision-language-action model (VLA) pretrained exclusively on public data that sets a new Pareto frontier between performance and model size: on LIBERO-Plus, RoboTwin Clean2Rand and RoboCasa365, it is outperformed only by models about twice its size.
10. Oct 17, 2026 · Tutorial / explainer · 14 sources
    Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio
    In this post, we show how to run the latest frontier-class model — Muse Spark Open, with nearly a trillion parameters — right from your desk, using the latest version of MLX, released today.
11. Oct 17, 2026 · Opinion / analysis · 1 source
    Why not run Qwen 3.8 Flash Next with the full n-gram-table?
    I asked Grok and apparently that only works in llama.cpp et.al and is called the bf16 sidecar, Strata needs FP8 but that should be full quality, over my head to be honest.
12. Oct 17, 2026 · Opinion / analysis · 1 source
    Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming
    I have been working on implementing adaptive KV streaming and RDNA 4 / ROCm optimizations for llama.cpp to enable running Qwen3.8-27B (IQ4XS weights, Q80/Q40 KV cache) with its full 262,144-token context window on a single 16 GB RX 9070 XT GPU at 80+ t/s decode.
13. Oct 17, 2026 · Opinion / analysis · 1 source
    model : add Kolibri-1 (Aleph-Alpha) support by riteme · Pull Request #30673 · ggml-org/llama.cpp
    Kolibri is Aleph Alpha's mixture-of-experts (MoE) reasoning model, with a focus on German and English.
14. Oct 17, 2026 · Tutorial / explainer · 1 source
    justxor/deepseek-local-guide-ru: 🐋 Полный гайд на русском: как установить и запустить DeepSeek локально (V4, R1) на ПК, Mac и сервере — Ollama, llama.cpp, LM Studio, vLLM, O
    🐋 Полный гайд на русском: как установить и запустить DeepSeek локально (V4, R1) на ПК, Mac и сервере — Ollama, llama.cpp, LM Studio, vLLM, Open WebUI, API, RAG, агенты, fine-tuning, лучшие фишки Language: Shell.
15. Oct 16, 2026 · Opinion / analysis · 1 source
    VaeZip: store VAE-encoded latent datasets in 5-10x fewer space with almost no quality loss
    This is VAE latent files that take much more space than original data, while VAE should compress data, not making it more heavy.
16. Oct 16, 2026 · Open-source release · 5 sources
    Welcome Beam: a 501B open model you can own, customize, and scale
    Beam is focused on coding and agentic use cases.
17. Oct 16, 2026 · Opinion / analysis · 1 source
    Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~40-50 tok/sec decode & 1300-1500 tok/sec prefill at 131k ctx on 12GB VRAM + 32GB RAM + NVME
    I got Qwen3.8-Flash-Next-GSQ-RCO-Abliterated running at IQ3S with just 12GB VRAM + 32GB RAM + NVME, achieving 40-50 tok/sec decode & 1300-1500 tok/sec prefill @ 131k context (44k thinking budget), on a custom fork of Strata.
18. Oct 16, 2026 · Model release · 1 source
    Tyger: open-weights streaming ASR + full-duplex turn-control in one model
    Hi r/LocalLLaMA, we're releasing Tyger, a small model that handles full-duplex turn-control for voice agents.
19. Oct 16, 2026 · Research paper · 1 source
    FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper
    We present FUTRANSFORMER, a CUDA implementation of a Llama-style Transformer block for NVIDIA Hopper GPUs.
20. Oct 16, 2026 · Research paper · 1 source
    TUPLE-Bench: Evaluating and Mitigating Privacy-Utility Trade-offs in Typed Decision Models
    We introduce TUPLE-Bench (Typed decision models' Utility and Privacy Leakage Evaluation Benchmark), a benchmark of paired synthetic cases in which recipients may learn a coarse operational category but not its fine-grained cause.
21. Oct 16, 2026 · Research paper · 1 source
    SETAREH Transform: A Sorting-Assisted Haar-Wavelet Sparsification Method for Approximate Inner-Product Computation
    This paper proposes the SETAREH (Sorting-assisted Energy-concentrating Transform for Approximate Representation using Haar wavelets) transform, a Haar-wavelet-based sparsification approach for approximating the inner product between a fixed high-dimensional coefficient vector and successive input vectors.
22. Oct 16, 2026 · Opinion / analysis · 1 source
    MOSS-Image INT8 ComfyUI
    For now, it requires the MOSS-Image custom node, so I’ve included both the install info and a ready-to-use workflow in the repo.
23. Oct 16, 2026 · Research paper · 1 source
    ADT: All-Atom Diffusion Tokenizer for 3D Molecular Structures
    Discrete structural tokenization provides a promising route to autoregressive 3D molecular generation.
24. Oct 16, 2026 · Research paper · 1 source
    Chinese Decision: A Family of Language Models for Single-Pass Probabilistic Decision-Making
    We present Chinese Decision, a family of Chinese-language models for single-pass probabilistic decision-making.
25. Oct 16, 2026 · Benchmark result · 1 source
    Qwen Flash Next potential SOTA Q2 nobody talks about
    Also enhanced loop resitance is advertised.
26. Oct 16, 2026 · Research paper · 1 source
    SpikeQ: Spike-Guided Post-Training Quantization for Spiking Neural Networks
    On neuromorphic chips, spiking neural networks (SNNs) consume far less energy than artificial neural networks (ANNs), but the limited memory of these chips requires low-bit synaptic weights and membrane potentials.
27. Oct 16, 2026 · Product / feature launch · 1 source
    Liquid AI Releases d1-ultra: A Multimodal Decision Model That Scores 65 on the AA Decision Index
    Liquid AI has released d1-ultra, its most capable decision model to date.
28. Oct 16, 2026 · Research paper · 1 source
    Reusing Calibration Gains across Model Representations
    Scale calibration learns an update for one model representation.
29. Oct 15, 2026 · Research paper · 1 source
    Noise-shaped Sampling and Annealing for Scalable On-the-edge Training of Binary and Ternary AI Models
    The training loss of binary and ternary neural networks, expanded as a multilinear polynomial over the discrete weights, is structurally equivalent to the energy of a hierarchical high-order Ising model.
30. Oct 15, 2026 · Model release · 5 sources
    NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
    NVIDIA Nemotron 3.5 Super VL is an open, customizable multimodal model for these specialized workflows.

Often appears with