Train LoRA directly on Quantized GGUFs, no PyTorch, no Parent(FP16) needed, just your GGUF and the data
train against the quantized GGUF file itself
Proof1 community thread
Key points
- I've seen this one floating around quite a bit now and since this is something that can be done without the parent and or other dependencies i thought i'd share something.
- use the exact same GGUF for inference afterwards, without converting the model
- (QLoRA does train through a 4-bit base, but in PyTorch and on its own NF4 format, not on the GGUF you serve.)
- With runner the 16-bit nor the PyTorch package is needed, this is done through "freezing" the weights of the current X-bit GGUF, only a "small" LoRA adapter is trained on top.
Sources (1)
- [1]Train LoRA directly on Quantized GGUFs, no PyTorch, no Parent(FP16) needed, just your GGUF and the datar/LocalLLaMA (top, daily) · Oct 14, 05:34 AM
- train against the quantized GGUF file itself
I've seen this one floating around quite a bit now and since this is something that can be done without the parent and or other dependencies i thought i'd share something.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026Gausify Orbit is now open source: turn AI-generated 360 orbit videos into Gaussian Splats, entirely in your browser
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
- Oct 13, 2026unslothai/unsloth v0.1.907-beta: Large Performance Gains + More Browser Use
- Oct 13, 2026Gemini 4 Argon: Built to solve your hardest problems
- Oct 13, 2026I built a pytorch library of pretrained models for 3D point clouds following timm standards [P]
- Oct 13, 2026SonicBoom: 52 tok/s decode on a laptop RTX 3050 (llama.cpp: 45) with the same quantized kernels, the difference is a graph compiler. No Python.
