Mica v0.5: The smartest 4B decision model is also the fastest one. 20 ms per answer, 25x faster than Jev
I'm sharing Mica v0.5, the follow-up to Mica v0.1.
Proof1 community thread
Key points
- On speed, it takes 19.8 ms per decision at 256 tokens and about 1.2 s at 32k on a single RTX 6000 Ada.
- That's up to 3.8x faster than the other 4B judges I measured, 12 to 19x faster than Perplexity's 27B decider, and 25 to 340x faster than the API decision models.
- On top of the optimizations described on HF, Mica Fast makes long inputs about 1.4 to 1.5x faster again.
- On accuracy, it gets 84/111 on JevBench Hard, the best of any 4B model I could find (only Microsoft's 9B Decision-1 is higher).
Sources (1)
- [1]Mica v0.5: The smartest 4B decision model is also the fastest one. 20 ms per answer, 25x faster than Jevr/LocalLLaMA (top, daily) · Oct 21, 07:00 PM
I'm sharing Mica v0.5, the follow-up to Mica v0.1.
On speed, it takes 19.8 ms per decision at 256 tokens and about 1.2 s at 32k on a single RTX 6000 Ada.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 21, 2026Granite Speech 5.0: Transcribe, Translate, and Attribute more Accurately and Faster
- Oct 21, 2026Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets
- Oct 20, 2026XHToken/Spark-X2.6-4B · Hugging Face
- Oct 20, 2026[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
- Oct 20, 2026AnyLanguageModel 1.0: One Year of Local Models on Apple Platforms
- Oct 19, 2026Developers can now access SageMaker Studio through the new AWS experience
