A 35B multimodal model running on a phone with ~3GB memory? MiniCPM-V 4.7 demo is pretty wild

Key points

• A 35B-A3B MoE vision-language model running locally on mobile devices is already interesting, but the use cases are what caught my attention:1
• They also released a smaller MiniCPM-V 4.7 1B model for more resource-constrained devices.1
• Another interesting addition is Self-Replay Fine-Tuning (SRFT), which aims to let developers customize the model for specific tasks while keeping general capabilities.1
• The direction is interesting: instead of sending every image/question to the cloud, we're getting closer to having capable multimodal assistants running directly on phones.1

Sources (1)

• [1]A 35B multimodal model running on a phone with ~3GB memory? MiniCPM-V 4.7 demo is pretty wild
  r/LocalLLaMA (top, daily) · Oct 14, 03:53 PM

  Just saw this demo from OpenBMB for MiniCPM-V 4.7, and the on-device capability is pretty impressive.

  A 35B-A3B MoE vision-language model running locally on mobile devices is already interesting, but the use cases are what caught my attention:

Extractive summary: sentences quoted from the sources.