Memory as Weights: Internalizing Long-Term History for Streaming Videos

Key points

• Streaming vision-language models (VLMs) must continuously integrate information from the past while remaining responsive to current observations and user queries.1
• MaLoW uses a recurrent HyperNetwork to incrementally consolidate incoming video observations into fixed-size LoRA weights, allowing historical information to be carried forward without revisiting the full video stream.1
• To support training, we further construct MaLoW-Instruct, a large-scale streaming video dataset containing 381K question-answer pairs over 56K videos.1
• We conduct extensive experiments to demonstrate that MaLoW achieves strong performance on OVO-Bench, StreamingBench, and StreamArena.1

Sources (2)

• [1]Memory as Weights: Internalizing Long-Term History for Streaming Videos
  Hugging Face Daily Papers · Oct 15, 12:00 AM

  We propose MaLoW, a streaming memory framework that instead internalizes historical context into evolving lightweight parameter updates.

  Streaming vision-language models (VLMs) must continuously integrate information from the past while remaining responsive to current observations and user queries.

• [2]Memory as Weights: Internalizing Long-Term History for Streaming Videos
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 03:29 PM · same content

Extractive summary: sentences quoted from the sources.