SAKURA: Serving Attention Key-Value State Under Rate Adaptation for Private Image Editing on Phones

Key points

• Keeping request-specific inference on the device avoids sending personal images and prompts to the cloud.1
• We introduce SAKURA, a cloud-edge architecture that delivers reusable attention state to mobile devices under a bandwidth budget.1
• To enable efficient execution, we develop a pipelined runtime that overlaps cache transfer and reconstruction with on-device execution.1
• We evaluate SAKURA using three editing tasks and two diffusion models, using a Snapdragon 8 Elite Gen 5 phone.1

Sources (1)

• [1]SAKURA: Serving Attention Key-Value State Under Rate Adaptation for Private Image Editing on Phones
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 01:35 PM

  Diffusion transformers are widely used in image-editing services.

  Keeping request-specific inference on the device avoids sending personal images and prompts to the cloud.

Extractive summary: sentences quoted from the sources.