Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?

Key points

• Concretely, we explore multiple ways to disrupt the original image (e.g., Crop, Shuffle, Mask) to construct self-paired training data, and build a strong baseline (TextRIRO}) upon a conditional diffusion architecture.1
• Our experiments show that combining character-column shuffling with image cropping (Shuffle&Crop) effectively destroys text semantics while preserving key style cues, enabling the model to learn editing behaviors directly from the real text image.1
• With this concise yet real-oriented design, TextRIRO achieves promising performance in both accuracy and visual realism across real-world benchmarks, substantially mitigating the synthetic-to-real domain gap.1
• Furthermore, we leverage TextRIRO to build TextRIRO-3M, a large-scale realistic dataset for long-text recognition by concatenating edited and original images.1

Sources (2)

• [1]Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?
  Hugging Face Daily Papers · Oct 20, 12:00 AM

  Aiming to modify textual content while preserving the original style, Scene Text Editing (STE) has significant practical value in many applications.

  Concretely, we explore multiple ways to disrupt the original image (e.g., Crop, Shuffle, Mask) to construct self-paired training data, and build a strong baseline (TextRIRO}) upon a conditional diffusion architecture.

• [2]Real In, Real Out: What If We Only Use Real Data for Scene Text Editing?
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:58 AM · same content

Extractive summary: sentences quoted from the sources.