Text-Only vs. Image-Aware VLM Judges for Manga Translation Evaluation
In manga translation, visual context outside speech-bubble text, including speaker identities, referents, and scene tone, is depicted in page images, making it natural to refer to images during translation evaluation.
ProofPaper ↗
Key points
- However, most conventional manga translation evaluations rely solely on text-only automatic metrics, leaving the impact of image input insufficiently explored.
- We propose a practical validation framework that employs the same vision-language model (VLM) as both a text-only judge (TOJ) and an image-aware judge (IAJ) to investigate evaluation divergences associated with image-aware evaluation procedures.
- We use a Japanese-English manga dataset of official translations aligned at the page and speech-bubble levels.
- First, adding page images to TOJ improved the primary VLM's detection of deliberately injected, visually dependent degradations.
Sources (1)
- [1]Text-Only vs. Image-Aware VLM Judges for Manga Translation EvaluationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 07:59 AM
In manga translation, visual context outside speech-bubble text, including speaker identities, referents, and scene tone, is depicted in page images, making it natural to refer to images during translation evaluation.
However, most conventional manga translation evaluations rely solely on text-only automatic metrics, leaving the impact of image input insufficiently explored.
Extractive summary: sentences quoted from the sources.