SIREN: Situation-aware Index for Response and Emergency ActioNability in Crisis Translation

Key points

• Machine translation (MT) is increasingly utilized in multilingual crisis communication, yet conventional evaluation does not directly assess whether translated emergency messages preserve their actionability.1
• At the error level, we apply it to HOPE-annotated Italian--English translations produced by ChatGPT-4o and Google Translate: 31.63% and 48.73% of errors, respectively, compromise actionability, predominantly by affecting the protective action.1
• At the message level, we evaluate English--French translations produced by TranslateGemma-4B and Google Translate, with actionability preserved in 85.34% and 92.67% of translations, respectively.1
• Alongside SIREN, we contribute an English--French crisis-translation resource with human actionability annotations and post-edited reference translations, supporting further research on function-oriented evaluation for crisis communication.1

Sources (1)

• [1]SIREN: Situation-aware Index for Response and Emergency ActioNability in Crisis Translation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 12:06 PM

  We introduce SIREN (Situation-aware Index for Response and Emergency ActioNability), a human evaluation framework assessing whether translation choices compromise protective action, directive force, and urgency of the source text.

  Machine translation (MT) is increasingly utilized in multilingual crisis communication, yet conventional evaluation does not directly assess whether translated emergency messages preserve their actionability.

Extractive summary: sentences quoted from the sources.