Variational and Decomposed method for Mitigating Bimodal Bias in Robust Visual Question Answering

Key points

• In this paper, we investigate the multimodal bias in counting and non-counting questions.1
• We provide valuable interpretations of VQA counting mechanism: quantity information stems from the magnitude of the multimodal features in specific dimensions.1
• Thus, we propose a method to deal with them differently: Variational and Decomposed method for mitigating Bimodal Bias (VDB).1
• We introduce variational bias models utilizing the principle of Variational Auto-Encoder to learn mainly language bias with slight vision effect, and a bimodal bias model for bimodal bias combining language and vision sides.1

Sources (1)

• [1]Variational and Decomposed method for Mitigating Bimodal Bias in Robust Visual Question Answering
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 02:42 AM

  Visual question answering (VQA) models are known to suffer from the bias issue, which refers to overfitting the regularities of frequent patterns.

  In this paper, we investigate the multimodal bias in counting and non-counting questions.

Extractive summary: sentences quoted from the sources.