SCMerge: Spatial-Consistency-Aware Token Pair Merging for Efficient Semantic Segmentation

Key points

• Vision Transformer architectures have showcased impressive performance on semantic segmentation, yet their quadratic attention complexity hinders practical applications.1
• However, existing token merging methods primarily rely on feature similarity to identify similar tokens, overlooking spatial consistency, which may introduce segmentation errors in semantic segmentation.1
• The proposed method is flexible and can be plugged into existing vision Transformer-based semantic segmentation pipelines.1
• Extensive experiments on the benchmark datasets ADE20K, Pascal Context, and Cityscapes show that SCMerge consistently achieves a better trade-off between segmentation accuracy and efficiency than baseline methods.1

Sources (1)

• [1]SCMerge: Spatial-Consistency-Aware Token Pair Merging for Efficient Semantic Segmentation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 10:23 AM

  In this work, we propose SCMerge, a training-free token merging method that jointly considers feature similarity and spatial consistency in pairing similar tokens.

  Vision Transformer architectures have showcased impressive performance on semantic segmentation, yet their quadratic attention complexity hinders practical applications.

Extractive summary: sentences quoted from the sources.