QCATS: Query Context-Aware Transformer Slicing for Efficient Predictive Query Processing

Key points

• In-database predictive query processing increasingly applies Transformer-based models within relational pipelines.1
• However, existing in-database inference typically exposes only tuple-level model inputs to the inference runtime, leaving relational predicates and metadata statistics invisible to neural execution planning.1
• QCATS executes at query granularity: instead of routing individual tokens or tuples during inference, it uses query predicates and metadata statistics to pre-select context-aligned FFN slices before model execution.1
• Experiments on four predictive-query workloads with BERT-base and Qwen-0.6B show that QCATS achieves up to 4.42x latency reduction while preserving prediction accuracy comparable to dense baselines.1

Sources (1)

• [1]QCATS: Query Context-Aware Transformer Slicing for Efficient Predictive Query Processing
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 11:52 AM

  In this paper, we propose QCATS, a query context-aware transformer slicing framework that enables efficient sparse inference inside database systems.

  In-database predictive query processing increasingly applies Transformer-based models within relational pipelines.

Extractive summary: sentences quoted from the sources.