CARTA: Self-Supervised Reusable Transaction Embeddings via Heterogeneous Tokenization and Cross-Attention

Key points

• Acquiring fraud labels for payment transactions is expensive and inconsistent across tasks, and high-cardinality identifiers such as merchant names make one-hot feature pipelines infeasible at scale.1
• CARTA uses type-specific projections with learned embedding tables for high-cardinality identifiers, trained with token excision and InfoNCE contrastive loss.1
• On IEEE-CIS, its linear probe at 1% of labels exceeds a raw-feature probe at 100%; on IBM TabFormer (2M rows), its frozen embedding reaches GBDT AUPRC 0.508, above frequency encoding (0.411) and the merchant-dropping baseline (0.022), though fully-supervised CatBoost is stronger.1
• CARTA is a label-free, reusable representation that complements task-specific supervised models.1

Sources (1)

• [1]CARTA: Self-Supervised Reusable Transaction Embeddings via Heterogeneous Tokenization and Cross-Attention
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 11:56 PM

  We present CARTA, a self-supervised encoder that learns a compact 16-dim embedding from raw transactions without labels; the frozen embedding is then reused for label-efficient fraud probing and exploratory unsupervised risk stratification without retraining the encoder.

  Acquiring fraud labels for payment transactions is expensive and inconsistent across tasks, and high-cardinality identifiers such as merchant names make one-hot feature pipelines infeasible at scale.

Extractive summary: sentences quoted from the sources.