MERIT: Scaling KG Foundation Models with Relation Transformers

Key points

• KGFMs have been pioneered by ULTRA which builds a relation graph from co-occurrences that match motif templates, and passes messages over it to obtain relation representations.1
• More recent KGFMs such as MOTIF and TRIX increase the size of this relation graph, and FLOCK replaces message-passing with probabilistic random-walk ensembles.1
• To reverse this trend, we design MERIT, the first KGFM with a transformer relation encoder: every relation name attends to every other, and the motifs enter as a soft prior rather than as edges.1
• Fine-tuned for complex query answering like UltraQuery, MERITQuery improves over it at matched cost.1

Sources (1)

• [1]MERIT: Scaling KG Foundation Models with Relation Transformers
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 02:14 PM

  Knowledge graph (KG) foundation models (KGFMs) predict links on unseen graphs zero-shot.

  KGFMs have been pioneered by ULTRA which builds a relation graph from co-occurrences that match motif templates, and passes messages over it to obtain relation representations.

Extractive summary: sentences quoted from the sources.