FedAlphaEdit: Null-Space-Aligned Merging for Collaborative Knowledge Editing
Null-space-constrained editing methods such as AlphaEdit mathematically guarantee that each update leaves unrelated knowledge intact, while collaborative frameworks such as CollabEdit aggregate edits from multiple clients without data sharing.
ProofPaper ↗
Key points
- Multiple institutions may each hold their own private knowledge edits and wish to integrate them into a single large language model without sharing raw edit requests.
- Guided by this analysis, we propose FedAlphaEdit.
- To our knowledge, this is the first collaborative knowledge editing framework that aligns both local editing and the server-side merging rule under a single null-space principle for preserving existing knowledge.
- FedAlphaEdit builds on null-space-aligned merging, in which clients share projected statistics and the server provably recovers the result of editing everything in one place under a one-shot idealization.
Sources (1)
- [1]FedAlphaEdit: Null-Space-Aligned Merging for Collaborative Knowledge EditingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 12:25 AM
Null-space-constrained editing methods such as AlphaEdit mathematically guarantee that each update leaves unrelated knowledge intact, while collaborative frameworks such as CollabEdit aggregate edits from multiple clients without data sharing.
Multiple institutions may each hold their own private knowledge edits and wish to integrate them into a single large language model without sharing raw edit requests.
Extractive summary: sentences quoted from the sources.