FedGraph: Defending Federated Large Language Model Fine-Tuning Against Backdoor Attacks via Graph-Based Aggregation
Federated fine-tuning of large language models (LLMs) allows multiple clients to collaboratively adapt models without sharing raw data, offering strong privacy benefits.
ProofPaper ↗
Key points
- In this work, we make a key observation specific to federated LLM fine-tuning: malicious clients tend to exhibit strong consistency in their model updates, whereas benign clients produce more diverse and dispersed updates.
- Motivated by this observation, we propose FedGraph, a graph-based aggregation framework that explicitly models relationships among client updates.
- By representing clients as nodes in a dynamically constructed graph and extracting structural features that characterize their relative positions and interactions, FedGraph derives compact client representations that effectively separate malicious and benign behaviors.
- Extensive experiments demonstrate that FedGraph significantly improves backdoor defense.
Sources (1)
- [1]FedGraph: Defending Federated Large Language Model Fine-Tuning Against Backdoor Attacks via Graph-Based AggregationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 02:08 PM
Federated fine-tuning of large language models (LLMs) allows multiple clients to collaboratively adapt models without sharing raw data, offering strong privacy benefits.
In this work, we make a key observation specific to federated LLM fine-tuning: malicious clients tend to exhibit strong consistency in their model updates, whereas benign clients produce more diverse and dispersed updates.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs
- Oct 13, 2026SpecCloak: Probing Harmful Code Generation through Engineering Specifications
- Oct 12, 2026Density-Matrix Reinforcement Learning
- Oct 12, 2026Revisiting Natural Language Prompts from Large Language Models for Open-Vocabulary Video Recognition
- Oct 12, 2026Rethinking LLM-Guided Visual Question Answering: Does Deeper Knowledge Integration Really Help?
- Oct 12, 2026Spectral-Driven Adaptive-Rank Optimizer for Large Language Model Reinforcement Learning with Verifiable Rewards