Interfaces and Modules: A Scalable Representation for Weight-Space Learning across Architectures
Weight-space learning across architectures requires representations that respect parameter symmetries and scale to large networks.
ProofPaper ↗
Key points
- Existing equivariant tensor encoders require architecture-specific symmetry specifications, while neuron graphs make global attention costly.
- We introduce a typed graph organized around shared coordinate-permutation actions.
- Its nodes, called interfaces, represent coordinate spaces whose bound tensor axes must be permuted jointly; operation occurrences, called modules, form typed hyperedges.
- The Universal Weight Encoder (UWE) combines local equivariant updates to parameter features with global attention over invariant interface summaries.
Sources (1)
- [1]Interfaces and Modules: A Scalable Representation for Weight-Space Learning across ArchitecturesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 02:40 PM
Weight-space learning across architectures requires representations that respect parameter symmetries and scale to large networks.
Existing equivariant tensor encoders require architecture-specific symmetry specifications, while neuron graphs make global attention costly.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026tencent/Hy-Memory-30B-A3B
- Oct 15, 2026Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio
- Oct 15, 2026Welcome FLUX.3 Image Klein in Diffusers
- Oct 14, 2026mistralai/Timestral-209M
- Oct 14, 2026Train LoRA directly on Quantized GGUFs, no PyTorch, no Parent(FP16) needed, just your GGUF and the data
- Oct 13, 2026Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules