Masked Attention Pooling Is Invariant to Padding and Permutation: A Formal Proof in Lean
Attention-based multiple instance learning (MIL) pools a variable-size bag of instance embeddings into one vector.
ProofPaper ↗
Key points
- In batched training, bags are padded to a common length, and the pooled output is expected not to depend on how much padding was added or on the order in which instances are stored.
- Masking with a large negative constant is exact only after floating-point underflow.
- We define masked attention pooling with a guarded normalizer and prove in Lean 4 with mathlib that it is invariant under any bijective reindexing of rows and under appending any finite number of masked rows with arbitrary real features and logits.
- Using the jax-lean certificate pipeline, we connect these theorems to the Jaxpr traced from the JAX kernel itself.
Sources (1)
- [1]Masked Attention Pooling Is Invariant to Padding and Permutation: A Formal Proof in LeanarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 05:20 PM
Attention-based multiple instance learning (MIL) pools a variable-size bag of instance embeddings into one vector.
In batched training, bags are padded to a common length, and the pooled output is expected not to depend on how much padding was added or on the order in which instances are stored.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 12, 2026CLASSO: a practical primal path-following algorithm for the constrained LASSO
- Oct 12, 2026Making Tensor Decoupling More Robust and Less Parametric
- Oct 11, 2026ClinEdge: An Expert-Graded Benchmark of Real Patient Edge Cases Where Frontier Multimodal Models Fail
- Oct 10, 2026MARL-BattleGrounds: A Novel JAX-native Benchmark for Heterogeneous and Competitive Multi-agent RL
- Oct 10, 2026Push the Limit: Learning High-Speed Sprinting and Aggressive Cornering on a Full-Scale Humanoid
- Oct 10, 2026Counterfactual Regret for Reflective Agents with Lean-Verified Guarantees