INCONVENIENT: A Benchmark to Assess LLM Generalization Capabilities from Convenience Samples
We introduce INCONVENIENT, a benchmark designed to study robustness to convenience-sample bias.
ProofPaper ↗
Key points
- Machine learning models for tabular prediction typically assume that training data are representative of the population on which they will be deployed.
- Existing tabular distribution-shift benchmarks primarily construct out-of-distribution (OOD) settings by partitioning observations across naturally occurring domains.
- Rather than relying on naturally occurring domain partitions, INCONVENIENT directly perturbs feature--target relationships in the training distribution while preserving the original population distribution as the deployment environment.
- Unlike existing distribution-shift benchmarks, INCONVENIENT provides explicit control over both the direction and severity of relationship shifts, enabling a systematic assessment of robustness to convenience sampling.
Sources (1)
- [1]INCONVENIENT: A Benchmark to Assess LLM Generalization Capabilities from Convenience SamplesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 12:15 AM
We introduce INCONVENIENT, a benchmark designed to study robustness to convenience-sample bias.
Machine learning models for tabular prediction typically assume that training data are representative of the population on which they will be deployed.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries
- Oct 14, 2026Optimal Hydra: Model Heterogeneous Federated Learning for LLMs using Data-Informed Hydra-Nets
- Oct 14, 2026Who's Behind the Text? Robust Model Authorship Attribution
- Oct 14, 2026Beyond Corrected Actions: Training LLM Agents on Textual Critiques
- Oct 14, 2026Sparse Autoencoders Are a Shared Coordinate System for Symbolic Matching
- Oct 13, 2026Plan or Retrieve First? A Scale Study of Tool Discovery Architectures
