Rarity of Singletons Matters & Supplementary Materials

Key points

• Estimating missing-mass and unseen-species counts from finite samples relies heavily on rare-event occupancy statistics, particularly singletons.1
• Using singleton-based fragility measures derived from probability mass and occupancy counts, we formulate an entropy--Kullback--Leibler (KL) optimization that adjusts the reliance on base estimates according to their fragility.1
• Across 360 experimental configurations spanning synthetic distributions and real-world language corpora, our method remains competitive in stable regimes while improving estimation accuracy and stability in fragmented, long-tailed, and small-sample settings.1
• The findings demonstrate that accounting for the fragility of rare observations can improve the reliability of inference about unobserved species and their associated probability mass.1

Sources (1)

• [1]Rarity of Singletons Matters & Supplementary Materials
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 07:57 PM

  We propose a fragility-aware (FA) framework that explicitly quantifies this sensitivity and adaptively regularizes conventional missing-mass and unseen-species estimators.

  Estimating missing-mass and unseen-species counts from finite samples relies heavily on rare-event occupancy statistics, particularly singletons.

Extractive summary: sentences quoted from the sources.