When Parameter Gradient Flows Close in Representation Space

Key points

• We study when one entire represented mobility map works for every autonomous loss from one fixed initialization.1
• For two-factor matrix representations wider than both output dimensions, with individually full-row-rank factors, we classify all admissible joint ranks: closure occurs precisely at effective scalar-balanced minimal joint rank or a two-value signed spectral family at full joint rank.1
• For general representations with constant ambient representation and gradient-Lie ranks, ordinary closure is equivalent to a local invariant-level metric in the analytic case and specified smooth regimes.1
• We also separate testing families: linear losses locally admit closure at every submersion, whereas one fixed translated strongly convex quadratic family fails it on a delayed-bracket example.1

Sources (1)

• [1]When Parameter Gradient Flows Close in Representation Space
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 01:43 AM

  The represented gradient flow of a parameterized model depends on a mobility that can retain hidden parameter information.

  We study when one entire represented mobility map works for every autonomous loss from one fixed initialization.

Extractive summary: sentences quoted from the sources.