Making Tensor Decoupling More Robust and Less Parametric

Key points

• We identify two limitations of recent methods.1
• First, their performance is not invariant to mismatched output or gradient magnitudes, and second, their spline internals require manual tuning of their degrees of freedom.1
• The first is a preprocessing step that rescales the input Jacobian tensor, removing both the bias towards larger magnitude outputs and the ill-conditioning of certain factor updates caused by such a mismatch.1
• Finally, we release an efficient open-source JAX implementation of our algorithm.1

Sources (1)

• [1]Making Tensor Decoupling More Robust and Less Parametric
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 03:15 PM

  Tensor decoupling algorithms recover compact representations of multivariate functions by combining linear factor matrices with univariate internal nonlinearities.

  We identify two limitations of recent methods.

Extractive summary: sentences quoted from the sources.