Turning TS2Vec into a Foundation Model for Time Series Classification

Key points

• Yet TS2Vec offers a compact convolutional architecture that operates directly on temporal samples and accepts variable-length inputs without resampling.1
• We ask whether these properties can be retained when the encoder is pre-trained only once, and show that they can: TS2Vec can essentially be turned into a foundation model.1
• The result, TS2Vec-FM, is a 1.6M-parameter dilated convolutional network that, kept frozen, reaches 83.3% on UCR-128 and 75.3% on UEA-27 under linear probing, within 0.3 pp of MantisV2, the current state of the art, and above TS2Vec trained separately on each target dataset.1
• What we find most interesting is how different TS2Vec-FM is from other foundation models in design: near-parity is reached without attention, without a patch tokenizer, and without intermediate-layer extraction, using an "old-school" convolutional encoder and a two-layer MLP stem.1

Sources (1)

• [1]Turning TS2Vec into a Foundation Model for Time Series Classification
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:47 PM

  TS2Vec is one of the strongest self-supervised methods for time series classification, introduced in 2022.

  Yet TS2Vec offers a compact convolutional architecture that operates directly on temporal samples and accepts variable-length inputs without resampling.

Extractive summary: sentences quoted from the sources.