Text-to-Image Generative Modeling Without Language Pretraining

Key points

• We ask whether language pretraining is necessary for strong T2I generation.1
• We introduce Halcyon, a family of early-fusion multimodal autoregressive models pretrained entirely from scratch on GPIC, a dataset of 100M image-text pairs.1
• Halcyon also learns image captioning by replacing caption dropout with I2T supervision, maintaining competitive T2I generation quality under the same pretraining budget.1
• We show that strong T2I systems can be trained from scratch without internet-scale language pretraining.1

Sources (2)

• [1]Text-to-Image Generative Modeling Without Language Pretraining
  Hugging Face Daily Papers · Oct 19, 12:00 AM

  Modern text-to-image (T2I) systems use internet-scale language pretraining for text-conditioned image generation.

  We ask whether language pretraining is necessary for strong T2I generation.

• [2]Text-to-Image Generative Modeling Without Language Pretraining
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 05:59 PM · same content

Extractive summary: sentences quoted from the sources.