MicroCLIP: Connecting Text and Microscopy Images via Contrastive Language--Image Pretraining
In this work, we introduce MicroCLIP, a vision-language model that learns generalizable cellular and subcellular representations from natural language supervision for open-set image classification and retrieval.
ProofPaper ↗
Key points
- Understanding fluorescence microscopy images is fundamental to automating biological research.
- To pretrain MicroCLIP, we construct MicroPool-2M, a dataset of approximately 2.5 million fluorescence microscopy image-text pairs spanning diverse cellular structures, cell lines, and imaging conditions.
- Aligning microscopy images with biological concepts in natural language is challenging because most images lack descriptive captions and biological knowledge annotations.
- Extensive experiments demonstrate that MicroCLIP achieves leading performance in zero-shot classification and text-image retrieval on microscopy images.
Sources (1)
- [1]MicroCLIP: Connecting Text and Microscopy Images via Contrastive Language--Image PretrainingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 07:26 AM
In this work, we introduce MicroCLIP, a vision-language model that learns generalizable cellular and subcellular representations from natural language supervision for open-set image classification and retrieval.
Understanding fluorescence microscopy images is fundamental to automating biological research.
Extractive summary: sentences quoted from the sources.