Abstract: Image encoder pre-training has experienced a substantial evolution due to contrastive learning, facilitating the extraction of intricate feature representations from unlabeled datasets.

Penguin-VL is a compact vision-language model family built to study how far multimodal efficiency can be pushed by redesigning the vision encoder, rather than only scaling data or model size.