Abstract: Image encoder pre-training has experienced a substantial evolution due to contrastive learning, facilitating the extraction of intricate feature representations from unlabeled datasets.