The training of deep learning models generally requires a large amount of annotated data for effective convergence and generalisation. However, obtaining high-quality annotations is a laboursome and expensive process due to the need of expert radiologists for the labelling task. The study of semi-supervised learning in medical image analysis is then of crucial importance given that it is much less expensive to obtain unlabelled images than to acquire images labelled by expert radiologists. Essentially, semi-supervised methods leverage large sets of unlabelled data to enable better training convergence and generalisation than using only the small set of labelled images. In this paper, we propose Self-supervised Mean Teacher for Semi-supervised (S$^2$MTS$^2$) learning that combines self-supervised mean-teacher pre-training with semi-supervised fine-tuning. The main innovation of S$^2$MTS$^2$ is the self-supervised mean-teacher pre-training based on the joint contrastive learning, which uses an infinite number of pairs of positive query and key features to improve the mean-teacher representation. The model is then fine-tuned using the exponential moving average teacher framework trained with semi-supervised learning. We validate S$^2$MTS$^2$ on the multi-label classification problems from Chest X-ray14 and CheXpert, and the multi-class classification from ISIC2018, where we show that it outperforms the previous SOTA semi-supervised learning methods by a large margin.

翻译：深层次学习模型的培训一般需要大量附加说明的数据,以便有效趋同和概括。然而,获得高质量的说明是一个艰巨而昂贵的过程,因为需要专业放射学家进行贴标签工作。研究医学图像分析中的半监督学习具有关键重要性,因为获得无标签图像比获得专家放射学家贴标签的图像要便宜得多。基本上,半监督的方法利用大量未贴标签的数据来进行更好的培训趋同和概括。在本文件中,我们提议为半监督的半监督的中层教师学习(S+2$MTS=2美元),将自我监督的中层教师前培训与半监督的微调相结合。S+14$MTS=2的主要创新是基于联合对比学习的自我监督的中层教师前培训。我们建议为半监督的中层教师学习提供了无限数量的正面查询和关键特征,以便改进中层教师的中层学习。我们经过培训的S-BS-BS-S-S-S-S-IL-S-S-IL-IL-S-IL-S-IL-S-S-IS-IL-IL-IL-IL-IL-I-I-I-I-I-I-I-I-I-I-I-IL-IL-IL-IL-I-I-I-I-I-I-I-I-I-I-I-I-I-I-IL-I-I-I-I-IL-IL-IL-IL-I-I-I-IL-IL-IL-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-