I noticed on the rules that the use of other datasets is prohibited and using a pre-trained model does imply leveraging knowledge gained from another dataset, as the pre-trained model's weights are learned from a different dataset. So does that mean pre-trained models are prohibited by extension.
From the rules (datasets section): You may use pretrained models as long as they are openly available to everyone.