I know it's late already but are we supposed to label the data ourselves, as I can't see any labels in the CSV files
No, the train set is already provided in the data section, and their respective labels.
.......and you're not late at all. If you need more help getting started, check @Siwar_NASRI's comment here: https://zindi.africa/competitions/makerere-fall-armyworm-crop-challenge/discussions/11699
Oh thank you so much for taking the time to respond, I really appreciate. I get it now, but then how come about the test set doesn't get labels(at least in regression problems they do get)?
Nah, the test set is the set you'll make predictions on and submit, they are never given to participants.