My best score is train.mean() *2.1 , 75 private score or something which can take me to 16th or 17th position ..ofcourse I didn't select it ..would have been completely pointless .

Testing with 90 records was a good idea, did you have a good correlations with CV and Private ? Any specific distribution you selected this 90 values ?

Because if you chose the 90 images for testing , how did you ensure they share the same distributions as the 90 images in the submission file?

Yes I did but some were overfitted. I chose them randomly, I didn't spend much time on this but it might have helped when selecting a final submission.

Interesting ... but too few degrees of freedom, can't draw reliable inference from that observation alone :-)

Clever

Having access to some of the agbd data from ground truth seems necessary to build good model.

So you could be #3 by submitting 105 ... perhaps there are big outliers and top guys managed to get close to them, and 105 is good average of outliers. Perhaps top guys just got close to 105, then this is just random.

We were very careful not to overfit, our public LB was not at all what we selected, but still we slid down quite a bit. Our best private LB sub, which we also selected, has average 112 fwiw (EDIT that sub has public 48 and private 79 fwiw)

Based on another discussion by @MohamedDHIAB I think he just subbed constant to be #10, so he must be slightly off 105 ...

How did you get that impressive score what approach did you use ?

@Koleshjr Nothing too serious here. The major lift in my case is the computation of various vegetation Indices:(using spyndex library); followed by aggregation of these indices.

Reference: https://github.com/awesome-spectral-indices/awesome-spectral-indices/blob/main/output/spectral-indices-table.csv

Train Dataset Splitting Technique:

Target (biomass) values based splitting method:

Here, my thorough obsevation shows that it's quite hard for any model to predict the required higher count of biomass in shaded regions in Cote d’Ivoire based on GEDI, Sentinel-2 and ground truth biomass data provided. Henceforth, the generated train dataset above were splitted into two parts as follows below, where:

• Only train target (biomass) values that are greater than 60 are selected for model training in part 1, and
• Only train target (biomass) values that are less than and equal to 60 are selected for model training in part 2 respectively.

And this resulted to two-stage modelling parts.

Features Selection Technique: Create a wrapper class that has all the built in statistical tests required to perform feature selection and takes some basic inputs from user and spits out the required features (using scoring="f_regression").

Model Trained: CatBoostRegressor with 5kolds CV (for each of the Model training in part 1 and part 2).

Final Stage: Averaging predictions coming from the two-stage modelling parts as a single final prediction.

(Lest i forget, i also assigned weight ratio 90/10 to part 1 and part 2 of the model training predictions respectively). Note: This method gave better score than "Averaging" above.

Nice, thanks for sharing @MICADEE

Just to clarify ... in feature selection, you basically use f_regression after fitting y=f(x) where x = one of the indices output (or aggregate output) and this is how you select which indices to use in the end?

Uwc @skaak.

Yes exactly.

Something like this below showing only part of the aforementioned:

self.n_features = n_features

if problem_type == "classification":

valid_scoring = {

"f_classif": f_classif,

"chi2": chi2,

"mutual_info_classif": mutual_info_classif

}

else:

valid_scoring = {

"f_regression": f_regression,

"mutual_info_regression": mutual_info_regression

}

if scoring not in valid_scoring:

raise Exception("Invalid scoring function")

Thanks again @MICADEE

This is fine for 1d. If I understand correctly, you test features one by one. Perhaps there are two, say x1 and x2, that do not work individually but together they work pretty well ie in 2d.

Not that I would do it differently tbh, but I've heard about stepwise regression, foreward selection and backward elimination to use for that. I only saw it implemented in SAS and never used it myself, but I wonder what those would select, and if it would be better, but, again tbh, I would not even bother myself. Those techniques are anyhow linear afaik and today's techniques are geared towards milking all out of the data.

fwiw I started this comp like that, calculating lots of bands (there was some nice public kernel to help with that) but after a while and no progress I switched to sort of global approach, just running all the data through a single model without really extracting features. It worked a bit better in CV and public LB, so that's what I ended with, but I wonder if those earlier models would not have done a bit better given the erratic LB.

Oh well, congrats on your performance here, I see you worked very hard.

Using average of two models should be robust but here it was not ...

BTW if metric was e.g. RMSE( log(actual) - log(predicted) ) do you think it would be better / less random?

Then the problem is not outliers, but the fact that variance increases as value (actual) increase and you need to transform that away, but that still won't give you better LB if LB uses RMSE metric. Then RMSE is wrong metric here ...