🌍 Join the Buzz: 3rd place solution

INEGI UN-GGIM Human Settlement Detection Challenge by ITU

Helping Mexico

1 000 CHF

Challenge completed ~1 year ago

Skills you will learn

Classification

201 joined

84 active

Info Data Chat Leaderboard

Start

Sep 02, 24

Sep 28, 24

Reveal

Oct 10, 24

nymfree

3rd place solution

Notebooks · 10 Oct 2024, 09:51 · 4

- TLDR; An ensemble of 1D CNN models

Details

- Seeing that 2D CNN approaches were limited as it was difficult for a human to detect whether the class of an image was `1` or `0`, I decided to switch to a 1D CNN approach.

- A key to success here is normalization of the input data. I found the recommended band specific normalization factors to be underperformant and opted to normalize all channels by 4500 (didn't try to many things here - though I think that better normalization could have led to an even better result).

- Used lots of batchnorm and large dropout to stabilize training.

- Kernel size of the CNN layers was an important parameter.

- Trained on all 6 channels.

The final result was an ensemble of 1d CNN models with different kernel sizes. The model itself is

# Dataset
class CustomDataset(torch.utils.data.Dataset):
    def __init__(self, images, labels, test=TEST):
        self.images = images
        self.labels = labels
        self.test = test
    def __len__(self):
        return len(self.images) 
    def __getitem__(self, index):
        image = self.images[index]
        image = image.flatten()
        image = image / 4500.0
        image = torch.tensor(image).float()
        if self.test:
            return image
        label = self.labels[index]
        label = torch.tensor(label, dtype=torch.float)
        return image, torch.unsqueeze(label, 0)
        
# Model
class CNNModel(nn.Module):
    def __init__(self):
        # Define the CNN layers
        self.conv1 = nn.Conv1d(in_channels=1, out_channels=32, kernel_size=3, padding=1)
        self.bn1 = nn.BatchNorm1d(32)
        self.conv2 = nn.Conv1d(in_channels=32, out_channels=64, kernel_size=3, padding=1)
        self.bn2 = nn.BatchNorm1d(64)
        self.conv3 = nn.Conv1d(in_channels=64, out_channels=128, kernel_size=3, padding=1)
        self.bn3 = nn.BatchNorm1d(128)
        self.conv4 = nn.Conv1d(in_channels=128, out_channels=256, kernel_size=3, padding=1)
        self.bn4 = nn.BatchNorm1d(256)
        self.conv5 = nn.Conv1d(in_channels=256, out_channels=128, kernel_size=3, padding=1)
        self.bn5 = nn.BatchNorm1d(128)
        self.pool = nn.MaxPool1d(kernel_size=2)
        self.gap = nn.AdaptiveAvgPool1d(1)
        self.fc1 = nn.Linear(128, 64)
        self.fc2 = nn.Linear(64, 1)
        self.dropout = nn.Dropout(p=0.6)
        
   def forward(self, x):
       x = x.view(x.size(0), 1, -1)
       x = F.relu(self.bn1(self.conv1(x)))
       x = self.pool(F.relu(self.bn2(self.conv2(x))))
       x = self.pool(F.relu(self.bn3(self.conv3(x))))
       x = self.pool(F.relu(self.bn4(self.conv4(x))))
       x = self.pool(F.relu(self.bn5(self.conv5(x))))
                  
       x = self.gap(x)
       x = x.view(x.size(0), -1)
       x = self.dropout(x)
       
       x = F.relu(self.fc1(x))
       x = torch.sigmoid(self.fc2(x))
       return x

Discussion 4 answers

Nayal_17

@nymfree congo on your win. Can you explain why you choose normalization factor 4500, is it through experiments or there is any particular reason for that.

10 Oct 2024, 09:56

Upvotes 0

nymfree

Thanks. For satellite imaging, spectral bands have different normalization factors. If I remember correctly, it is 3000, 2500 and 2500 for blue, green and red respectively. And some other values for the other channels.

I found channel specific normalization not to perform better than a single normalization factor for all. experimented with 3000, 3500, 4000, 4500 and 5000. 4500 gave better CV.

replied to Nayal_1710 Oct 2024, 10:13

Upvotes 3

robson_dsp

Congratulations on your performance, colleague. How did you solve the class imbalance problem? I created several balanced datasets with the same number of class 1 and the same number of class 0. But in each dataset, I used different samples from class 0. I did an ensemble of 2D convolutional neural networks and obtained an AUC of 0.93 on my test set, which corresponds to 25% of the training set. When I submitted my official score, it was 0.504. My other attempt was to leave the dataset imbalanced and use focal loss along with ResNet50. I achieved an AUC of 0.95 on my test set and an AUC of 0.501 on the leaderboard. I don’t know what I did so wrong for the models to degrade so much on the official test set. Below is a bit of the code with ResNet50.

def focal_loss(gamma=2., alpha=.25):

    def focal_loss_fixed(y_true, y_pred):

        epsilon = K.epsilon()

        y_pred = K.clip(y_pred, epsilon, 1. - epsilon)

        pt_1 = tf.where(tf.equal(y_true, 1), y_pred, tf.ones_like(y_pred))

        pt_0 = tf.where(tf.equal(y_true, 0), y_pred, tf.zeros_like(y_pred))

        loss_1 = -alpha * K.pow(1. - pt_1, gamma) * K.log(pt_1)

        loss_0 = -(1 - alpha) * K.pow(pt_0, gamma) * K.log(1. - pt_0)

        return K.mean(loss_1 + loss_0)

    return focal_loss_fixed

with tpu_strategy.scope():

    base_model = keras.applications.resnet50.ResNet50(weights="imagenet", include_top=False)

    avg    = tf.keras.layers.GlobalAveragePooling2D()(base_model.output)

    output = tf.keras.layers.Dense(1, activation="sigmoid")(avg)

    model  = keras.models.Model(inputs=base_model.input, outputs=output)

    for layer in base_model.layers:

        layer.trainable = False

    model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.01),

                  loss=focal_loss(alpha=0.25, gamma=2.0),

                  metrics=[tf.keras.metrics.AUC(name='auc'),

                           tf.keras.metrics.Precision(name='precision'),

                           tf.keras.metrics.Recall(name='recall'), ])

history = model.fit(train_set, validation_data=val_set,

                    steps_per_epoch=int(0.75 * train_size / batch_size),

                    validation_steps=int(0.15 * train_size / batch_size),

                    epochs=epochs, batch_size=batch_size,

                    callbacks=callbacks, verbose=True)

12 Oct 2024, 00:44

Upvotes 0

nymfree

I didn't do anything special regarding class imbalance. I trained on the whole dataset using 5 folds. The folds were stratified such that they contained about the same proportion of 1s and 0s.

2D convs were not successfull in this competition, in my opinion. The other reason you could have been getting high AUC and LB equivalent to random guessing might be because you didn't use the id_map.csv file to properly sort your predictions. There is some other discussion on the forum about this.

replied to robson_dsp12 Oct 2024, 06:49

Upvotes 0

Join the largest network for
data scientists and AI builders

About FAQs

Status