Hi everyone, I hope you are doing well. the challenge's description says : The evaluation metric for this challenge is the F1 score. Any clue which one is it ? macro-F1 or weighted-F1 ? Thanks.
Weighted-F1
Thanks @CodeJoe, can you send the source please ? i can't find it in the challenge description.
I deciphered it from the scores, Macro scores were way less (~80) whereby weighted looks very similar to the LB.
I see, i'll investigate it later. Thanks for your help 🙏
after further investigation, @CodeJoe was actually right, the challeng is using weighted-F1.
i've tested it using this code snippet :
import math
def p_from_score_macro(S): # S = (1/3) * 2p/(1+p) -> p = 3S/(2-3S) return (3*S) / (2 - 3*S)
def p_from_score_weighted(S): # S = 2p^2/(1+p) -> 2p^2 - S p - S = 0 return (S + math.sqrt(S*S + 8*S)) / 4 def p_from_score_micro(S): # micro-F1 == accuracy return S
# S_low, S_med, S_high from the LB: S_low, S_med, S_high = 0.0, 0.0, 0.0 # fill these
for name, f in [ ("macro", p_from_score_macro), ("weighted", p_from_score_weighted), ("micro", p_from_score_micro),]:
ps = [f(S_low), f(S_med), f(S_high)] print(name, ps, "sum =", sum(ps))
Weighted-F1
Thanks @CodeJoe, can you send the source please ? i can't find it in the challenge description.
I deciphered it from the scores, Macro scores were way less (~80) whereby weighted looks very similar to the LB.
I see, i'll investigate it later. Thanks for your help 🙏
after further investigation, @CodeJoe was actually right, the challeng is using weighted-F1.
i've tested it using this code snippet :
def p_from_score_macro(S): # S = (1/3) * 2p/(1+p) -> p = 3S/(2-3S) return (3*S) / (2 - 3*S)def p_from_score_weighted(S): # S = 2p^2/(1+p) -> 2p^2 - S p - S = 0 return (S + math.sqrt(S*S + 8*S)) / 4 def p_from_score_micro(S): # micro-F1 == accuracy return Sfor name, f in [ ("macro", p_from_score_macro), ("weighted", p_from_score_weighted), ("micro", p_from_score_micro),]:ps = [f(S_low), f(S_med), f(S_high)] print(name, ps, "sum =", sum(ps))