Transparency of private leaderboard

#1

Do you believe the private leaderboard? I would like to get your opinions.

First question

Top 100 participants ranking on the Public leaderboard from the Feedback phase will have access to new the Check phase environment

Rank #1 team, Hzk, is not in the public leaderboard. All team in the check and verification phase should be ranked in top 100 of the public leaderboard??

Second question

The scores of private leaderboard has to be shown because the scores has already been calculated, right?? Why are the scores hidden?

Third question

Do you think “Was my code run correctly??” Because our codes were run by the organizers manually.
It is easy to make a mistake…

Forth question

Should the organizer open the test environment and our codes before KDD conference for transparency? If the verification was not performed correctly, it should be re-run.

Fifth question

There should be participants who want to check the codes of other teams. I’d like to check the code, such as the copy of the environment, the multi-agent algorithm which uses the multiple environment, and so on. There are many ways to cheat the rules… Can we check it?

4 Likes

KDD 2019 | Policy Learning for Malaria Elimination
#2
ID Name Score
471 Hzk 27.51338676
470 RL-Hacker 26.57160963
232 Alpha 25.58103156
367 vlad 24.79111349
218 et373 24.55199456
226 Plato 21.68589721
317 LOLs 18.36882472
216 DoPlus 9.755636278
214 El 8.986782034
438 Anand 8.800777597
383 Pidgey 8.370028016
274 Winter_Is_Coming 7.455849098
376 ZCYM 6.417596042
506 JINGJINGXIAO 5.914394111
421 MathCo 5.870184194
396 NCKU_shallow_learning 5.624212873
348 Bonum 5.592861217
375 NULL 4.499057396
496 it_bites 3.520599066
319 mr_sandman 3.438478984
284 xinxinxinxinxin 1.950938848
288 ka2yama 1.751666869
417 Ironball 1.708535288
363 gnemoto 1.541115892
360 All_you_need_is_deep_learning 1.333807658
491 rohansaphal 0.619751826
386 A2 0.185512041
442 fightraccoon -0.323796657
213 NTT_DOCOMO -1.215853829
394 Ashish -2.54799673
402 saite_fan -
493 ENORMOUS_HAMMER_wang -
0 Likes

#3

@ikki407 I hope most of your questions answered in the other forum but would just like to add that I strongly encourage the sharing of solutions from teams via Github. Thanks for your questions and engagement.

1 Like

#4

@oetbent Thank you for your hard work!!

1 Like