I thought the low train AUC was due to the underfitting, but after some experiments I found that it is not as thought.
The low train AUC was caused by the difference between the negative examples we used for training and the ones for evaluating AUC.
the negative examples we used for training is not the same as the negative examples we used for evaluating train auc.
here is the results
max_sampled | 5 | 9 | 10 | 11 | 20 | 25 | 30 |
train auc | 0.583485 | 0.582856 | 0.586314 | 0.598418 | 0.596159 | 0.608818 | 0.599637 |
test_auc | 0.315679 | 0.310560 | 0.319777 | 0.313288 | 0.321055 | 0.287646 | 0.294142 |
so the MAX_SAMPLED is not a major cause here
I am thinking it may not be good enough in auc score but probably we can turn to pre@k?