Hyperparameter ranges for all considered baseline models

Algorithms topK shrink Similarity Normalize α β ℓ2 norm
kNN algorithms [5 to 1000] [0 to 1000] [cosine, jaccard, asymmetric, dice, tversky] [True, False] — — —
Graph based algorithms [5 to 1000] — — — [0 to 2], strategy: uniform distribution [0 to 2], strategy: uniform distribution —
EASER — — — — — — [1 to 1e7], strategy: log-uniform distribution

Optimal hyperparameter values

Algorithms topK shrink Similarity Normalize α β ℓ2 norm
ItemkNN 805 544 cosine True — — —
UserkNN 1000 1000 cosine True — — —
P3α 117 — — False 0.4490914092362502 — —
RP3β 479 — — True 0.41257885094571617 0.6050046063241745 —
EASER — — — — — — 210.9260335725507

Statistics of the provided train-test splits

Users Items Interactions Density Data leakage Interactions per usermin Interactions per itemmin Interactions per usermax Interactions per itemmax Testing users
943 1,682 100,000 0.06305 No 20 1 737 583 943

Full result table

Algorithm R@5 R@10 Cov@5 Cov@10 Nov@5 Nov@10 F1@5 F1@10 T-Time (s) P-Time (s) Average P-Time Per User (ms)
Random 0.005 0.008 0.932 0.996 0.028 0.055 1.036 1.215 0 1.062 1.126
TopPop 0.052 0.087 0.025 0.040 0.023 0.046 0.049 0.092 0.000 1.131 1.200
ItemkNN 0.090 0.141 0.057 0.080 0.024 0.048 0.085 0.139 2.100 1.375 1.459
UserkNN 0.073 0.110 0.036 0.055 0.023 0.047 0.061 0.110 1.632 1.412 1.497
P3α 0.105 0.162 0.104 0.153 0.024 0.049 0.136 0.224 0.547 1.211 1.284
RP3β 0.106 0.174 0.160 0.220 0.025 0.050 0.197 0.303 2.116 1.280 1.357
EASE 0.119 0.185 0.179 0.250 0.025 0.050 0.217 0.338 0.516 1.270 1.346
NCF 0.084 0.138 0.390 0.539 0.030 0.060 0.454 0.696 772.322 9.697 10.283
LFM 0.080 0.138 0.367 0.486 0.030 0.062 0.431 0.637 533.672 10.416 11.046

Optimal hyperparameter values

Algorithms topK shrink Similarity Normalize α β ℓ2 norm
ItemkNN 623 193 cosine True — — —
UserkNN 149 0 cosine True — — —
P3α 85 — — True 0.4590018257740354 — —
RP3β 363 — — False 0.45815768400879797 0.39431553250158924 —
EASER — — — — — — 47.11431042076403

Statistics of the provided train-test splits

Users Items Interactions Density Data leakage Interactions per usermin Interactions per itemmin Interactions per usermax Interactions per itemmax Testing users
5,541 3,568 66,809 0.00338 No 5 5 578 272 5,541

Full result table

Algorithm R@5 R@10 Cov@5 Cov@10 Novelty@5 Nov@10 F1@5 F1@10 T-Time (s) P-Time (s) Average P-Time Per User (ms)
Random 0.001 0.002 1 1 0.016 0.033 1.066 1.132 0 5.936 1.071
TopPop 0.020 0.033 0.004 0.007 0.011 0.023 0.015 0.031 0.001 5.888 1.063
ItemkNN 0.153 0.211 0.831 0.918 0.016 0.031 0.885 1.035 2.451 6.545 1.181
UserkNN 0.146 0.201 0.468 0.638 0.014 0.028 0.501 0.721 2.101 6.375 1.150
P3α 0.155 0.212 0.571 0.724 0.014 0.029 0.610 0.816 1.018 6.189 1.117
RP3β 0.160 0.219 0.782 0.897 0.015 0.030 0.832 1.007 2.781 6.346 1.145
EASER 0.158 0.214 0.553 0.721 0.014 0.029 0.591 0.812 1.455 6.416 1.158
NCF 0.067 0.112 0.800 0.985 0.017 0.033 0.856 1.117 266.793 75.829 13.685
LFM 0.061 0.106 0.392 0.524 0.014 0.029 0.423 0.599 251.867 71.837 12.965

Optimal hyperparameter values

Algorithms topK shrink Similarity Normalize α β ℓ2 norm
ItemkNN 1000 1000 cosine False — — —
UserkNN 255 1000 cosine False — — —
P3α 431 — — False 0.5496778369228872 — —
RP3β 1000 — — False 0.5472822005926639 0.15218913649100715 —
EASER — — — — — — 105.27021898818253

Statistics of the provided train-test splits

Users Items Interactions Density Data leakage Interactions per usermin Interactions per itemmin Interactions per usermax Interactions per itemmax Testing users
8,159 5,862 101,599 0.00212 No 1 1 189 216 8,159

Full result table

Algorithm R@5 R@10 Cov@5 Cov@10 Novelty@5 Nov@10 F1@5 F1@10 T-Time (s) P-Time (s) Average P-Time Per User (ms)
Random 0.000 0.001 0.999 1 0.009 0.018 1.036 1.074 0 9.230 1.131
TopPop 0.006 0.010 0.002 0.003 0.007 0.015 0.009 0.018 0.002 9.080 0.001
ItemkNN 0.041 0.058 0.350 0.512 0.008 0.017 0.368 0.555 2.355 9.359 1.147
UserkNN 0.042 0.062 0.447 0.628 0.009 0.018 0.467 0.679 3.998 9.315 1.142
P3α 0.043 0.062 0.481 0.658 0.009 0.018 0.502 0.711 2.602 9.358 1.147
RP3β 0.045 0.064 0.656 0.785 0.009 0.018 0.683 0.847 2.747 9.342 1.145
EASER 0.050 0.067 0.456 0.639 0.009 0.018 0.477 0.691 3.589 9.498 1.164
NCF 0.023 0.037 0.581 0.770 0.010 0.021 0.609 0.840 430.310 180.238 22.091
LFM 0.019 0.032 0.255 0.403 0.011 0.022 0.274 0.453 529.520 163.878 20.086