Feature Extraction
Transformers
Safetensors
sentence-transformers
minicpm
mteb
custom_code
Eval Results (legacy)
Instructions to use openbmb/MiniCPM-Embedding-Light with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openbmb/MiniCPM-Embedding-Light with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="openbmb/MiniCPM-Embedding-Light", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("openbmb/MiniCPM-Embedding-Light", trust_remote_code=True, device_map="auto") - sentence-transformers
How to use openbmb/MiniCPM-Embedding-Light with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("openbmb/MiniCPM-Embedding-Light", trust_remote_code=True) sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
metadata
model-index:
- name: no_model_name_available
results:
- dataset:
config: default
name: MTEB ArguAna (default)
revision: c22ab2a51041ffd869aaddef7af8d8215647e41a
split: test
type: mteb/arguana
metrics:
- type: main_score
value: 55.24400000000001
- type: map_at_1
value: 30.014000000000003
- type: map_at_10
value: 45.943
- type: map_at_100
value: 46.763
- type: map_at_1000
value: 46.769
- type: map_at_20
value: 46.655
- type: map_at_3
value: 40.446
- type: map_at_5
value: 43.646
- type: mrr_at_1
value: 30.867709815078236
- type: mrr_at_10
value: 46.258184876605945
- type: mrr_at_100
value: 47.07814278884162
- type: mrr_at_1000
value: 47.08356304305195
- type: mrr_at_20
value: 46.966353170422785
- type: mrr_at_3
value: 40.82503556187766
- type: mrr_at_5
value: 43.96159317211951
- type: nauc_map_at_1000_diff1
value: 6.886880780895957
- type: nauc_map_at_1000_max
value: -11.375399079859044
- type: nauc_map_at_1000_std
value: -17.223274601671246
- type: nauc_map_at_100_diff1
value: 6.892991544199656
- type: nauc_map_at_100_max
value: -11.362002243225842
- type: nauc_map_at_100_std
value: -17.210174370727724
- type: nauc_map_at_10_diff1
value: 6.736414727787148
- type: nauc_map_at_10_max
value: -11.37831967023241
- type: nauc_map_at_10_std
value: -17.33404964799479
- type: nauc_map_at_1_diff1
value: 8.923783715191092
- type: nauc_map_at_1_max
value: -13.126385593275344
- type: nauc_map_at_1_std
value: -16.436273755848433
- type: nauc_map_at_20_diff1
value: 6.932185704189038
- type: nauc_map_at_20_max
value: -11.275898951773595
- type: nauc_map_at_20_std
value: -17.176054921422146
- type: nauc_map_at_3_diff1
value: 6.662076492295663
- type: nauc_map_at_3_max
value: -11.953142843760919
- type: nauc_map_at_3_std
value: -17.833788606042795
- type: nauc_map_at_5_diff1
value: 6.7271193830238305
- type: nauc_map_at_5_max
value: -11.246510557780741
- type: nauc_map_at_5_std
value: -17.91087506838237
- type: nauc_mrr_at_1000_diff1
value: 4.53297749205669
- type: nauc_mrr_at_1000_max
value: -12.39580136551332
- type: nauc_mrr_at_1000_std
value: -17.222685971057704
- type: nauc_mrr_at_100_diff1
value: 4.5394879919492
- type: nauc_mrr_at_100_max
value: -12.382262051447933
- type: nauc_mrr_at_100_std
value: -17.209623392767387
- type: nauc_mrr_at_10_diff1
value: 4.447864108553873
- type: nauc_mrr_at_10_max
value: -12.367015266918003
- type: nauc_mrr_at_10_std
value: -17.332132779063926
- type: nauc_mrr_at_1_diff1
value: 6.426933190532652
- type: nauc_mrr_at_1_max
value: -13.235478775299327
- type: nauc_mrr_at_1_std
value: -15.961449323758286
- type: nauc_mrr_at_20_diff1
value: 4.567557598785181
- type: nauc_mrr_at_20_max
value: -12.310223981001874
- type: nauc_mrr_at_20_std
value: -17.192843175662443
- type: nauc_mrr_at_3_diff1
value: 4.6716216832813995
- type: nauc_mrr_at_3_max
value: -12.817044392980431
- type: nauc_mrr_at_3_std
value: -17.571117325539486
- type: nauc_mrr_at_5_diff1
value: 4.519516858596568
- type: nauc_mrr_at_5_max
value: -12.229322555388574
- type: nauc_mrr_at_5_std
value: -17.942953586089775
- type: nauc_ndcg_at_1000_diff1
value: 6.817814873013132
- type: nauc_ndcg_at_1000_max
value: -10.873094930178674
- type: nauc_ndcg_at_1000_std
value: -17.085008322125926
- type: nauc_ndcg_at_100_diff1
value: 6.953133785546433
- type: nauc_ndcg_at_100_max
value: -10.531957616408311
- type: nauc_ndcg_at_100_std
value: -16.71818159673947
- type: nauc_ndcg_at_10_diff1
value: 6.295178227770737
- type: nauc_ndcg_at_10_max
value: -10.397265988000354
- type: nauc_ndcg_at_10_std
value: -17.198159100054138
- type: nauc_ndcg_at_1_diff1
value: 8.923783715191092
- type: nauc_ndcg_at_1_max
value: -13.126385593275344
- type: nauc_ndcg_at_1_std
value: -16.436273755848433
- type: nauc_ndcg_at_20_diff1
value: 7.2649927395618015
- type: nauc_ndcg_at_20_max
value: -9.899142517358202
- type: nauc_ndcg_at_20_std
value: -16.553478867005055
- type: nauc_ndcg_at_3_diff1
value: 6.292726274188458
- type: nauc_ndcg_at_3_max
value: -11.588413127917905
- type: nauc_ndcg_at_3_std
value: -18.390076696815374
- type: nauc_ndcg_at_5_diff1
value: 6.309839035230393
- type: nauc_ndcg_at_5_max
value: -10.271508945864587
- type: nauc_ndcg_at_5_std
value: -18.576733710870386
- type: nauc_precision_at_1000_diff1
value: 5.9271867403828296
- type: nauc_precision_at_1000_max
value: -6.072894615934814
- type: nauc_precision_at_1000_std
value: -16.135433674090358
- type: nauc_precision_at_100_diff1
value: 25.726925747145064
- type: nauc_precision_at_100_max
value: 40.584090685342254
- type: nauc_precision_at_100_std
value: 38.14370530299473
- type: nauc_precision_at_10_diff1
value: 3.6781677623945166
- type: nauc_precision_at_10_max
value: -3.7754580942378686
- type: nauc_precision_at_10_std
value: -15.457609944502806
- type: nauc_precision_at_1_diff1
value: 8.923783715191092
- type: nauc_precision_at_1_max
value: -13.126385593275344
- type: nauc_precision_at_1_std
value: -16.436273755848433
- type: nauc_precision_at_20_diff1
value: 19.746235929102436
- type: nauc_precision_at_20_max
value: 18.35495615357464
- type: nauc_precision_at_20_std
value: 0.15511059086479054
- type: nauc_precision_at_3_diff1
value: 5.3134343636124575
- type: nauc_precision_at_3_max
value: -10.512709496977122
- type: nauc_precision_at_3_std
value: -20.070916931616274
- type: nauc_precision_at_5_diff1
value: 4.921722409140018
- type: nauc_precision_at_5_max
value: -6.629368447070168
- type: nauc_precision_at_5_std
value: -20.999434723756796
- type: nauc_recall_at_1000_diff1
value: 5.927186740377048
- type: nauc_recall_at_1000_max
value: -6.0728946159395
- type: nauc_recall_at_1000_std
value: -16.135433674089697
- type: nauc_recall_at_100_diff1
value: 25.726925747144634
- type: nauc_recall_at_100_max
value: 40.58409068534266
- type: nauc_recall_at_100_std
value: 38.143705302995684
- type: nauc_recall_at_10_diff1
value: 3.67816776239454
- type: nauc_recall_at_10_max
value: -3.7754580942378464
- type: nauc_recall_at_10_std
value: -15.457609944502648
- type: nauc_recall_at_1_diff1
value: 8.923783715191092
- type: nauc_recall_at_1_max
value: -13.126385593275344
- type: nauc_recall_at_1_std
value: -16.436273755848433
- type: nauc_recall_at_20_diff1
value: 19.7462359291029
- type: nauc_recall_at_20_max
value: 18.35495615357484
- type: nauc_recall_at_20_std
value: 0.15511059086445783
- type: nauc_recall_at_3_diff1
value: 5.31343436361246
- type: nauc_recall_at_3_max
value: -10.512709496977116
- type: nauc_recall_at_3_std
value: -20.07091693161628
- type: nauc_recall_at_5_diff1
value: 4.92172240913999
- type: nauc_recall_at_5_max
value: -6.6293684470701635
- type: nauc_recall_at_5_std
value: -20.999434723756817
- type: ndcg_at_1
value: 30.014000000000003
- type: ndcg_at_10
value: 55.24400000000001
- type: ndcg_at_100
value: 58.5
- type: ndcg_at_1000
value: 58.616
- type: ndcg_at_20
value: 57.787
- type: ndcg_at_3
value: 43.934
- type: ndcg_at_5
value: 49.718
- type: precision_at_1
value: 30.014000000000003
- type: precision_at_10
value: 8.514
- type: precision_at_100
value: 0.988
- type: precision_at_1000
value: 0.1
- type: precision_at_20
value: 4.755
- type: precision_at_3
value: 18.018
- type: precision_at_5
value: 13.627
- type: recall_at_1
value: 30.014000000000003
- type: recall_at_10
value: 85.135
- type: recall_at_100
value: 98.791
- type: recall_at_1000
value: 99.644
- type: recall_at_20
value: 95.092
- type: recall_at_3
value: 54.054
- type: recall_at_5
value: 68.137
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackAndroidRetrieval (default)
revision: f46a197baaae43b4f621051089b82a364682dfeb
split: test
type: mteb/cqadupstack-android
metrics:
- type: main_score
value: 51.13999999999999
- type: map_at_1
value: 31.875999999999998
- type: map_at_10
value: 44.532
- type: map_at_100
value: 46.038000000000004
- type: map_at_1000
value: 46.157
- type: map_at_20
value: 45.366
- type: map_at_3
value: 40.839999999999996
- type: map_at_5
value: 43.114000000000004
- type: mrr_at_1
value: 38.91273247496424
- type: mrr_at_10
value: 50.152315098667025
- type: mrr_at_100
value: 50.874290542898336
- type: mrr_at_1000
value: 50.906905801947886
- type: mrr_at_20
value: 50.56376794173934
- type: mrr_at_3
value: 47.32951835956128
- type: mrr_at_5
value: 49.217930376728624
- type: nauc_map_at_1000_diff1
value: 52.36476606146739
- type: nauc_map_at_1000_max
value: 40.490237958252365
- type: nauc_map_at_1000_std
value: -1.8845317344415389
- type: nauc_map_at_100_diff1
value: 52.33703450857222
- type: nauc_map_at_100_max
value: 40.47486544690021
- type: nauc_map_at_100_std
value: -1.9095157213200196
- type: nauc_map_at_10_diff1
value: 52.48310495583547
- type: nauc_map_at_10_max
value: 40.09186714297923
- type: nauc_map_at_10_std
value: -2.4313790543381026
- type: nauc_map_at_1_diff1
value: 58.01365735208089
- type: nauc_map_at_1_max
value: 36.11029923363937
- type: nauc_map_at_1_std
value: -6.239781340568959
- type: nauc_map_at_20_diff1
value: 52.32501194633051
- type: nauc_map_at_20_max
value: 40.428051588019095
- type: nauc_map_at_20_std
value: -1.9264869828333264
- type: nauc_map_at_3_diff1
value: 53.0110474606367
- type: nauc_map_at_3_max
value: 39.68533032391943
- type: nauc_map_at_3_std
value: -3.6285465207403065
- type: nauc_map_at_5_diff1
value: 52.54981116327183
- type: nauc_map_at_5_max
value: 40.0906539376223
- type: nauc_map_at_5_std
value: -2.4919327393076585
- type: nauc_mrr_at_1000_diff1
value: 50.82302249679038
- type: nauc_mrr_at_1000_max
value: 39.9254848913621
- type: nauc_mrr_at_1000_std
value: -1.5984511554516092
- type: nauc_mrr_at_100_diff1
value: 50.80412980753116
- type: nauc_mrr_at_100_max
value: 39.91435056267046
- type: nauc_mrr_at_100_std
value: -1.6230096004155266
- type: nauc_mrr_at_10_diff1
value: 50.83275055092101
- type: nauc_mrr_at_10_max
value: 39.96103690324146
- type: nauc_mrr_at_10_std
value: -1.6041269189934009
- type: nauc_mrr_at_1_diff1
value: 54.88420521179768
- type: nauc_mrr_at_1_max
value: 38.655520078749
- type: nauc_mrr_at_1_std
value: -5.11895317994502
- type: nauc_mrr_at_20_diff1
value: 50.6713327014749
- type: nauc_mrr_at_20_max
value: 39.86580174795927
- type: nauc_mrr_at_20_std
value: -1.5092495858061845
- type: nauc_mrr_at_3_diff1
value: 51.453247777585034
- type: nauc_mrr_at_3_max
value: 40.44141531265311
- type: nauc_mrr_at_3_std
value: -2.1147720426347343
- type: nauc_mrr_at_5_diff1
value: 50.876949710774376
- type: nauc_mrr_at_5_max
value: 40.355459699136375
- type: nauc_mrr_at_5_std
value: -1.5067292573055617
- type: nauc_ndcg_at_1000_diff1
value: 50.68097915475126
- type: nauc_ndcg_at_1000_max
value: 40.83729728637769
- type: nauc_ndcg_at_1000_std
value: 0.28080722973815964
- type: nauc_ndcg_at_100_diff1
value: 50.12941283880048
- type: nauc_ndcg_at_100_max
value: 40.48586542448946
- type: nauc_ndcg_at_100_std
value: -0.009202303401808673
- type: nauc_ndcg_at_10_diff1
value: 50.41920422355683
- type: nauc_ndcg_at_10_max
value: 40.24351748132248
- type: nauc_ndcg_at_10_std
value: -0.7873431225714408
- type: nauc_ndcg_at_1_diff1
value: 54.88420521179768
- type: nauc_ndcg_at_1_max
value: 38.655520078749
- type: nauc_ndcg_at_1_std
value: -5.11895317994502
- type: nauc_ndcg_at_20_diff1
value: 49.86234903821364
- type: nauc_ndcg_at_20_max
value: 40.49553152163194
- type: nauc_ndcg_at_20_std
value: 0.09200925511478703
- type: nauc_ndcg_at_3_diff1
value: 51.223218754841625
- type: nauc_ndcg_at_3_max
value: 41.14642149091258
- type: nauc_ndcg_at_3_std
value: -1.8066493294127863
- type: nauc_ndcg_at_5_diff1
value: 50.39826734269378
- type: nauc_ndcg_at_5_max
value: 41.144848115666704
- type: nauc_ndcg_at_5_std
value: -0.25392100548648994
- type: nauc_precision_at_1000_diff1
value: -13.397374814996027
- type: nauc_precision_at_1000_max
value: -10.545825090982524
- type: nauc_precision_at_1000_std
value: -4.747735055324635
- type: nauc_precision_at_100_diff1
value: -8.535156679740567
- type: nauc_precision_at_100_max
value: 0.2336644050756316
- type: nauc_precision_at_100_std
value: 3.7114388215461003
- type: nauc_precision_at_10_diff1
value: 12.682314960533228
- type: nauc_precision_at_10_max
value: 23.26944470666603
- type: nauc_precision_at_10_std
value: 6.859952428715359
- type: nauc_precision_at_1_diff1
value: 54.88420521179768
- type: nauc_precision_at_1_max
value: 38.655520078749
- type: nauc_precision_at_1_std
value: -5.11895317994502
- type: nauc_precision_at_20_diff1
value: 3.9707906855321737
- type: nauc_precision_at_20_max
value: 16.30149037090611
- type: nauc_precision_at_20_std
value: 9.989513894902677
- type: nauc_precision_at_3_diff1
value: 31.858096738933366
- type: nauc_precision_at_3_max
value: 36.75008912797151
- type: nauc_precision_at_3_std
value: 1.6737865905809408
- type: nauc_precision_at_5_diff1
value: 22.716343790472106
- type: nauc_precision_at_5_max
value: 32.166437247193954
- type: nauc_precision_at_5_std
value: 6.62634536361429
- type: nauc_recall_at_1000_diff1
value: 36.30008006500106
- type: nauc_recall_at_1000_max
value: 49.71542662448003
- type: nauc_recall_at_1000_std
value: 60.31596168981692
- type: nauc_recall_at_100_diff1
value: 33.80857401221326
- type: nauc_recall_at_100_max
value: 36.02080829251763
- type: nauc_recall_at_100_std
value: 13.804209608741388
- type: nauc_recall_at_10_diff1
value: 41.117948804602406
- type: nauc_recall_at_10_max
value: 35.413166134512764
- type: nauc_recall_at_10_std
value: 2.6304195844088727
- type: nauc_recall_at_1_diff1
value: 58.01365735208089
- type: nauc_recall_at_1_max
value: 36.11029923363937
- type: nauc_recall_at_1_std
value: -6.239781340568959
- type: nauc_recall_at_20_diff1
value: 37.16004044588308
- type: nauc_recall_at_20_max
value: 36.560062732504555
- type: nauc_recall_at_20_std
value: 6.973789257435653
- type: nauc_recall_at_3_diff1
value: 47.174099321926924
- type: nauc_recall_at_3_max
value: 39.949088090651244
- type: nauc_recall_at_3_std
value: -0.8509590046361389
- type: nauc_recall_at_5_diff1
value: 43.036701638853245
- type: nauc_recall_at_5_max
value: 38.83941791509091
- type: nauc_recall_at_5_std
value: 2.789976323154503
- type: ndcg_at_1
value: 38.913
- type: ndcg_at_10
value: 51.13999999999999
- type: ndcg_at_100
value: 56.501000000000005
- type: ndcg_at_1000
value: 58.17099999999999
- type: ndcg_at_20
value: 53.178000000000004
- type: ndcg_at_3
value: 45.853
- type: ndcg_at_5
value: 48.760999999999996
- type: precision_at_1
value: 38.913
- type: precision_at_10
value: 9.757
- type: precision_at_100
value: 1.542
- type: precision_at_1000
value: 0.202
- type: precision_at_20
value: 5.722
- type: precision_at_3
value: 22.365
- type: precision_at_5
value: 16.252
- type: recall_at_1
value: 31.875999999999998
- type: recall_at_10
value: 64.08500000000001
- type: recall_at_100
value: 86.193
- type: recall_at_1000
value: 96.299
- type: recall_at_20
value: 71.21499999999999
- type: recall_at_3
value: 49.053000000000004
- type: recall_at_5
value: 57.118
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackEnglishRetrieval (default)
revision: ad9991cb51e31e31e430383c75ffb2885547b5f0
split: test
type: mteb/cqadupstack-english
metrics:
- type: main_score
value: 50.296
- type: map_at_1
value: 33.804
- type: map_at_10
value: 44.538
- type: map_at_100
value: 45.879999999999995
- type: map_at_1000
value: 46.007
- type: map_at_20
value: 45.281
- type: map_at_3
value: 41.342
- type: map_at_5
value: 43.151
- type: mrr_at_1
value: 42.29299363057325
- type: mrr_at_10
value: 50.706703063390975
- type: mrr_at_100
value: 51.36107673889394
- type: mrr_at_1000
value: 51.40319751345043
- type: mrr_at_20
value: 51.08645797135475
- type: mrr_at_3
value: 48.57749469214439
- type: mrr_at_5
value: 49.84182590233549
- type: nauc_map_at_1000_diff1
value: 54.56683725530487
- type: nauc_map_at_1000_max
value: 48.49408187949932
- type: nauc_map_at_1000_std
value: -3.5839028904579275
- type: nauc_map_at_100_diff1
value: 54.566859139948406
- type: nauc_map_at_100_max
value: 48.44953277653265
- type: nauc_map_at_100_std
value: -3.705886194658946
- type: nauc_map_at_10_diff1
value: 54.88281767981147
- type: nauc_map_at_10_max
value: 47.84874704653376
- type: nauc_map_at_10_std
value: -5.315305072400622
- type: nauc_map_at_1_diff1
value: 60.03024707119591
- type: nauc_map_at_1_max
value: 42.38572687085187
- type: nauc_map_at_1_std
value: -9.996537931467572
- type: nauc_map_at_20_diff1
value: 54.65673141969173
- type: nauc_map_at_20_max
value: 48.16868250940177
- type: nauc_map_at_20_std
value: -4.379488489858532
- type: nauc_map_at_3_diff1
value: 56.607842591824486
- type: nauc_map_at_3_max
value: 47.00891548339969
- type: nauc_map_at_3_std
value: -7.968421330238387
- type: nauc_map_at_5_diff1
value: 55.471191976038895
- type: nauc_map_at_5_max
value: 47.95637524120086
- type: nauc_map_at_5_std
value: -6.031637734399086
- type: nauc_mrr_at_1000_diff1
value: 54.39110691442512
- type: nauc_mrr_at_1000_max
value: 50.83821880005344
- type: nauc_mrr_at_1000_std
value: 0.6159692166702286
- type: nauc_mrr_at_100_diff1
value: 54.3696480803914
- type: nauc_mrr_at_100_max
value: 50.825993216095796
- type: nauc_mrr_at_100_std
value: 0.6052644921054788
- type: nauc_mrr_at_10_diff1
value: 54.371705478192155
- type: nauc_mrr_at_10_max
value: 50.79974257079621
- type: nauc_mrr_at_10_std
value: 0.3456650373233527
- type: nauc_mrr_at_1_diff1
value: 58.54031210437416
- type: nauc_mrr_at_1_max
value: 49.803488614239384
- type: nauc_mrr_at_1_std
value: -2.1741861057029577
- type: nauc_mrr_at_20_diff1
value: 54.35208768506187
- type: nauc_mrr_at_20_max
value: 50.822643574668305
- type: nauc_mrr_at_20_std
value: 0.578954360224674
- type: nauc_mrr_at_3_diff1
value: 55.41178985271698
- type: nauc_mrr_at_3_max
value: 51.30845595473003
- type: nauc_mrr_at_3_std
value: -0.37094021691855295
- type: nauc_mrr_at_5_diff1
value: 54.68364424829058
- type: nauc_mrr_at_5_max
value: 51.18624010202796
- type: nauc_mrr_at_5_std
value: 0.42103990655648693
- type: nauc_ndcg_at_1000_diff1
value: 52.3916781475271
- type: nauc_ndcg_at_1000_max
value: 49.93608862860138
- type: nauc_ndcg_at_1000_std
value: 1.5381095519614094
- type: nauc_ndcg_at_100_diff1
value: 51.90307196170437
- type: nauc_ndcg_at_100_max
value: 49.62275383422297
- type: nauc_ndcg_at_100_std
value: 0.9383703365058415
- type: nauc_ndcg_at_10_diff1
value: 52.58533497903788
- type: nauc_ndcg_at_10_max
value: 48.93940151477497
- type: nauc_ndcg_at_10_std
value: -1.9515406201397911
- type: nauc_ndcg_at_1_diff1
value: 58.54031210437416
- type: nauc_ndcg_at_1_max
value: 49.803488614239384
- type: nauc_ndcg_at_1_std
value: -2.1741861057029577
- type: nauc_ndcg_at_20_diff1
value: 52.2331243341277
- type: nauc_ndcg_at_20_max
value: 49.260105351011234
- type: nauc_ndcg_at_20_std
value: -0.3053807541983415
- type: nauc_ndcg_at_3_diff1
value: 54.791680253521605
- type: nauc_ndcg_at_3_max
value: 49.799479073639425
- type: nauc_ndcg_at_3_std
value: -3.359256810768316
- type: nauc_ndcg_at_5_diff1
value: 53.52759078473191
- type: nauc_ndcg_at_5_max
value: 49.94665111909378
- type: nauc_ndcg_at_5_std
value: -2.0072207277928458
- type: nauc_precision_at_1000_diff1
value: -15.110589414777442
- type: nauc_precision_at_1000_max
value: 4.326406684690144
- type: nauc_precision_at_1000_std
value: 30.76243679362501
- type: nauc_precision_at_100_diff1
value: -9.83664890888201
- type: nauc_precision_at_100_max
value: 14.399672821054294
- type: nauc_precision_at_100_std
value: 34.2742542251923
- type: nauc_precision_at_10_diff1
value: 9.164611778072883
- type: nauc_precision_at_10_max
value: 31.343758942964474
- type: nauc_precision_at_10_std
value: 22.428752877162957
- type: nauc_precision_at_1_diff1
value: 58.54031210437416
- type: nauc_precision_at_1_max
value: 49.803488614239384
- type: nauc_precision_at_1_std
value: -2.1741861057029577
- type: nauc_precision_at_20_diff1
value: 0.44929305898966365
- type: nauc_precision_at_20_max
value: 24.726708632266963
- type: nauc_precision_at_20_std
value: 29.25463805200049
- type: nauc_precision_at_3_diff1
value: 31.885633304061674
- type: nauc_precision_at_3_max
value: 45.891846997032395
- type: nauc_precision_at_3_std
value: 8.966423211748696
- type: nauc_precision_at_5_diff1
value: 20.161267334635085
- type: nauc_precision_at_5_max
value: 40.52868728124918
- type: nauc_precision_at_5_std
value: 17.21689901677931
- type: nauc_recall_at_1000_diff1
value: 37.66458313022363
- type: nauc_recall_at_1000_max
value: 53.91611634060227
- type: nauc_recall_at_1000_std
value: 34.524934918397186
- type: nauc_recall_at_100_diff1
value: 36.69397605115916
- type: nauc_recall_at_100_max
value: 47.01882921655524
- type: nauc_recall_at_100_std
value: 14.464239623489867
- type: nauc_recall_at_10_diff1
value: 44.14298927469828
- type: nauc_recall_at_10_max
value: 44.46655806020608
- type: nauc_recall_at_10_std
value: -1.4928423828526263
- type: nauc_recall_at_1_diff1
value: 60.03024707119591
- type: nauc_recall_at_1_max
value: 42.38572687085187
- type: nauc_recall_at_1_std
value: -9.996537931467572
- type: nauc_recall_at_20_diff1
value: 41.812238373115406
- type: nauc_recall_at_20_max
value: 45.375693604348776
- type: nauc_recall_at_20_std
value: 5.593760130004999
- type: nauc_recall_at_3_diff1
value: 51.77404626512503
- type: nauc_recall_at_3_max
value: 46.029602435907144
- type: nauc_recall_at_3_std
value: -7.471014645024994
- type: nauc_recall_at_5_diff1
value: 47.747156115149316
- type: nauc_recall_at_5_max
value: 47.14363391070899
- type: nauc_recall_at_5_std
value: -2.735138773478397
- type: ndcg_at_1
value: 42.293
- type: ndcg_at_10
value: 50.296
- type: ndcg_at_100
value: 54.778999999999996
- type: ndcg_at_1000
value: 56.663
- type: ndcg_at_20
value: 52.06699999999999
- type: ndcg_at_3
value: 46.041
- type: ndcg_at_5
value: 48.076
- type: precision_at_1
value: 42.293
- type: precision_at_10
value: 9.459
- type: precision_at_100
value: 1.505
- type: precision_at_1000
value: 0.19499999999999998
- type: precision_at_20
value: 5.567
- type: precision_at_3
value: 22.038
- type: precision_at_5
value: 15.693999999999999
- type: recall_at_1
value: 33.804
- type: recall_at_10
value: 59.971
- type: recall_at_100
value: 78.879
- type: recall_at_1000
value: 90.647
- type: recall_at_20
value: 66.408
- type: recall_at_3
value: 47.392
- type: recall_at_5
value: 53.032999999999994
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackGamingRetrieval (default)
revision: 4885aa143210c98657558c04aaf3dc47cfb54340
split: test
type: mteb/cqadupstack-gaming
metrics:
- type: main_score
value: 60.527
- type: map_at_1
value: 42.26
- type: map_at_10
value: 54.742000000000004
- type: map_at_100
value: 55.74
- type: map_at_1000
value: 55.791000000000004
- type: map_at_20
value: 55.364999999999995
- type: map_at_3
value: 51.5
- type: map_at_5
value: 53.301
- type: mrr_at_1
value: 48.025078369905955
- type: mrr_at_10
value: 58.06321839080471
- type: mrr_at_100
value: 58.64161796465698
- type: mrr_at_1000
value: 58.6678243740702
- type: mrr_at_20
value: 58.43083836697793
- type: mrr_at_3
value: 55.59038662486947
- type: mrr_at_5
value: 57.0606060606062
- type: nauc_map_at_1000_diff1
value: 57.68469941839035
- type: nauc_map_at_1000_max
value: 45.59142506456117
- type: nauc_map_at_1000_std
value: -5.396910224737828
- type: nauc_map_at_100_diff1
value: 57.65643922567225
- type: nauc_map_at_100_max
value: 45.56968717832569
- type: nauc_map_at_100_std
value: -5.412306450764012
- type: nauc_map_at_10_diff1
value: 57.77897303010143
- type: nauc_map_at_10_max
value: 45.12978752020397
- type: nauc_map_at_10_std
value: -6.345317352092361
- type: nauc_map_at_1_diff1
value: 60.697479048814365
- type: nauc_map_at_1_max
value: 38.05823293393858
- type: nauc_map_at_1_std
value: -9.161664737666927
- type: nauc_map_at_20_diff1
value: 57.682622717223964
- type: nauc_map_at_20_max
value: 45.447874846429414
- type: nauc_map_at_20_std
value: -5.764047463778564
- type: nauc_map_at_3_diff1
value: 58.216303078314745
- type: nauc_map_at_3_max
value: 42.82177195912272
- type: nauc_map_at_3_std
value: -9.177422772027217
- type: nauc_map_at_5_diff1
value: 57.831978664769515
- type: nauc_map_at_5_max
value: 44.3955327920567
- type: nauc_map_at_5_std
value: -7.251060540492739
- type: nauc_mrr_at_1000_diff1
value: 58.245524895038514
- type: nauc_mrr_at_1000_max
value: 47.34405771276604
- type: nauc_mrr_at_1000_std
value: -3.6164361713181763
- type: nauc_mrr_at_100_diff1
value: 58.230318675335035
- type: nauc_mrr_at_100_max
value: 47.35167087188821
- type: nauc_mrr_at_100_std
value: -3.593102604414633
- type: nauc_mrr_at_10_diff1
value: 58.227433914992886
- type: nauc_mrr_at_10_max
value: 47.422409583960516
- type: nauc_mrr_at_10_std
value: -3.7789443073829387
- type: nauc_mrr_at_1_diff1
value: 61.37480745418378
- type: nauc_mrr_at_1_max
value: 44.46335494127829
- type: nauc_mrr_at_1_std
value: -6.398833684107047
- type: nauc_mrr_at_20_diff1
value: 58.24797494245645
- type: nauc_mrr_at_20_max
value: 47.37904127771602
- type: nauc_mrr_at_20_std
value: -3.731995817052959
- type: nauc_mrr_at_3_diff1
value: 58.48756237849568
- type: nauc_mrr_at_3_max
value: 46.879616437231576
- type: nauc_mrr_at_3_std
value: -4.8092665335271745
- type: nauc_mrr_at_5_diff1
value: 58.26422531157446
- type: nauc_mrr_at_5_max
value: 47.34152451776259
- type: nauc_mrr_at_5_std
value: -3.9663476308504384
- type: nauc_ndcg_at_1000_diff1
value: 57.000164558241295
- type: nauc_ndcg_at_1000_max
value: 47.99688089374478
- type: nauc_ndcg_at_1000_std
value: -1.9126357723237575
- type: nauc_ndcg_at_100_diff1
value: 56.57305126432451
- type: nauc_ndcg_at_100_max
value: 48.19680136315171
- type: nauc_ndcg_at_100_std
value: -1.3608938883073507
- type: nauc_ndcg_at_10_diff1
value: 56.743775747671584
- type: nauc_ndcg_at_10_max
value: 47.78713158147057
- type: nauc_ndcg_at_10_std
value: -3.626852899788665
- type: nauc_ndcg_at_1_diff1
value: 61.37480745418378
- type: nauc_ndcg_at_1_max
value: 44.46335494127829
- type: nauc_ndcg_at_1_std
value: -6.398833684107047
- type: nauc_ndcg_at_20_diff1
value: 56.643175903925126
- type: nauc_ndcg_at_20_max
value: 48.076058985517875
- type: nauc_ndcg_at_20_std
value: -2.7492310018221398
- type: nauc_ndcg_at_3_diff1
value: 57.453965014993194
- type: nauc_ndcg_at_3_max
value: 45.06455205146776
- type: nauc_ndcg_at_3_std
value: -7.18539473462169
- type: nauc_ndcg_at_5_diff1
value: 56.89313351488996
- type: nauc_ndcg_at_5_max
value: 46.975542478071205
- type: nauc_ndcg_at_5_std
value: -4.784218744384206
- type: nauc_precision_at_1000_diff1
value: -11.239238315601717
- type: nauc_precision_at_1000_max
value: 17.990146685827984
- type: nauc_precision_at_1000_std
value: 31.90294609019782
- type: nauc_precision_at_100_diff1
value: -6.07236496776092
- type: nauc_precision_at_100_max
value: 25.0065905878517
- type: nauc_precision_at_100_std
value: 32.4525492436836
- type: nauc_precision_at_10_diff1
value: 16.041481355774245
- type: nauc_precision_at_10_max
value: 38.98173567292828
- type: nauc_precision_at_10_std
value: 17.243686720689265
- type: nauc_precision_at_1_diff1
value: 61.37480745418378
- type: nauc_precision_at_1_max
value: 44.46335494127829
- type: nauc_precision_at_1_std
value: -6.398833684107047
- type: nauc_precision_at_20_diff1
value: 5.8908160166614225
- type: nauc_precision_at_20_max
value: 34.34013274376776
- type: nauc_precision_at_20_std
value: 25.036513902310325
- type: nauc_precision_at_3_diff1
value: 36.0695938539411
- type: nauc_precision_at_3_max
value: 42.97015346497544
- type: nauc_precision_at_3_std
value: 0.31420720813888914
- type: nauc_precision_at_5_diff1
value: 26.966818469960817
- type: nauc_precision_at_5_max
value: 43.357320830990105
- type: nauc_precision_at_5_std
value: 10.016079169758655
- type: nauc_recall_at_1000_diff1
value: 41.07443433584645
- type: nauc_recall_at_1000_max
value: 74.35800718581933
- type: nauc_recall_at_1000_std
value: 63.881916604417945
- type: nauc_recall_at_100_diff1
value: 43.279390556626055
- type: nauc_recall_at_100_max
value: 59.337990830765655
- type: nauc_recall_at_100_std
value: 29.244613920045246
- type: nauc_recall_at_10_diff1
value: 49.39792783361698
- type: nauc_recall_at_10_max
value: 50.72607017464337
- type: nauc_recall_at_10_std
value: 1.812193906608468
- type: nauc_recall_at_1_diff1
value: 60.697479048814365
- type: nauc_recall_at_1_max
value: 38.05823293393858
- type: nauc_recall_at_1_std
value: -9.161664737666927
- type: nauc_recall_at_20_diff1
value: 47.88035753247517
- type: nauc_recall_at_20_max
value: 53.59722889727142
- type: nauc_recall_at_20_std
value: 7.720302507706711
- type: nauc_recall_at_3_diff1
value: 54.05535531328973
- type: nauc_recall_at_3_max
value: 43.92041482065396
- type: nauc_recall_at_3_std
value: -9.011510885504624
- type: nauc_recall_at_5_diff1
value: 51.539089243348634
- type: nauc_recall_at_5_max
value: 47.82878589262871
- type: nauc_recall_at_5_std
value: -3.0309072835580673
- type: ndcg_at_1
value: 48.025
- type: ndcg_at_10
value: 60.527
- type: ndcg_at_100
value: 64.21
- type: ndcg_at_1000
value: 65.273
- type: ndcg_at_20
value: 62.256
- type: ndcg_at_3
value: 55.1
- type: ndcg_at_5
value: 57.733000000000004
- type: precision_at_1
value: 48.025
- type: precision_at_10
value: 9.599
- type: precision_at_100
value: 1.2409999999999999
- type: precision_at_1000
value: 0.13699999999999998
- type: precision_at_20
value: 5.361
- type: precision_at_3
value: 24.326
- type: precision_at_5
value: 16.602
- type: recall_at_1
value: 42.26
- type: recall_at_10
value: 74.278
- type: recall_at_100
value: 89.681
- type: recall_at_1000
value: 97.436
- type: recall_at_20
value: 80.567
- type: recall_at_3
value: 59.818000000000005
- type: recall_at_5
value: 66.202
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackGisRetrieval (default)
revision: 5003b3064772da1887988e05400cf3806fe491f2
split: test
type: mteb/cqadupstack-gis
metrics:
- type: main_score
value: 42.44
- type: map_at_1
value: 28.431
- type: map_at_10
value: 37.483
- type: map_at_100
value: 38.548
- type: map_at_1000
value: 38.626
- type: map_at_20
value: 38.088
- type: map_at_3
value: 34.988
- type: map_at_5
value: 36.301
- type: mrr_at_1
value: 30.16949152542373
- type: mrr_at_10
value: 39.28804591516455
- type: mrr_at_100
value: 40.2114211709097
- type: mrr_at_1000
value: 40.26624502428491
- type: mrr_at_20
value: 39.79719852386063
- type: mrr_at_3
value: 37.00564971751412
- type: mrr_at_5
value: 38.24293785310733
- type: nauc_map_at_1000_diff1
value: 46.32961961356468
- type: nauc_map_at_1000_max
value: 38.165852798192624
- type: nauc_map_at_1000_std
value: -0.9702769617726098
- type: nauc_map_at_100_diff1
value: 46.322895800738
- type: nauc_map_at_100_max
value: 38.154702745302046
- type: nauc_map_at_100_std
value: -0.9931864230580432
- type: nauc_map_at_10_diff1
value: 46.52143278450031
- type: nauc_map_at_10_max
value: 38.09424035307391
- type: nauc_map_at_10_std
value: -1.3823180760391551
- type: nauc_map_at_1_diff1
value: 51.63517035452425
- type: nauc_map_at_1_max
value: 36.69722252554175
- type: nauc_map_at_1_std
value: -3.831480331268112
- type: nauc_map_at_20_diff1
value: 46.21454269797542
- type: nauc_map_at_20_max
value: 38.19824729085207
- type: nauc_map_at_20_std
value: -1.064676446525204
- type: nauc_map_at_3_diff1
value: 47.1593516967249
- type: nauc_map_at_3_max
value: 36.850463483045466
- type: nauc_map_at_3_std
value: -2.603604341948898
- type: nauc_map_at_5_diff1
value: 46.851347485378945
- type: nauc_map_at_5_max
value: 37.28186536727122
- type: nauc_map_at_5_std
value: -2.4340539528920173
- type: nauc_mrr_at_1000_diff1
value: 45.576301316917046
- type: nauc_mrr_at_1000_max
value: 39.074110221039085
- type: nauc_mrr_at_1000_std
value: 0.34534056002843444
- type: nauc_mrr_at_100_diff1
value: 45.56601971007378
- type: nauc_mrr_at_100_max
value: 39.07924381253435
- type: nauc_mrr_at_100_std
value: 0.3486853785425758
- type: nauc_mrr_at_10_diff1
value: 45.56810190033719
- type: nauc_mrr_at_10_max
value: 39.106855111654276
- type: nauc_mrr_at_10_std
value: -0.011355717882556595
- type: nauc_mrr_at_1_diff1
value: 50.48453829424686
- type: nauc_mrr_at_1_max
value: 38.72068337757316
- type: nauc_mrr_at_1_std
value: -1.7255691478217123
- type: nauc_mrr_at_20_diff1
value: 45.4303967081555
- type: nauc_mrr_at_20_max
value: 39.157488031782066
- type: nauc_mrr_at_20_std
value: 0.322488192059293
- type: nauc_mrr_at_3_diff1
value: 46.402007141090145
- type: nauc_mrr_at_3_max
value: 38.516972397422585
- type: nauc_mrr_at_3_std
value: -0.6896323185509712
- type: nauc_mrr_at_5_diff1
value: 46.033801472314146
- type: nauc_mrr_at_5_max
value: 38.668578627970774
- type: nauc_mrr_at_5_std
value: -0.47866221242356105
- type: nauc_ndcg_at_1000_diff1
value: 44.22556271852293
- type: nauc_ndcg_at_1000_max
value: 39.197223132630974
- type: nauc_ndcg_at_1000_std
value: 1.6609266713997726
- type: nauc_ndcg_at_100_diff1
value: 43.948213318276515
- type: nauc_ndcg_at_100_max
value: 39.09172571243978
- type: nauc_ndcg_at_100_std
value: 1.7609867316747878
- type: nauc_ndcg_at_10_diff1
value: 44.262862454963646
- type: nauc_ndcg_at_10_max
value: 38.91822398512767
- type: nauc_ndcg_at_10_std
value: 0.19680397573589933
- type: nauc_ndcg_at_1_diff1
value: 50.48453829424686
- type: nauc_ndcg_at_1_max
value: 38.72068337757316
- type: nauc_ndcg_at_1_std
value: -1.7255691478217123
- type: nauc_ndcg_at_20_diff1
value: 43.369269498661
- type: nauc_ndcg_at_20_max
value: 39.236172489980106
- type: nauc_ndcg_at_20_std
value: 1.338917850299737
- type: nauc_ndcg_at_3_diff1
value: 45.88344269500731
- type: nauc_ndcg_at_3_max
value: 37.3029933446041
- type: nauc_ndcg_at_3_std
value: -2.1174725222875233
- type: nauc_ndcg_at_5_diff1
value: 45.291949971762975
- type: nauc_ndcg_at_5_max
value: 37.55334078742216
- type: nauc_ndcg_at_5_std
value: -1.6936327950688483
- type: nauc_precision_at_1000_diff1
value: -7.730434905061115
- type: nauc_precision_at_1000_max
value: 18.665222572509492
- type: nauc_precision_at_1000_std
value: 13.810754019276594
- type: nauc_precision_at_100_diff1
value: 9.982399206105127
- type: nauc_precision_at_100_max
value: 29.893424795351013
- type: nauc_precision_at_100_std
value: 13.132808977788613
- type: nauc_precision_at_10_diff1
value: 29.60406660861134
- type: nauc_precision_at_10_max
value: 41.963933867122
- type: nauc_precision_at_10_std
value: 6.9453321105070716
- type: nauc_precision_at_1_diff1
value: 50.48453829424686
- type: nauc_precision_at_1_max
value: 38.72068337757316
- type: nauc_precision_at_1_std
value: -1.7255691478217123
- type: nauc_precision_at_20_diff1
value: 21.275583575462562
- type: nauc_precision_at_20_max
value: 40.82934150965418
- type: nauc_precision_at_20_std
value: 11.295996637878691
- type: nauc_precision_at_3_diff1
value: 37.917326556588
- type: nauc_precision_at_3_max
value: 38.1141616253519
- type: nauc_precision_at_3_std
value: 0.2777687308102478
- type: nauc_precision_at_5_diff1
value: 35.78582736361838
- type: nauc_precision_at_5_max
value: 39.18304044328737
- type: nauc_precision_at_5_std
value: 0.902359580455655
- type: nauc_recall_at_1000_diff1
value: 22.451692165754082
- type: nauc_recall_at_1000_max
value: 46.17514739163666
- type: nauc_recall_at_1000_std
value: 35.224884802752754
- type: nauc_recall_at_100_diff1
value: 31.690153470492504
- type: nauc_recall_at_100_max
value: 40.73748687827252
- type: nauc_recall_at_100_std
value: 16.776743149355948
- type: nauc_recall_at_10_diff1
value: 36.85261287305019
- type: nauc_recall_at_10_max
value: 39.092523024632776
- type: nauc_recall_at_10_std
value: 4.002388966219084
- type: nauc_recall_at_1_diff1
value: 51.63517035452425
- type: nauc_recall_at_1_max
value: 36.69722252554175
- type: nauc_recall_at_1_std
value: -3.831480331268112
- type: nauc_recall_at_20_diff1
value: 32.439638047907884
- type: nauc_recall_at_20_max
value: 40.27654103353758
- type: nauc_recall_at_20_std
value: 9.161976985898443
- type: nauc_recall_at_3_diff1
value: 42.353276628212654
- type: nauc_recall_at_3_max
value: 35.321616606477946
- type: nauc_recall_at_3_std
value: -2.22584709718968
- type: nauc_recall_at_5_diff1
value: 40.64466428494762
- type: nauc_recall_at_5_max
value: 35.90255543446805
- type: nauc_recall_at_5_std
value: -1.220727925020192
- type: ndcg_at_1
value: 30.169
- type: ndcg_at_10
value: 42.44
- type: ndcg_at_100
value: 47.595
- type: ndcg_at_1000
value: 49.503
- type: ndcg_at_20
value: 44.458999999999996
- type: ndcg_at_3
value: 37.569
- type: ndcg_at_5
value: 39.741
- type: precision_at_1
value: 30.169
- type: precision_at_10
value: 6.3950000000000005
- type: precision_at_100
value: 0.946
- type: precision_at_1000
value: 0.11399999999999999
- type: precision_at_20
value: 3.6839999999999997
- type: precision_at_3
value: 15.781999999999998
- type: precision_at_5
value: 10.757
- type: recall_at_1
value: 28.431
- type: recall_at_10
value: 55.872
- type: recall_at_100
value: 79.401
- type: recall_at_1000
value: 93.623
- type: recall_at_20
value: 63.458000000000006
- type: recall_at_3
value: 42.584
- type: recall_at_5
value: 47.81
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackMathematicaRetrieval (default)
revision: 90fceea13679c63fe563ded68f3b6f06e50061de
split: test
type: mteb/cqadupstack-mathematica
metrics:
- type: main_score
value: 34.291
- type: map_at_1
value: 19.764
- type: map_at_10
value: 28.725
- type: map_at_100
value: 30.026999999999997
- type: map_at_1000
value: 30.144
- type: map_at_20
value: 29.374
- type: map_at_3
value: 25.685999999999996
- type: map_at_5
value: 27.538
- type: mrr_at_1
value: 24.502487562189053
- type: mrr_at_10
value: 33.714019189765445
- type: mrr_at_100
value: 34.626427656076494
- type: mrr_at_1000
value: 34.69228104298719
- type: mrr_at_20
value: 34.16244841496677
- type: mrr_at_3
value: 30.97014925373136
- type: mrr_at_5
value: 32.65547263681592
- type: nauc_map_at_1000_diff1
value: 35.57596229591453
- type: nauc_map_at_1000_max
value: 28.38203992282376
- type: nauc_map_at_1000_std
value: -0.690729162878796
- type: nauc_map_at_100_diff1
value: 35.5604393324392
- type: nauc_map_at_100_max
value: 28.419457028070088
- type: nauc_map_at_100_std
value: -0.6662063689718012
- type: nauc_map_at_10_diff1
value: 35.98010635277429
- type: nauc_map_at_10_max
value: 28.098878796132592
- type: nauc_map_at_10_std
value: -1.3684662626404016
- type: nauc_map_at_1_diff1
value: 39.50120884151396
- type: nauc_map_at_1_max
value: 28.45531349851952
- type: nauc_map_at_1_std
value: -1.9541675008858381
- type: nauc_map_at_20_diff1
value: 35.70885247227986
- type: nauc_map_at_20_max
value: 28.241968097999848
- type: nauc_map_at_20_std
value: -1.0783688781802567
- type: nauc_map_at_3_diff1
value: 36.62960019358422
- type: nauc_map_at_3_max
value: 27.651864166748158
- type: nauc_map_at_3_std
value: -1.8336204362277755
- type: nauc_map_at_5_diff1
value: 36.33961034230329
- type: nauc_map_at_5_max
value: 27.760001974081433
- type: nauc_map_at_5_std
value: -2.0066619580146496
- type: nauc_mrr_at_1000_diff1
value: 34.9491682537747
- type: nauc_mrr_at_1000_max
value: 28.316660783130075
- type: nauc_mrr_at_1000_std
value: 0.4076261826966661
- type: nauc_mrr_at_100_diff1
value: 34.91240146695054
- type: nauc_mrr_at_100_max
value: 28.339210985696
- type: nauc_mrr_at_100_std
value: 0.4333198774648973
- type: nauc_mrr_at_10_diff1
value: 35.02010270149501
- type: nauc_mrr_at_10_max
value: 28.09462862542247
- type: nauc_mrr_at_10_std
value: 0.13089152466665158
- type: nauc_mrr_at_1_diff1
value: 39.13821028058753
- type: nauc_mrr_at_1_max
value: 28.032556829304085
- type: nauc_mrr_at_1_std
value: -1.7975185722477194
- type: nauc_mrr_at_20_diff1
value: 35.00195546992316
- type: nauc_mrr_at_20_max
value: 28.290101173427235
- type: nauc_mrr_at_20_std
value: 0.359773410485394
- type: nauc_mrr_at_3_diff1
value: 35.20720807148267
- type: nauc_mrr_at_3_max
value: 27.30167569999325
- type: nauc_mrr_at_3_std
value: -0.8635949909966647
- type: nauc_mrr_at_5_diff1
value: 35.46207024573666
- type: nauc_mrr_at_5_max
value: 27.891834763623958
- type: nauc_mrr_at_5_std
value: -0.696818449554613
- type: nauc_ndcg_at_1000_diff1
value: 33.70427245790907
- type: nauc_ndcg_at_1000_max
value: 29.276969012460576
- type: nauc_ndcg_at_1000_std
value: 2.194555942099149
- type: nauc_ndcg_at_100_diff1
value: 32.87395869015526
- type: nauc_ndcg_at_100_max
value: 29.88029945297434
- type: nauc_ndcg_at_100_std
value: 2.70304791945048
- type: nauc_ndcg_at_10_diff1
value: 34.502280192943154
- type: nauc_ndcg_at_10_max
value: 28.320249495045204
- type: nauc_ndcg_at_10_std
value: -0.1924925962793944
- type: nauc_ndcg_at_1_diff1
value: 39.13821028058753
- type: nauc_ndcg_at_1_max
value: 28.032556829304085
- type: nauc_ndcg_at_1_std
value: -1.7975185722477194
- type: nauc_ndcg_at_20_diff1
value: 33.79122969701137
- type: nauc_ndcg_at_20_max
value: 28.870024567697943
- type: nauc_ndcg_at_20_std
value: 0.7305874205815353
- type: nauc_ndcg_at_3_diff1
value: 35.329659534748394
- type: nauc_ndcg_at_3_max
value: 27.07741989089553
- type: nauc_ndcg_at_3_std
value: -1.643048735061724
- type: nauc_ndcg_at_5_diff1
value: 35.28956734308783
- type: nauc_ndcg_at_5_max
value: 27.69245066519595
- type: nauc_ndcg_at_5_std
value: -1.629873689869274
- type: nauc_precision_at_1000_diff1
value: -5.694759247422408
- type: nauc_precision_at_1000_max
value: -0.9408325343255068
- type: nauc_precision_at_1000_std
value: 5.7655023451839265
- type: nauc_precision_at_100_diff1
value: 0.8779676665401134
- type: nauc_precision_at_100_max
value: 13.132848409570602
- type: nauc_precision_at_100_std
value: 11.850507881087424
- type: nauc_precision_at_10_diff1
value: 20.8071512227297
- type: nauc_precision_at_10_max
value: 22.883196969880657
- type: nauc_precision_at_10_std
value: 3.239454376922377
- type: nauc_precision_at_1_diff1
value: 39.13821028058753
- type: nauc_precision_at_1_max
value: 28.032556829304085
- type: nauc_precision_at_1_std
value: -1.7975185722477194
- type: nauc_precision_at_20_diff1
value: 15.132010740443071
- type: nauc_precision_at_20_max
value: 21.052364899121738
- type: nauc_precision_at_20_std
value: 5.804469246961994
- type: nauc_precision_at_3_diff1
value: 30.208155830085186
- type: nauc_precision_at_3_max
value: 24.16749026183028
- type: nauc_precision_at_3_std
value: -1.5797872699611786
- type: nauc_precision_at_5_diff1
value: 26.38635435647173
- type: nauc_precision_at_5_max
value: 22.683436023693865
- type: nauc_precision_at_5_std
value: -1.5402517263725906
- type: nauc_recall_at_1000_diff1
value: 21.01573139251367
- type: nauc_recall_at_1000_max
value: 36.707056738115696
- type: nauc_recall_at_1000_std
value: 29.60248226406072
- type: nauc_recall_at_100_diff1
value: 19.72306401627329
- type: nauc_recall_at_100_max
value: 35.676894052767516
- type: nauc_recall_at_100_std
value: 16.899170359903692
- type: nauc_recall_at_10_diff1
value: 29.869444323362888
- type: nauc_recall_at_10_max
value: 26.908177270931944
- type: nauc_recall_at_10_std
value: 1.2933628719022767
- type: nauc_recall_at_1_diff1
value: 39.50120884151396
- type: nauc_recall_at_1_max
value: 28.45531349851952
- type: nauc_recall_at_1_std
value: -1.9541675008858381
- type: nauc_recall_at_20_diff1
value: 27.047582643971186
- type: nauc_recall_at_20_max
value: 28.732557711218572
- type: nauc_recall_at_20_std
value: 4.336006930018247
- type: nauc_recall_at_3_diff1
value: 32.86864754117765
- type: nauc_recall_at_3_max
value: 25.94332206909362
- type: nauc_recall_at_3_std
value: -1.1620399702795094
- type: nauc_recall_at_5_diff1
value: 32.342666750682774
- type: nauc_recall_at_5_max
value: 26.349619752191987
- type: nauc_recall_at_5_std
value: -1.529754929810119
- type: ndcg_at_1
value: 24.502
- type: ndcg_at_10
value: 34.291
- type: ndcg_at_100
value: 40.288000000000004
- type: ndcg_at_1000
value: 42.925000000000004
- type: ndcg_at_20
value: 36.263
- type: ndcg_at_3
value: 28.945
- type: ndcg_at_5
value: 31.782
- type: precision_at_1
value: 24.502
- type: precision_at_10
value: 6.331
- type: precision_at_100
value: 1.076
- type: precision_at_1000
value: 0.14300000000000002
- type: precision_at_20
value: 3.7249999999999996
- type: precision_at_3
value: 13.764999999999999
- type: precision_at_5
value: 10.299
- type: recall_at_1
value: 19.764
- type: recall_at_10
value: 46.336
- type: recall_at_100
value: 72.56099999999999
- type: recall_at_1000
value: 91.169
- type: recall_at_20
value: 53.333
- type: recall_at_3
value: 32.048
- type: recall_at_5
value: 39.031
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackPhysicsRetrieval (default)
revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4
split: test
type: mteb/cqadupstack-physics
metrics:
- type: main_score
value: 49.303000000000004
- type: map_at_1
value: 31.924000000000003
- type: map_at_10
value: 43.146
- type: map_at_100
value: 44.554
- type: map_at_1000
value: 44.66
- type: map_at_20
value: 43.976
- type: map_at_3
value: 39.875
- type: map_at_5
value: 41.745
- type: mrr_at_1
value: 38.69104908565929
- type: mrr_at_10
value: 48.47644255007098
- type: mrr_at_100
value: 49.34802027997514
- type: mrr_at_1000
value: 49.380583664727936
- type: mrr_at_20
value: 49.02909013388238
- type: mrr_at_3
value: 45.92556945781194
- type: mrr_at_5
value: 47.43663779274935
- type: nauc_map_at_1000_diff1
value: 48.84550774397288
- type: nauc_map_at_1000_max
value: 35.81794187175693
- type: nauc_map_at_1000_std
value: 0.22536794057131404
- type: nauc_map_at_100_diff1
value: 48.85268386507044
- type: nauc_map_at_100_max
value: 35.79056749059193
- type: nauc_map_at_100_std
value: 0.19018082136844575
- type: nauc_map_at_10_diff1
value: 48.93103596642255
- type: nauc_map_at_10_max
value: 35.415728335693316
- type: nauc_map_at_10_std
value: -0.516949849063086
- type: nauc_map_at_1_diff1
value: 53.21728600893859
- type: nauc_map_at_1_max
value: 32.54727663088595
- type: nauc_map_at_1_std
value: -3.0188785305607313
- type: nauc_map_at_20_diff1
value: 48.81835507489666
- type: nauc_map_at_20_max
value: 35.52328960790258
- type: nauc_map_at_20_std
value: -0.13946969398009854
- type: nauc_map_at_3_diff1
value: 49.60985515546279
- type: nauc_map_at_3_max
value: 35.10299804197055
- type: nauc_map_at_3_std
value: -0.9762979035483001
- type: nauc_map_at_5_diff1
value: 49.11002536317524
- type: nauc_map_at_5_max
value: 35.10854989975629
- type: nauc_map_at_5_std
value: -0.8965490079931548
- type: nauc_mrr_at_1000_diff1
value: 48.78698962612968
- type: nauc_mrr_at_1000_max
value: 37.55353465130176
- type: nauc_mrr_at_1000_std
value: 1.536431144069894
- type: nauc_mrr_at_100_diff1
value: 48.78242829567701
- type: nauc_mrr_at_100_max
value: 37.551385270948074
- type: nauc_mrr_at_100_std
value: 1.5277115106405326
- type: nauc_mrr_at_10_diff1
value: 48.6143150482425
- type: nauc_mrr_at_10_max
value: 37.60644981991345
- type: nauc_mrr_at_10_std
value: 1.325623423551219
- type: nauc_mrr_at_1_diff1
value: 52.61335953411452
- type: nauc_mrr_at_1_max
value: 37.03819912101619
- type: nauc_mrr_at_1_std
value: 0.9518117747648859
- type: nauc_mrr_at_20_diff1
value: 48.69769812994518
- type: nauc_mrr_at_20_max
value: 37.45441542221455
- type: nauc_mrr_at_20_std
value: 1.3952548275759868
- type: nauc_mrr_at_3_diff1
value: 48.73125478258506
- type: nauc_mrr_at_3_max
value: 37.56704254021896
- type: nauc_mrr_at_3_std
value: 1.093041093510586
- type: nauc_mrr_at_5_diff1
value: 48.597626652161495
- type: nauc_mrr_at_5_max
value: 37.525828508835374
- type: nauc_mrr_at_5_std
value: 1.0516478037316366
- type: nauc_ndcg_at_1000_diff1
value: 47.3499766836802
- type: nauc_ndcg_at_1000_max
value: 37.39676833940487
- type: nauc_ndcg_at_1000_std
value: 2.8152529026496786
- type: nauc_ndcg_at_100_diff1
value: 47.230060964543306
- type: nauc_ndcg_at_100_max
value: 37.20558423020247
- type: nauc_ndcg_at_100_std
value: 2.820393724231608
- type: nauc_ndcg_at_10_diff1
value: 47.08709208532369
- type: nauc_ndcg_at_10_max
value: 36.026920423014595
- type: nauc_ndcg_at_10_std
value: 0.2240933751762523
- type: nauc_ndcg_at_1_diff1
value: 52.61335953411452
- type: nauc_ndcg_at_1_max
value: 37.03819912101619
- type: nauc_ndcg_at_1_std
value: 0.9518117747648859
- type: nauc_ndcg_at_20_diff1
value: 46.88625942179394
- type: nauc_ndcg_at_20_max
value: 36.03488052587188
- type: nauc_ndcg_at_20_std
value: 1.236442994633607
- type: nauc_ndcg_at_3_diff1
value: 47.689152724401644
- type: nauc_ndcg_at_3_max
value: 36.62153374841597
- type: nauc_ndcg_at_3_std
value: 0.17430453803003293
- type: nauc_ndcg_at_5_diff1
value: 47.21794498716229
- type: nauc_ndcg_at_5_max
value: 35.95631541728226
- type: nauc_ndcg_at_5_std
value: -0.236310820705172
- type: nauc_precision_at_1000_diff1
value: -16.705059737681164
- type: nauc_precision_at_1000_max
value: 3.7599790944969422
- type: nauc_precision_at_1000_std
value: 11.59711423430256
- type: nauc_precision_at_100_diff1
value: -7.864826711824362
- type: nauc_precision_at_100_max
value: 13.424126682433425
- type: nauc_precision_at_100_std
value: 16.579823214278466
- type: nauc_precision_at_10_diff1
value: 13.696824092012164
- type: nauc_precision_at_10_max
value: 26.891499011605752
- type: nauc_precision_at_10_std
value: 9.856864763440592
- type: nauc_precision_at_1_diff1
value: 52.61335953411452
- type: nauc_precision_at_1_max
value: 37.03819912101619
- type: nauc_precision_at_1_std
value: 0.9518117747648859
- type: nauc_precision_at_20_diff1
value: 5.324857846153909
- type: nauc_precision_at_20_max
value: 21.81087795894526
- type: nauc_precision_at_20_std
value: 13.412775538465915
- type: nauc_precision_at_3_diff1
value: 31.242550398921416
- type: nauc_precision_at_3_max
value: 34.5829539652064
- type: nauc_precision_at_3_std
value: 6.347216244175967
- type: nauc_precision_at_5_diff1
value: 22.492425498194702
- type: nauc_precision_at_5_max
value: 30.31201235418867
- type: nauc_precision_at_5_std
value: 7.402646440989876
- type: nauc_recall_at_1000_diff1
value: 33.65968769734123
- type: nauc_recall_at_1000_max
value: 55.900889031212344
- type: nauc_recall_at_1000_std
value: 47.170982728564866
- type: nauc_recall_at_100_diff1
value: 35.166052113250174
- type: nauc_recall_at_100_max
value: 38.64602564308866
- type: nauc_recall_at_100_std
value: 18.52024249114377
- type: nauc_recall_at_10_diff1
value: 38.630076351635104
- type: nauc_recall_at_10_max
value: 32.649892256730524
- type: nauc_recall_at_10_std
value: 0.3076556226560256
- type: nauc_recall_at_1_diff1
value: 53.21728600893859
- type: nauc_recall_at_1_max
value: 32.54727663088595
- type: nauc_recall_at_1_std
value: -3.0188785305607313
- type: nauc_recall_at_20_diff1
value: 36.15521966085387
- type: nauc_recall_at_20_max
value: 31.22467911315917
- type: nauc_recall_at_20_std
value: 4.15858720739749
- type: nauc_recall_at_3_diff1
value: 43.02781903441642
- type: nauc_recall_at_3_max
value: 34.38891242158629
- type: nauc_recall_at_3_std
value: -0.757582965687976
- type: nauc_recall_at_5_diff1
value: 40.64825711893622
- type: nauc_recall_at_5_max
value: 32.852174581823164
- type: nauc_recall_at_5_std
value: -1.5443471014639971
- type: ndcg_at_1
value: 38.690999999999995
- type: ndcg_at_10
value: 49.303000000000004
- type: ndcg_at_100
value: 54.908
- type: ndcg_at_1000
value: 56.631
- type: ndcg_at_20
value: 51.707
- type: ndcg_at_3
value: 44.159
- type: ndcg_at_5
value: 46.676
- type: precision_at_1
value: 38.690999999999995
- type: precision_at_10
value: 8.863999999999999
- type: precision_at_100
value: 1.359
- type: precision_at_1000
value: 0.168
- type: precision_at_20
value: 5.2170000000000005
- type: precision_at_3
value: 20.982
- type: precision_at_5
value: 14.802999999999999
- type: recall_at_1
value: 31.924000000000003
- type: recall_at_10
value: 62.144
- type: recall_at_100
value: 85.097
- type: recall_at_1000
value: 95.883
- type: recall_at_20
value: 70.33
- type: recall_at_3
value: 47.991
- type: recall_at_5
value: 54.376000000000005
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackProgrammersRetrieval (default)
revision: 6184bc1440d2dbc7612be22b50686b8826d22b32
split: test
type: mteb/cqadupstack-programmers
metrics:
- type: main_score
value: 46.049
- type: map_at_1
value: 28.131
- type: map_at_10
value: 39.662
- type: map_at_100
value: 41.043
- type: map_at_1000
value: 41.152
- type: map_at_20
value: 40.453
- type: map_at_3
value: 36.07
- type: map_at_5
value: 37.974000000000004
- type: mrr_at_1
value: 34.58904109589041
- type: mrr_at_10
value: 44.79560774081322
- type: mrr_at_100
value: 45.65666456472965
- type: mrr_at_1000
value: 45.709171236475285
- type: mrr_at_20
value: 45.31922159656318
- type: mrr_at_3
value: 42.00913242009132
- type: mrr_at_5
value: 43.55593607305935
- type: nauc_map_at_1000_diff1
value: 46.38739929047425
- type: nauc_map_at_1000_max
value: 43.67319031779447
- type: nauc_map_at_1000_std
value: 1.4513770492879865
- type: nauc_map_at_100_diff1
value: 46.37460352931119
- type: nauc_map_at_100_max
value: 43.682012188986
- type: nauc_map_at_100_std
value: 1.4877623084621687
- type: nauc_map_at_10_diff1
value: 46.41679005550057
- type: nauc_map_at_10_max
value: 43.290283078254724
- type: nauc_map_at_10_std
value: 0.724933334439044
- type: nauc_map_at_1_diff1
value: 51.955635644994494
- type: nauc_map_at_1_max
value: 38.626441719712105
- type: nauc_map_at_1_std
value: -7.853806336363088
- type: nauc_map_at_20_diff1
value: 46.366590683816305
- type: nauc_map_at_20_max
value: 43.703205314864775
- type: nauc_map_at_20_std
value: 1.3730427800568408
- type: nauc_map_at_3_diff1
value: 46.54309601264766
- type: nauc_map_at_3_max
value: 41.4421671346271
- type: nauc_map_at_3_std
value: -2.046024542775744
- type: nauc_map_at_5_diff1
value: 46.825945954585876
- type: nauc_map_at_5_max
value: 42.78209268447956
- type: nauc_map_at_5_std
value: -0.6247715944388534
- type: nauc_mrr_at_1000_diff1
value: 46.261124400909296
- type: nauc_mrr_at_1000_max
value: 43.53298904167663
- type: nauc_mrr_at_1000_std
value: 2.770814524756863
- type: nauc_mrr_at_100_diff1
value: 46.233007095232125
- type: nauc_mrr_at_100_max
value: 43.52855419929045
- type: nauc_mrr_at_100_std
value: 2.794136786390839
- type: nauc_mrr_at_10_diff1
value: 46.08770762278749
- type: nauc_mrr_at_10_max
value: 43.41765687740783
- type: nauc_mrr_at_10_std
value: 2.5671137130095314
- type: nauc_mrr_at_1_diff1
value: 51.809200988712114
- type: nauc_mrr_at_1_max
value: 42.143027437627914
- type: nauc_mrr_at_1_std
value: -2.3449452777919904
- type: nauc_mrr_at_20_diff1
value: 46.25571447063488
- type: nauc_mrr_at_20_max
value: 43.668461419521385
- type: nauc_mrr_at_20_std
value: 2.905193962569393
- type: nauc_mrr_at_3_diff1
value: 46.13198756634534
- type: nauc_mrr_at_3_max
value: 42.81581457224921
- type: nauc_mrr_at_3_std
value: 1.5019842150987233
- type: nauc_mrr_at_5_diff1
value: 46.336329247406574
- type: nauc_mrr_at_5_max
value: 43.3726801047392
- type: nauc_mrr_at_5_std
value: 1.785280852484876
- type: nauc_ndcg_at_1000_diff1
value: 44.76884896858142
- type: nauc_ndcg_at_1000_max
value: 44.743593389767376
- type: nauc_ndcg_at_1000_std
value: 5.128912401544881
- type: nauc_ndcg_at_100_diff1
value: 44.45603468652885
- type: nauc_ndcg_at_100_max
value: 45.1312552973692
- type: nauc_ndcg_at_100_std
value: 6.1977329173102556
- type: nauc_ndcg_at_10_diff1
value: 44.269977956663155
- type: nauc_ndcg_at_10_max
value: 44.20807172796694
- type: nauc_ndcg_at_10_std
value: 4.018614242669085
- type: nauc_ndcg_at_1_diff1
value: 51.809200988712114
- type: nauc_ndcg_at_1_max
value: 42.143027437627914
- type: nauc_ndcg_at_1_std
value: -2.3449452777919904
- type: nauc_ndcg_at_20_diff1
value: 44.38671110266267
- type: nauc_ndcg_at_20_max
value: 45.44627226400632
- type: nauc_ndcg_at_20_std
value: 5.881156176221577
- type: nauc_ndcg_at_3_diff1
value: 44.116413020706624
- type: nauc_ndcg_at_3_max
value: 42.15844844057657
- type: nauc_ndcg_at_3_std
value: 0.945350635355976
- type: nauc_ndcg_at_5_diff1
value: 45.04867798354583
- type: nauc_ndcg_at_5_max
value: 43.744553860302545
- type: nauc_ndcg_at_5_std
value: 1.7040194566243974
- type: nauc_precision_at_1000_diff1
value: -17.071841539111407
- type: nauc_precision_at_1000_max
value: -6.812201002414789
- type: nauc_precision_at_1000_std
value: 6.126921524534793
- type: nauc_precision_at_100_diff1
value: -4.425411120928697
- type: nauc_precision_at_100_max
value: 12.18167765551403
- type: nauc_precision_at_100_std
value: 19.969852618617573
- type: nauc_precision_at_10_diff1
value: 15.005915826607222
- type: nauc_precision_at_10_max
value: 34.32379569960632
- type: nauc_precision_at_10_std
value: 20.737838659477493
- type: nauc_precision_at_1_diff1
value: 51.809200988712114
- type: nauc_precision_at_1_max
value: 42.143027437627914
- type: nauc_precision_at_1_std
value: -2.3449452777919904
- type: nauc_precision_at_20_diff1
value: 9.122026694212213
- type: nauc_precision_at_20_max
value: 30.380306449619386
- type: nauc_precision_at_20_std
value: 24.397028114454507
- type: nauc_precision_at_3_diff1
value: 27.71449724894348
- type: nauc_precision_at_3_max
value: 39.87494121351271
- type: nauc_precision_at_3_std
value: 11.74417947241022
- type: nauc_precision_at_5_diff1
value: 24.122446650573583
- type: nauc_precision_at_5_max
value: 39.179070381037874
- type: nauc_precision_at_5_std
value: 15.648339545201068
- type: nauc_recall_at_1000_diff1
value: 26.963361455157138
- type: nauc_recall_at_1000_max
value: 52.72393844073636
- type: nauc_recall_at_1000_std
value: 47.794042273843154
- type: nauc_recall_at_100_diff1
value: 31.195248222228507
- type: nauc_recall_at_100_max
value: 48.54839997778405
- type: nauc_recall_at_100_std
value: 30.326533549898492
- type: nauc_recall_at_10_diff1
value: 35.79622283233253
- type: nauc_recall_at_10_max
value: 43.6290597156324
- type: nauc_recall_at_10_std
value: 12.396685032720631
- type: nauc_recall_at_1_diff1
value: 51.955635644994494
- type: nauc_recall_at_1_max
value: 38.626441719712105
- type: nauc_recall_at_1_std
value: -7.853806336363088
- type: nauc_recall_at_20_diff1
value: 34.664562887859994
- type: nauc_recall_at_20_max
value: 48.48107873635332
- type: nauc_recall_at_20_std
value: 21.018166338397783
- type: nauc_recall_at_3_diff1
value: 38.84225790574717
- type: nauc_recall_at_3_max
value: 39.698141314194885
- type: nauc_recall_at_3_std
value: 1.2468146990639082
- type: nauc_recall_at_5_diff1
value: 39.41667868602902
- type: nauc_recall_at_5_max
value: 42.99091761218747
- type: nauc_recall_at_5_std
value: 4.1892902250914
- type: ndcg_at_1
value: 34.589
- type: ndcg_at_10
value: 46.049
- type: ndcg_at_100
value: 51.536
- type: ndcg_at_1000
value: 53.529
- type: ndcg_at_20
value: 48.4
- type: ndcg_at_3
value: 40.35
- type: ndcg_at_5
value: 42.760999999999996
- type: precision_at_1
value: 34.589
- type: precision_at_10
value: 8.539
- type: precision_at_100
value: 1.304
- type: precision_at_1000
value: 0.166
- type: precision_at_20
value: 5
- type: precision_at_3
value: 19.559
- type: precision_at_5
value: 13.950000000000001
- type: recall_at_1
value: 28.131
- type: recall_at_10
value: 59.939
- type: recall_at_100
value: 82.627
- type: recall_at_1000
value: 95.45
- type: recall_at_20
value: 68.369
- type: recall_at_3
value: 43.772
- type: recall_at_5
value: 50.163999999999994
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackRetrieval (default)
revision: CQADupstackRetrieval_is_a_combined_dataset
split: test
type: CQADupstackRetrieval_is_a_combined_dataset
metrics:
- type: main_score
value: 44.13575
- type: ndcg_at_10
value: 44.13575
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackStatsRetrieval (default)
revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a
split: test
type: mteb/cqadupstack-stats
metrics:
- type: main_score
value: 38.804
- type: map_at_1
value: 25.962000000000003
- type: map_at_10
value: 34.043
- type: map_at_100
value: 35.057
- type: map_at_1000
value: 35.154999999999994
- type: map_at_20
value: 34.569
- type: map_at_3
value: 31.617
- type: map_at_5
value: 32.782
- type: mrr_at_1
value: 28.834355828220858
- type: mrr_at_10
value: 36.64895559450774
- type: mrr_at_100
value: 37.52109868527602
- type: mrr_at_1000
value: 37.58255846708029
- type: mrr_at_20
value: 37.1319073794014
- type: mrr_at_3
value: 34.483640081799614
- type: mrr_at_5
value: 35.4192229038855
- type: nauc_map_at_1000_diff1
value: 58.42950192992681
- type: nauc_map_at_1000_max
value: 39.57355161523077
- type: nauc_map_at_1000_std
value: -0.5220553270563432
- type: nauc_map_at_100_diff1
value: 58.42574558879428
- type: nauc_map_at_100_max
value: 39.52993415358903
- type: nauc_map_at_100_std
value: -0.5668597324244167
- type: nauc_map_at_10_diff1
value: 58.43165347626888
- type: nauc_map_at_10_max
value: 39.447117032745595
- type: nauc_map_at_10_std
value: -1.0373470721350924
- type: nauc_map_at_1_diff1
value: 65.73633577347968
- type: nauc_map_at_1_max
value: 39.0696949583001
- type: nauc_map_at_1_std
value: -5.304093973753286
- type: nauc_map_at_20_diff1
value: 58.39114669791427
- type: nauc_map_at_20_max
value: 39.41064134295085
- type: nauc_map_at_20_std
value: -0.7917416841854326
- type: nauc_map_at_3_diff1
value: 59.11590522226747
- type: nauc_map_at_3_max
value: 38.26475464013714
- type: nauc_map_at_3_std
value: -2.943733722500886
- type: nauc_map_at_5_diff1
value: 58.919769893574845
- type: nauc_map_at_5_max
value: 39.483910146456026
- type: nauc_map_at_5_std
value: -1.519504597509289
- type: nauc_mrr_at_1000_diff1
value: 58.37662835928848
- type: nauc_mrr_at_1000_max
value: 40.235639182367514
- type: nauc_mrr_at_1000_std
value: 0.6808602347707579
- type: nauc_mrr_at_100_diff1
value: 58.3737130577029
- type: nauc_mrr_at_100_max
value: 40.21394358949754
- type: nauc_mrr_at_100_std
value: 0.6650992295553855
- type: nauc_mrr_at_10_diff1
value: 58.28984061656766
- type: nauc_mrr_at_10_max
value: 40.27354401071113
- type: nauc_mrr_at_10_std
value: 0.3819749700769928
- type: nauc_mrr_at_1_diff1
value: 65.35744135482541
- type: nauc_mrr_at_1_max
value: 40.49685618535152
- type: nauc_mrr_at_1_std
value: -2.89692739010175
- type: nauc_mrr_at_20_diff1
value: 58.41116378626607
- type: nauc_mrr_at_20_max
value: 40.2097330177334
- type: nauc_mrr_at_20_std
value: 0.4883281594311455
- type: nauc_mrr_at_3_diff1
value: 58.32268537202345
- type: nauc_mrr_at_3_max
value: 39.60551750734993
- type: nauc_mrr_at_3_std
value: -0.5422035519181851
- type: nauc_mrr_at_5_diff1
value: 58.49794116135833
- type: nauc_mrr_at_5_max
value: 40.46516712824574
- type: nauc_mrr_at_5_std
value: 0.27677884851191253
- type: nauc_ndcg_at_1000_diff1
value: 56.02653149727346
- type: nauc_ndcg_at_1000_max
value: 40.7747963713926
- type: nauc_ndcg_at_1000_std
value: 3.4614519159124564
- type: nauc_ndcg_at_100_diff1
value: 56.179008089355754
- type: nauc_ndcg_at_100_max
value: 40.31351560345152
- type: nauc_ndcg_at_100_std
value: 3.0653796725120244
- type: nauc_ndcg_at_10_diff1
value: 55.79999674519117
- type: nauc_ndcg_at_10_max
value: 39.79284055565316
- type: nauc_ndcg_at_10_std
value: 0.8563610405924191
- type: nauc_ndcg_at_1_diff1
value: 65.35744135482541
- type: nauc_ndcg_at_1_max
value: 40.49685618535152
- type: nauc_ndcg_at_1_std
value: -2.89692739010175
- type: nauc_ndcg_at_20_diff1
value: 55.861797313495444
- type: nauc_ndcg_at_20_max
value: 39.73779003757593
- type: nauc_ndcg_at_20_std
value: 1.5726810381128054
- type: nauc_ndcg_at_3_diff1
value: 56.32691432224729
- type: nauc_ndcg_at_3_max
value: 38.299858217583335
- type: nauc_ndcg_at_3_std
value: -1.2531898063666884
- type: nauc_ndcg_at_5_diff1
value: 56.69644375971069
- type: nauc_ndcg_at_5_max
value: 40.06424776988713
- type: nauc_ndcg_at_5_std
value: 0.2901775965386671
- type: nauc_precision_at_1000_diff1
value: -3.877307233721265
- type: nauc_precision_at_1000_max
value: 10.127068189809519
- type: nauc_precision_at_1000_std
value: 16.96119800274081
- type: nauc_precision_at_100_diff1
value: 15.274802731381993
- type: nauc_precision_at_100_max
value: 24.062912766433968
- type: nauc_precision_at_100_std
value: 19.540545867214952
- type: nauc_precision_at_10_diff1
value: 33.233598860699345
- type: nauc_precision_at_10_max
value: 36.0057496175431
- type: nauc_precision_at_10_std
value: 13.825026781947852
- type: nauc_precision_at_1_diff1
value: 65.35744135482541
- type: nauc_precision_at_1_max
value: 40.49685618535152
- type: nauc_precision_at_1_std
value: -2.89692739010175
- type: nauc_precision_at_20_diff1
value: 29.307063075506235
- type: nauc_precision_at_20_max
value: 31.96503432704136
- type: nauc_precision_at_20_std
value: 14.778922344816682
- type: nauc_precision_at_3_diff1
value: 44.453460492200634
- type: nauc_precision_at_3_max
value: 36.14859388336832
- type: nauc_precision_at_3_std
value: 4.800212967579488
- type: nauc_precision_at_5_diff1
value: 40.22205190799202
- type: nauc_precision_at_5_max
value: 38.28741836288616
- type: nauc_precision_at_5_std
value: 10.202239026170842
- type: nauc_recall_at_1000_diff1
value: 27.802656406916533
- type: nauc_recall_at_1000_max
value: 48.91523333513017
- type: nauc_recall_at_1000_std
value: 45.37280285855605
- type: nauc_recall_at_100_diff1
value: 44.062936821497814
- type: nauc_recall_at_100_max
value: 39.1044271123917
- type: nauc_recall_at_100_std
value: 18.138636734517977
- type: nauc_recall_at_10_diff1
value: 46.496466286643226
- type: nauc_recall_at_10_max
value: 38.17775853832056
- type: nauc_recall_at_10_std
value: 4.3907344638907375
- type: nauc_recall_at_1_diff1
value: 65.73633577347968
- type: nauc_recall_at_1_max
value: 39.0696949583001
- type: nauc_recall_at_1_std
value: -5.304093973753286
- type: nauc_recall_at_20_diff1
value: 45.67627485306954
- type: nauc_recall_at_20_max
value: 37.696455752206795
- type: nauc_recall_at_20_std
value: 7.269773462867943
- type: nauc_recall_at_3_diff1
value: 50.10543215562817
- type: nauc_recall_at_3_max
value: 35.83938191040353
- type: nauc_recall_at_3_std
value: -1.607497533365206
- type: nauc_recall_at_5_diff1
value: 50.314565492560014
- type: nauc_recall_at_5_max
value: 39.46480829071993
- type: nauc_recall_at_5_std
value: 2.671269661710202
- type: ndcg_at_1
value: 28.834
- type: ndcg_at_10
value: 38.804
- type: ndcg_at_100
value: 43.832
- type: ndcg_at_1000
value: 46.115
- type: ndcg_at_20
value: 40.604
- type: ndcg_at_3
value: 34.292
- type: ndcg_at_5
value: 36.004000000000005
- type: precision_at_1
value: 28.834
- type: precision_at_10
value: 6.227
- type: precision_at_100
value: 0.9570000000000001
- type: precision_at_1000
value: 0.123
- type: precision_at_20
value: 3.581
- type: precision_at_3
value: 14.826
- type: precision_at_5
value: 10.123
- type: recall_at_1
value: 25.962000000000003
- type: recall_at_10
value: 50.723
- type: recall_at_100
value: 73.67099999999999
- type: recall_at_1000
value: 90.277
- type: recall_at_20
value: 57.504
- type: recall_at_3
value: 38.091
- type: recall_at_5
value: 42.345
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackTexRetrieval (default)
revision: 46989137a86843e03a6195de44b09deda022eec7
split: test
type: mteb/cqadupstack-tex
metrics:
- type: main_score
value: 32.388
- type: map_at_1
value: 19.933999999999997
- type: map_at_10
value: 27.544999999999998
- type: map_at_100
value: 28.666000000000004
- type: map_at_1000
value: 28.788999999999998
- type: map_at_20
value: 28.132
- type: map_at_3
value: 25.129
- type: map_at_5
value: 26.484
- type: mrr_at_1
value: 23.640743289745355
- type: mrr_at_10
value: 31.322100416216113
- type: mrr_at_100
value: 32.24068530425795
- type: mrr_at_1000
value: 32.30753289159229
- type: mrr_at_20
value: 31.819007382675256
- type: mrr_at_3
value: 29.043358568479068
- type: mrr_at_5
value: 30.38024776324855
- type: nauc_map_at_1000_diff1
value: 39.432095405114644
- type: nauc_map_at_1000_max
value: 32.250472208788125
- type: nauc_map_at_1000_std
value: 1.3765499216188093
- type: nauc_map_at_100_diff1
value: 39.42983559790836
- type: nauc_map_at_100_max
value: 32.244292042207256
- type: nauc_map_at_100_std
value: 1.3793997130642306
- type: nauc_map_at_10_diff1
value: 39.70729732520478
- type: nauc_map_at_10_max
value: 31.966755036344125
- type: nauc_map_at_10_std
value: 0.7385295090218716
- type: nauc_map_at_1_diff1
value: 44.495430151054336
- type: nauc_map_at_1_max
value: 29.982114439794643
- type: nauc_map_at_1_std
value: -1.3447506401404055
- type: nauc_map_at_20_diff1
value: 39.49298909549211
- type: nauc_map_at_20_max
value: 32.1695194426593
- type: nauc_map_at_20_std
value: 1.0902148211264926
- type: nauc_map_at_3_diff1
value: 40.53389807029239
- type: nauc_map_at_3_max
value: 31.715356191118403
- type: nauc_map_at_3_std
value: 0.28056703408869377
- type: nauc_map_at_5_diff1
value: 40.14442565222247
- type: nauc_map_at_5_max
value: 32.00739475392889
- type: nauc_map_at_5_std
value: 0.4955596992864427
- type: nauc_mrr_at_1000_diff1
value: 40.009061081268925
- type: nauc_mrr_at_1000_max
value: 33.86591429721166
- type: nauc_mrr_at_1000_std
value: 1.7077448741657502
- type: nauc_mrr_at_100_diff1
value: 39.997390874284235
- type: nauc_mrr_at_100_max
value: 33.86945351838866
- type: nauc_mrr_at_100_std
value: 1.71696314581838
- type: nauc_mrr_at_10_diff1
value: 40.12509379811874
- type: nauc_mrr_at_10_max
value: 33.76283934044446
- type: nauc_mrr_at_10_std
value: 1.2430566264956315
- type: nauc_mrr_at_1_diff1
value: 44.88110341617864
- type: nauc_mrr_at_1_max
value: 32.906287852960105
- type: nauc_mrr_at_1_std
value: -1.0513930999916752
- type: nauc_mrr_at_20_diff1
value: 39.99671065649182
- type: nauc_mrr_at_20_max
value: 33.89080116879488
- type: nauc_mrr_at_20_std
value: 1.5802790695054365
- type: nauc_mrr_at_3_diff1
value: 41.027093163682906
- type: nauc_mrr_at_3_max
value: 34.039923001219144
- type: nauc_mrr_at_3_std
value: 0.8961188704027852
- type: nauc_mrr_at_5_diff1
value: 40.54230047257113
- type: nauc_mrr_at_5_max
value: 33.9871480791131
- type: nauc_mrr_at_5_std
value: 1.07622865744921
- type: nauc_ndcg_at_1000_diff1
value: 37.02626579816157
- type: nauc_ndcg_at_1000_max
value: 33.20569609845541
- type: nauc_ndcg_at_1000_std
value: 4.5171709831731635
- type: nauc_ndcg_at_100_diff1
value: 36.93759649232734
- type: nauc_ndcg_at_100_max
value: 33.35556871630857
- type: nauc_ndcg_at_100_std
value: 4.992101563679995
- type: nauc_ndcg_at_10_diff1
value: 38.000896296292694
- type: nauc_ndcg_at_10_max
value: 32.743050430383946
- type: nauc_ndcg_at_10_std
value: 1.9987517822933054
- type: nauc_ndcg_at_1_diff1
value: 44.88110341617864
- type: nauc_ndcg_at_1_max
value: 32.906287852960105
- type: nauc_ndcg_at_1_std
value: -1.0513930999916752
- type: nauc_ndcg_at_20_diff1
value: 37.352476202026196
- type: nauc_ndcg_at_20_max
value: 33.20013220157046
- type: nauc_ndcg_at_20_std
value: 3.197292569536845
- type: nauc_ndcg_at_3_diff1
value: 39.61034850071899
- type: nauc_ndcg_at_3_max
value: 32.9343480599293
- type: nauc_ndcg_at_3_std
value: 0.9875909156053545
- type: nauc_ndcg_at_5_diff1
value: 38.96326291598868
- type: nauc_ndcg_at_5_max
value: 33.04999857261579
- type: nauc_ndcg_at_5_std
value: 1.3562104035097535
- type: nauc_precision_at_1000_diff1
value: 0.8579892758292412
- type: nauc_precision_at_1000_max
value: 11.535374503730141
- type: nauc_precision_at_1000_std
value: 4.220107922006484
- type: nauc_precision_at_100_diff1
value: 10.701201287064533
- type: nauc_precision_at_100_max
value: 23.655145916975318
- type: nauc_precision_at_100_std
value: 11.987167226091426
- type: nauc_precision_at_10_diff1
value: 25.33168949376103
- type: nauc_precision_at_10_max
value: 31.85413101475395
- type: nauc_precision_at_10_std
value: 5.499203751576525
- type: nauc_precision_at_1_diff1
value: 44.88110341617864
- type: nauc_precision_at_1_max
value: 32.906287852960105
- type: nauc_precision_at_1_std
value: -1.0513930999916752
- type: nauc_precision_at_20_diff1
value: 20.281798578465178
- type: nauc_precision_at_20_max
value: 31.04004571929362
- type: nauc_precision_at_20_std
value: 8.18652916257154
- type: nauc_precision_at_3_diff1
value: 34.26701299048454
- type: nauc_precision_at_3_max
value: 35.941328871900105
- type: nauc_precision_at_3_std
value: 3.4997607852603614
- type: nauc_precision_at_5_diff1
value: 31.538017382058047
- type: nauc_precision_at_5_max
value: 35.08110227426111
- type: nauc_precision_at_5_std
value: 3.6887401822002412
- type: nauc_recall_at_1000_diff1
value: 15.044356004647014
- type: nauc_recall_at_1000_max
value: 30.822801246039845
- type: nauc_recall_at_1000_std
value: 28.946329988909845
- type: nauc_recall_at_100_diff1
value: 23.699532057756603
- type: nauc_recall_at_100_max
value: 32.27913363817225
- type: nauc_recall_at_100_std
value: 20.45848633440386
- type: nauc_recall_at_10_diff1
value: 31.17356412314004
- type: nauc_recall_at_10_max
value: 30.994576113766264
- type: nauc_recall_at_10_std
value: 4.62856199972417
- type: nauc_recall_at_1_diff1
value: 44.495430151054336
- type: nauc_recall_at_1_max
value: 29.982114439794643
- type: nauc_recall_at_1_std
value: -1.3447506401404055
- type: nauc_recall_at_20_diff1
value: 28.304017932372155
- type: nauc_recall_at_20_max
value: 31.856647031696383
- type: nauc_recall_at_20_std
value: 8.817413938730361
- type: nauc_recall_at_3_diff1
value: 35.95017861750422
- type: nauc_recall_at_3_max
value: 31.641984309143332
- type: nauc_recall_at_3_std
value: 2.3704976555263086
- type: nauc_recall_at_5_diff1
value: 34.063880393836534
- type: nauc_recall_at_5_max
value: 32.00850680189186
- type: nauc_recall_at_5_std
value: 2.9328158047750894
- type: ndcg_at_1
value: 23.641000000000002
- type: ndcg_at_10
value: 32.388
- type: ndcg_at_100
value: 37.749
- type: ndcg_at_1000
value: 40.537
- type: ndcg_at_20
value: 34.28
- type: ndcg_at_3
value: 28.058
- type: ndcg_at_5
value: 30.113
- type: precision_at_1
value: 23.641000000000002
- type: precision_at_10
value: 5.811999999999999
- type: precision_at_100
value: 0.9939999999999999
- type: precision_at_1000
value: 0.14200000000000002
- type: precision_at_20
value: 3.472
- type: precision_at_3
value: 13.111
- type: precision_at_5
value: 9.456000000000001
- type: recall_at_1
value: 19.933999999999997
- type: recall_at_10
value: 43.078
- type: recall_at_100
value: 67.242
- type: recall_at_1000
value: 86.973
- type: recall_at_20
value: 50.031000000000006
- type: recall_at_3
value: 31.062
- type: recall_at_5
value: 36.315999999999995
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackUnixRetrieval (default)
revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53
split: test
type: mteb/cqadupstack-unix
metrics:
- type: main_score
value: 44.722
- type: map_at_1
value: 29.072
- type: map_at_10
value: 39.25
- type: map_at_100
value: 40.426
- type: map_at_1000
value: 40.528999999999996
- type: map_at_20
value: 39.93
- type: map_at_3
value: 36.132
- type: map_at_5
value: 37.985
- type: mrr_at_1
value: 34.048507462686565
- type: mrr_at_10
value: 43.14698827292108
- type: mrr_at_100
value: 44.051192277034154
- type: mrr_at_1000
value: 44.107508349556845
- type: mrr_at_20
value: 43.737780068879694
- type: mrr_at_3
value: 40.593905472636806
- type: mrr_at_5
value: 42.20771144278602
- type: nauc_map_at_1000_diff1
value: 50.760419060905726
- type: nauc_map_at_1000_max
value: 49.17795203592375
- type: nauc_map_at_1000_std
value: 3.030817021699703
- type: nauc_map_at_100_diff1
value: 50.74246353752929
- type: nauc_map_at_100_max
value: 49.15485358641109
- type: nauc_map_at_100_std
value: 3.0086923977616165
- type: nauc_map_at_10_diff1
value: 50.73866025493935
- type: nauc_map_at_10_max
value: 48.9731969924837
- type: nauc_map_at_10_std
value: 2.7369727690804013
- type: nauc_map_at_1_diff1
value: 57.861789449807
- type: nauc_map_at_1_max
value: 50.609587661619436
- type: nauc_map_at_1_std
value: 0.11047312188031311
- type: nauc_map_at_20_diff1
value: 50.76511835975127
- type: nauc_map_at_20_max
value: 49.05141199070726
- type: nauc_map_at_20_std
value: 2.8183996550947534
- type: nauc_map_at_3_diff1
value: 52.35396925813111
- type: nauc_map_at_3_max
value: 48.731880168091394
- type: nauc_map_at_3_std
value: 1.5608406018263319
- type: nauc_map_at_5_diff1
value: 51.37723141537462
- type: nauc_map_at_5_max
value: 49.128536247887304
- type: nauc_map_at_5_std
value: 2.3100100968692416
- type: nauc_mrr_at_1000_diff1
value: 49.38317342582358
- type: nauc_mrr_at_1000_max
value: 48.86642358781332
- type: nauc_mrr_at_1000_std
value: 3.085969399146775
- type: nauc_mrr_at_100_diff1
value: 49.36913857935236
- type: nauc_mrr_at_100_max
value: 48.852276658001756
- type: nauc_mrr_at_100_std
value: 3.0924076269507066
- type: nauc_mrr_at_10_diff1
value: 49.2673387296456
- type: nauc_mrr_at_10_max
value: 48.73871789141773
- type: nauc_mrr_at_10_std
value: 2.8634537217769824
- type: nauc_mrr_at_1_diff1
value: 55.46027934372595
- type: nauc_mrr_at_1_max
value: 51.22116133200325
- type: nauc_mrr_at_1_std
value: 0.3150103421046114
- type: nauc_mrr_at_20_diff1
value: 49.29994747074829
- type: nauc_mrr_at_20_max
value: 48.84435864143889
- type: nauc_mrr_at_20_std
value: 3.0317480264943226
- type: nauc_mrr_at_3_diff1
value: 50.32456938936314
- type: nauc_mrr_at_3_max
value: 49.019560128653985
- type: nauc_mrr_at_3_std
value: 2.227284100045724
- type: nauc_mrr_at_5_diff1
value: 49.46346874023712
- type: nauc_mrr_at_5_max
value: 49.08809887285519
- type: nauc_mrr_at_5_std
value: 2.8688902354832124
- type: nauc_ndcg_at_1000_diff1
value: 48.42878157444301
- type: nauc_ndcg_at_1000_max
value: 49.08844432902424
- type: nauc_ndcg_at_1000_std
value: 5.380167289943043
- type: nauc_ndcg_at_100_diff1
value: 47.797582200418134
- type: nauc_ndcg_at_100_max
value: 48.596601939337454
- type: nauc_ndcg_at_100_std
value: 5.492043017969421
- type: nauc_ndcg_at_10_diff1
value: 47.625157750972804
- type: nauc_ndcg_at_10_max
value: 48.085974210688136
- type: nauc_ndcg_at_10_std
value: 3.9660254846310847
- type: nauc_ndcg_at_1_diff1
value: 55.46027934372595
- type: nauc_ndcg_at_1_max
value: 51.22116133200325
- type: nauc_ndcg_at_1_std
value: 0.3150103421046114
- type: nauc_ndcg_at_20_diff1
value: 47.637502104264385
- type: nauc_ndcg_at_20_max
value: 48.30145643136742
- type: nauc_ndcg_at_20_std
value: 4.505873937286856
- type: nauc_ndcg_at_3_diff1
value: 49.87044033257167
- type: nauc_ndcg_at_3_max
value: 48.08524541077855
- type: nauc_ndcg_at_3_std
value: 2.3430570236690613
- type: nauc_ndcg_at_5_diff1
value: 48.65339964003644
- type: nauc_ndcg_at_5_max
value: 48.60373734139397
- type: nauc_ndcg_at_5_std
value: 3.4063158959996382
- type: nauc_precision_at_1000_diff1
value: -10.775942551429775
- type: nauc_precision_at_1000_max
value: -1.7495021980399774
- type: nauc_precision_at_1000_std
value: 7.379058923086573
- type: nauc_precision_at_100_diff1
value: 0.37649047395049773
- type: nauc_precision_at_100_max
value: 13.820306869942492
- type: nauc_precision_at_100_std
value: 11.642351647126524
- type: nauc_precision_at_10_diff1
value: 18.216485349603282
- type: nauc_precision_at_10_max
value: 31.933237189917435
- type: nauc_precision_at_10_std
value: 8.389616933577662
- type: nauc_precision_at_1_diff1
value: 55.46027934372595
- type: nauc_precision_at_1_max
value: 51.22116133200325
- type: nauc_precision_at_1_std
value: 0.3150103421046114
- type: nauc_precision_at_20_diff1
value: 12.750940909763035
- type: nauc_precision_at_20_max
value: 27.1840822623236
- type: nauc_precision_at_20_std
value: 9.195054018005923
- type: nauc_precision_at_3_diff1
value: 36.379888969052914
- type: nauc_precision_at_3_max
value: 42.004013659004194
- type: nauc_precision_at_3_std
value: 4.579532665542333
- type: nauc_precision_at_5_diff1
value: 28.236832614984774
- type: nauc_precision_at_5_max
value: 39.10171500376415
- type: nauc_precision_at_5_std
value: 7.162006905424074
- type: nauc_recall_at_1000_diff1
value: 29.310425551550885
- type: nauc_recall_at_1000_max
value: 51.73371986372531
- type: nauc_recall_at_1000_std
value: 43.85287244908794
- type: nauc_recall_at_100_diff1
value: 33.105037284921565
- type: nauc_recall_at_100_max
value: 42.124001633147316
- type: nauc_recall_at_100_std
value: 19.4129898533571
- type: nauc_recall_at_10_diff1
value: 37.62933078849475
- type: nauc_recall_at_10_max
value: 42.79759235560673
- type: nauc_recall_at_10_std
value: 7.091258015370007
- type: nauc_recall_at_1_diff1
value: 57.861789449807
- type: nauc_recall_at_1_max
value: 50.609587661619436
- type: nauc_recall_at_1_std
value: 0.11047312188031311
- type: nauc_recall_at_20_diff1
value: 35.96890042211954
- type: nauc_recall_at_20_max
value: 42.335161622023385
- type: nauc_recall_at_20_std
value: 9.870452156772494
- type: nauc_recall_at_3_diff1
value: 45.075408938241324
- type: nauc_recall_at_3_max
value: 44.6546256631834
- type: nauc_recall_at_3_std
value: 2.763616443398486
- type: nauc_recall_at_5_diff1
value: 40.91464426027458
- type: nauc_recall_at_5_max
value: 45.07693534575636
- type: nauc_recall_at_5_std
value: 5.746035089435559
- type: ndcg_at_1
value: 34.049
- type: ndcg_at_10
value: 44.722
- type: ndcg_at_100
value: 49.864999999999995
- type: ndcg_at_1000
value: 52.071
- type: ndcg_at_20
value: 46.89
- type: ndcg_at_3
value: 39.434999999999995
- type: ndcg_at_5
value: 42.221
- type: precision_at_1
value: 34.049
- type: precision_at_10
value: 7.509
- type: precision_at_100
value: 1.118
- type: precision_at_1000
value: 0.14200000000000002
- type: precision_at_20
value: 4.352
- type: precision_at_3
value: 17.755000000000003
- type: precision_at_5
value: 12.668
- type: recall_at_1
value: 29.072
- type: recall_at_10
value: 57.251
- type: recall_at_100
value: 79.237
- type: recall_at_1000
value: 94.452
- type: recall_at_20
value: 64.976
- type: recall_at_3
value: 43.213
- type: recall_at_5
value: 50.169
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackWebmastersRetrieval (default)
revision: 160c094312a0e1facb97e55eeddb698c0abe3571
split: test
type: mteb/cqadupstack-webmasters
metrics:
- type: main_score
value: 43.692
- type: map_at_1
value: 28.375
- type: map_at_10
value: 38.009
- type: map_at_100
value: 39.724
- type: map_at_1000
value: 39.943
- type: map_at_20
value: 38.934000000000005
- type: map_at_3
value: 35.115
- type: map_at_5
value: 36.906
- type: mrr_at_1
value: 33.99209486166008
- type: mrr_at_10
value: 42.59418721375244
- type: mrr_at_100
value: 43.76246716346725
- type: mrr_at_1000
value: 43.800667883042095
- type: mrr_at_20
value: 43.364410286981844
- type: mrr_at_3
value: 40.250329380764185
- type: mrr_at_5
value: 41.65349143610015
- type: nauc_map_at_1000_diff1
value: 40.323280775311765
- type: nauc_map_at_1000_max
value: 39.58356051942929
- type: nauc_map_at_1000_std
value: 5.811505855792315
- type: nauc_map_at_100_diff1
value: 40.10834716642787
- type: nauc_map_at_100_max
value: 39.71909426675674
- type: nauc_map_at_100_std
value: 5.65192320670045
- type: nauc_map_at_10_diff1
value: 40.07680297377284
- type: nauc_map_at_10_max
value: 39.325423037685546
- type: nauc_map_at_10_std
value: 3.5550102422533576
- type: nauc_map_at_1_diff1
value: 46.78581974845936
- type: nauc_map_at_1_max
value: 38.76797022736912
- type: nauc_map_at_1_std
value: 1.3866055955234646
- type: nauc_map_at_20_diff1
value: 39.94515722154877
- type: nauc_map_at_20_max
value: 39.41157329525085
- type: nauc_map_at_20_std
value: 4.71995579048337
- type: nauc_map_at_3_diff1
value: 40.687289860813365
- type: nauc_map_at_3_max
value: 38.4852492560273
- type: nauc_map_at_3_std
value: 2.271640173400609
- type: nauc_map_at_5_diff1
value: 40.467336449417765
- type: nauc_map_at_5_max
value: 39.50649331907832
- type: nauc_map_at_5_std
value: 3.1735830189375736
- type: nauc_mrr_at_1000_diff1
value: 39.9936083347999
- type: nauc_mrr_at_1000_max
value: 38.8655305231559
- type: nauc_mrr_at_1000_std
value: 7.909840985525943
- type: nauc_mrr_at_100_diff1
value: 39.972318696309365
- type: nauc_mrr_at_100_max
value: 38.859087892031596
- type: nauc_mrr_at_100_std
value: 7.947613190121554
- type: nauc_mrr_at_10_diff1
value: 40.156868549352914
- type: nauc_mrr_at_10_max
value: 38.83763932557654
- type: nauc_mrr_at_10_std
value: 7.339915764916966
- type: nauc_mrr_at_1_diff1
value: 44.97693907176195
- type: nauc_mrr_at_1_max
value: 40.03919408146429
- type: nauc_mrr_at_1_std
value: 7.755266269078702
- type: nauc_mrr_at_20_diff1
value: 39.94043217345319
- type: nauc_mrr_at_20_max
value: 38.74720673612248
- type: nauc_mrr_at_20_std
value: 7.760572250915075
- type: nauc_mrr_at_3_diff1
value: 39.96111544338325
- type: nauc_mrr_at_3_max
value: 38.46421243593102
- type: nauc_mrr_at_3_std
value: 6.528611487141296
- type: nauc_mrr_at_5_diff1
value: 40.174967051016736
- type: nauc_mrr_at_5_max
value: 39.035593985981684
- type: nauc_mrr_at_5_std
value: 6.971950800211483
- type: nauc_ndcg_at_1000_diff1
value: 39.016898984942536
- type: nauc_ndcg_at_1000_max
value: 40.39491805572175
- type: nauc_ndcg_at_1000_std
value: 10.031391094809482
- type: nauc_ndcg_at_100_diff1
value: 38.32985292433329
- type: nauc_ndcg_at_100_max
value: 40.29019091167566
- type: nauc_ndcg_at_100_std
value: 10.460340040258803
- type: nauc_ndcg_at_10_diff1
value: 38.68317711882466
- type: nauc_ndcg_at_10_max
value: 38.59408942538687
- type: nauc_ndcg_at_10_std
value: 6.2333377757728625
- type: nauc_ndcg_at_1_diff1
value: 44.97693907176195
- type: nauc_ndcg_at_1_max
value: 40.03919408146429
- type: nauc_ndcg_at_1_std
value: 7.755266269078702
- type: nauc_ndcg_at_20_diff1
value: 38.151286655877044
- type: nauc_ndcg_at_20_max
value: 38.58195710047157
- type: nauc_ndcg_at_20_std
value: 8.127010352974342
- type: nauc_ndcg_at_3_diff1
value: 38.8797705249473
- type: nauc_ndcg_at_3_max
value: 37.93871443881654
- type: nauc_ndcg_at_3_std
value: 5.199744299613702
- type: nauc_ndcg_at_5_diff1
value: 39.30695488055343
- type: nauc_ndcg_at_5_max
value: 39.24390830410231
- type: nauc_ndcg_at_5_std
value: 6.393246441405309
- type: nauc_precision_at_1000_diff1
value: 14.399944609637044
- type: nauc_precision_at_1000_max
value: -10.17099485672005
- type: nauc_precision_at_1000_std
value: 20.66644829842888
- type: nauc_precision_at_100_diff1
value: 9.069079609228908
- type: nauc_precision_at_100_max
value: 4.065124737687779
- type: nauc_precision_at_100_std
value: 31.107985656647557
- type: nauc_precision_at_10_diff1
value: 17.000945636231037
- type: nauc_precision_at_10_max
value: 25.026721592124844
- type: nauc_precision_at_10_std
value: 20.07498948143639
- type: nauc_precision_at_1_diff1
value: 44.97693907176195
- type: nauc_precision_at_1_max
value: 40.03919408146429
- type: nauc_precision_at_1_std
value: 7.755266269078702
- type: nauc_precision_at_20_diff1
value: 11.47018887895978
- type: nauc_precision_at_20_max
value: 18.037165101696996
- type: nauc_precision_at_20_std
value: 29.730108768934087
- type: nauc_precision_at_3_diff1
value: 23.464000465007697
- type: nauc_precision_at_3_max
value: 31.99363298692347
- type: nauc_precision_at_3_std
value: 11.8643893735375
- type: nauc_precision_at_5_diff1
value: 20.110906670076435
- type: nauc_precision_at_5_max
value: 31.088178850932525
- type: nauc_precision_at_5_std
value: 16.73038318183611
- type: nauc_recall_at_1000_diff1
value: 35.652592229504734
- type: nauc_recall_at_1000_max
value: 62.73306941632778
- type: nauc_recall_at_1000_std
value: 59.493061172332794
- type: nauc_recall_at_100_diff1
value: 27.07609325471521
- type: nauc_recall_at_100_max
value: 42.62351469487428
- type: nauc_recall_at_100_std
value: 35.54812980928898
- type: nauc_recall_at_10_diff1
value: 31.01066630575034
- type: nauc_recall_at_10_max
value: 35.4692240854422
- type: nauc_recall_at_10_std
value: 6.02167165896485
- type: nauc_recall_at_1_diff1
value: 46.78581974845936
- type: nauc_recall_at_1_max
value: 38.76797022736912
- type: nauc_recall_at_1_std
value: 1.3866055955234646
- type: nauc_recall_at_20_diff1
value: 28.573133193243194
- type: nauc_recall_at_20_max
value: 33.59892465323569
- type: nauc_recall_at_20_std
value: 13.688824462521056
- type: nauc_recall_at_3_diff1
value: 33.67501521174446
- type: nauc_recall_at_3_max
value: 34.86523196412695
- type: nauc_recall_at_3_std
value: 1.3361875812123045
- type: nauc_recall_at_5_diff1
value: 33.4610375265344
- type: nauc_recall_at_5_max
value: 37.36574305043109
- type: nauc_recall_at_5_std
value: 3.7170099554233906
- type: ndcg_at_1
value: 33.992
- type: ndcg_at_10
value: 43.692
- type: ndcg_at_100
value: 50.139
- type: ndcg_at_1000
value: 52.112
- type: ndcg_at_20
value: 46.312
- type: ndcg_at_3
value: 39.495999999999995
- type: ndcg_at_5
value: 41.794
- type: precision_at_1
value: 33.992
- type: precision_at_10
value: 7.964
- type: precision_at_100
value: 1.617
- type: precision_at_1000
value: 0.245
- type: precision_at_20
value: 5.119
- type: precision_at_3
value: 18.445
- type: precision_at_5
value: 13.281
- type: recall_at_1
value: 28.375
- type: recall_at_10
value: 54.242999999999995
- type: recall_at_100
value: 82.636
- type: recall_at_1000
value: 94.22399999999999
- type: recall_at_20
value: 63.993
- type: recall_at_3
value: 42.074
- type: recall_at_5
value: 48.172
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackWordpressRetrieval (default)
revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
split: test
type: mteb/cqadupstack-wordpress
metrics:
- type: main_score
value: 35.977
- type: map_at_1
value: 23.333000000000002
- type: map_at_10
value: 31.046000000000003
- type: map_at_100
value: 32.055
- type: map_at_1000
value: 32.147
- type: map_at_20
value: 31.556
- type: map_at_3
value: 28.315
- type: map_at_5
value: 29.881
- type: mrr_at_1
value: 25.693160813308687
- type: mrr_at_10
value: 33.43257635771498
- type: mrr_at_100
value: 34.31548367605493
- type: mrr_at_1000
value: 34.37801297570286
- type: mrr_at_20
value: 33.86328992936981
- type: mrr_at_3
value: 30.745532963647566
- type: mrr_at_5
value: 32.335181762168816
- type: nauc_map_at_1000_diff1
value: 45.22718795765733
- type: nauc_map_at_1000_max
value: 36.57896344808718
- type: nauc_map_at_1000_std
value: -0.8709187735719646
- type: nauc_map_at_100_diff1
value: 45.203815992445854
- type: nauc_map_at_100_max
value: 36.56424208682807
- type: nauc_map_at_100_std
value: -0.895717972975496
- type: nauc_map_at_10_diff1
value: 45.55223833872233
- type: nauc_map_at_10_max
value: 36.72055103852916
- type: nauc_map_at_10_std
value: -1.4164435707064298
- type: nauc_map_at_1_diff1
value: 51.728825815393385
- type: nauc_map_at_1_max
value: 36.485675960281014
- type: nauc_map_at_1_std
value: -5.6156977207038485
- type: nauc_map_at_20_diff1
value: 45.32240910588799
- type: nauc_map_at_20_max
value: 36.73322607493656
- type: nauc_map_at_20_std
value: -0.9822167145344648
- type: nauc_map_at_3_diff1
value: 46.07401924963852
- type: nauc_map_at_3_max
value: 37.22001398614413
- type: nauc_map_at_3_std
value: -1.8496830553484993
- type: nauc_map_at_5_diff1
value: 45.494860339788026
- type: nauc_map_at_5_max
value: 36.26443248483769
- type: nauc_map_at_5_std
value: -1.7434057463650163
- type: nauc_mrr_at_1000_diff1
value: 46.21253943806437
- type: nauc_mrr_at_1000_max
value: 38.177432777205354
- type: nauc_mrr_at_1000_std
value: 0.7768638734725042
- type: nauc_mrr_at_100_diff1
value: 46.18580958749434
- type: nauc_mrr_at_100_max
value: 38.16083552209263
- type: nauc_mrr_at_100_std
value: 0.7694820893666625
- type: nauc_mrr_at_10_diff1
value: 46.4031003307697
- type: nauc_mrr_at_10_max
value: 38.331262989485204
- type: nauc_mrr_at_10_std
value: 0.516521121582369
- type: nauc_mrr_at_1_diff1
value: 52.54099054051612
- type: nauc_mrr_at_1_max
value: 38.64739617138923
- type: nauc_mrr_at_1_std
value: -3.732075482696759
- type: nauc_mrr_at_20_diff1
value: 46.25651789878755
- type: nauc_mrr_at_20_max
value: 38.37223683042208
- type: nauc_mrr_at_20_std
value: 0.8628381973898458
- type: nauc_mrr_at_3_diff1
value: 47.37035802829552
- type: nauc_mrr_at_3_max
value: 39.39203133385594
- type: nauc_mrr_at_3_std
value: -0.10049275141855617
- type: nauc_mrr_at_5_diff1
value: 46.38495559785434
- type: nauc_mrr_at_5_max
value: 38.029438017699604
- type: nauc_mrr_at_5_std
value: 0.033144589066499454
- type: nauc_ndcg_at_1000_diff1
value: 42.920860460202505
- type: nauc_ndcg_at_1000_max
value: 36.54736833868591
- type: nauc_ndcg_at_1000_std
value: 2.8313226967365432
- type: nauc_ndcg_at_100_diff1
value: 42.53476345813231
- type: nauc_ndcg_at_100_max
value: 35.93748082089683
- type: nauc_ndcg_at_100_std
value: 2.4176078459557475
- type: nauc_ndcg_at_10_diff1
value: 43.594795947394424
- type: nauc_ndcg_at_10_max
value: 36.695763579362655
- type: nauc_ndcg_at_10_std
value: 0.5106465386911283
- type: nauc_ndcg_at_1_diff1
value: 52.54099054051612
- type: nauc_ndcg_at_1_max
value: 38.64739617138923
- type: nauc_ndcg_at_1_std
value: -3.732075482696759
- type: nauc_ndcg_at_20_diff1
value: 42.954335807831306
- type: nauc_ndcg_at_20_max
value: 36.84558505388929
- type: nauc_ndcg_at_20_std
value: 2.041288642339101
- type: nauc_ndcg_at_3_diff1
value: 44.72319676900573
- type: nauc_ndcg_at_3_max
value: 38.16973225290572
- type: nauc_ndcg_at_3_std
value: -0.22968052359750002
- type: nauc_ndcg_at_5_diff1
value: 43.61533970987853
- type: nauc_ndcg_at_5_max
value: 36.00954769401293
- type: nauc_ndcg_at_5_std
value: -0.26032000690194024
- type: nauc_precision_at_1000_diff1
value: 0.7877763084973665
- type: nauc_precision_at_1000_max
value: 2.1801591614770293
- type: nauc_precision_at_1000_std
value: 14.214797840652245
- type: nauc_precision_at_100_diff1
value: 13.404592217945185
- type: nauc_precision_at_100_max
value: 22.958703884232143
- type: nauc_precision_at_100_std
value: 20.903308284925252
- type: nauc_precision_at_10_diff1
value: 31.74143406888776
- type: nauc_precision_at_10_max
value: 36.03995765213036
- type: nauc_precision_at_10_std
value: 9.41801821108275
- type: nauc_precision_at_1_diff1
value: 52.54099054051612
- type: nauc_precision_at_1_max
value: 38.64739617138923
- type: nauc_precision_at_1_std
value: -3.732075482696759
- type: nauc_precision_at_20_diff1
value: 26.617590644330164
- type: nauc_precision_at_20_max
value: 33.96374986677854
- type: nauc_precision_at_20_std
value: 15.46843694127063
- type: nauc_precision_at_3_diff1
value: 38.99506563253017
- type: nauc_precision_at_3_max
value: 39.80657156665246
- type: nauc_precision_at_3_std
value: 5.079287300413413
- type: nauc_precision_at_5_diff1
value: 34.90172497873929
- type: nauc_precision_at_5_max
value: 35.45469625301615
- type: nauc_precision_at_5_std
value: 6.480240051515346
- type: nauc_recall_at_1000_diff1
value: 16.070135288090835
- type: nauc_recall_at_1000_max
value: 32.03320279140135
- type: nauc_recall_at_1000_std
value: 38.785277645406524
- type: nauc_recall_at_100_diff1
value: 28.319781992014764
- type: nauc_recall_at_100_max
value: 26.75864253988515
- type: nauc_recall_at_100_std
value: 13.618232233364505
- type: nauc_recall_at_10_diff1
value: 36.29904326412429
- type: nauc_recall_at_10_max
value: 33.69300085607644
- type: nauc_recall_at_10_std
value: 3.697003202346052
- type: nauc_recall_at_1_diff1
value: 51.728825815393385
- type: nauc_recall_at_1_max
value: 36.485675960281014
- type: nauc_recall_at_1_std
value: -5.6156977207038485
- type: nauc_recall_at_20_diff1
value: 33.59302174152832
- type: nauc_recall_at_20_max
value: 34.063179006998254
- type: nauc_recall_at_20_std
value: 9.528686105553895
- type: nauc_recall_at_3_diff1
value: 39.628751291828415
- type: nauc_recall_at_3_max
value: 37.67952240312479
- type: nauc_recall_at_3_std
value: 1.6161823764259273
- type: nauc_recall_at_5_diff1
value: 36.68190217441154
- type: nauc_recall_at_5_max
value: 32.37947603111437
- type: nauc_recall_at_5_std
value: 1.6678770596515002
- type: ndcg_at_1
value: 25.692999999999998
- type: ndcg_at_10
value: 35.977
- type: ndcg_at_100
value: 41.094
- type: ndcg_at_1000
value: 43.44
- type: ndcg_at_20
value: 37.722
- type: ndcg_at_3
value: 30.596
- type: ndcg_at_5
value: 33.300999999999995
- type: precision_at_1
value: 25.692999999999998
- type: precision_at_10
value: 5.675
- type: precision_at_100
value: 0.8840000000000001
- type: precision_at_1000
value: 0.12
- type: precision_at_20
value: 3.253
- type: precision_at_3
value: 12.876999999999999
- type: precision_at_5
value: 9.316
- type: recall_at_1
value: 23.333000000000002
- type: recall_at_10
value: 49.014
- type: recall_at_100
value: 72.77
- type: recall_at_1000
value: 90.424
- type: recall_at_20
value: 55.718
- type: recall_at_3
value: 34.489
- type: recall_at_5
value: 41.069
task:
type: Retrieval
- dataset:
config: default
name: MTEB ClimateFEVER (default)
revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380
split: test
type: mteb/climate-fever
metrics:
- type: main_score
value: 40.665
- type: map_at_1
value: 18.331
- type: map_at_10
value: 30.631000000000004
- type: map_at_100
value: 32.78
- type: map_at_1000
value: 32.961
- type: map_at_20
value: 31.837
- type: map_at_3
value: 25.966
- type: map_at_5
value: 28.358
- type: mrr_at_1
value: 42.08469055374593
- type: mrr_at_10
value: 53.89806628405985
- type: mrr_at_100
value: 54.523521271715204
- type: mrr_at_1000
value: 54.544115472781066
- type: mrr_at_20
value: 54.318571521078155
- type: mrr_at_3
value: 51.08577633007607
- type: mrr_at_5
value: 52.76004343105332
- type: nauc_map_at_1000_diff1
value: 28.66279724592801
- type: nauc_map_at_1000_max
value: 40.25778426879266
- type: nauc_map_at_1000_std
value: 22.843404755367676
- type: nauc_map_at_100_diff1
value: 28.639439686161637
- type: nauc_map_at_100_max
value: 40.248969249981016
- type: nauc_map_at_100_std
value: 22.808242909424926
- type: nauc_map_at_10_diff1
value: 28.61506772153627
- type: nauc_map_at_10_max
value: 39.370345527055925
- type: nauc_map_at_10_std
value: 21.124956156063426
- type: nauc_map_at_1_diff1
value: 38.080854298707045
- type: nauc_map_at_1_max
value: 34.08902288212793
- type: nauc_map_at_1_std
value: 9.84432929683795
- type: nauc_map_at_20_diff1
value: 28.377267354165248
- type: nauc_map_at_20_max
value: 39.86577667800827
- type: nauc_map_at_20_std
value: 22.22605718850058
- type: nauc_map_at_3_diff1
value: 29.643624746333053
- type: nauc_map_at_3_max
value: 36.62660984521409
- type: nauc_map_at_3_std
value: 16.559180443901024
- type: nauc_map_at_5_diff1
value: 28.64868725155636
- type: nauc_map_at_5_max
value: 38.12417897902731
- type: nauc_map_at_5_std
value: 18.880981174442482
- type: nauc_mrr_at_1000_diff1
value: 31.469929815199254
- type: nauc_mrr_at_1000_max
value: 39.491437733953404
- type: nauc_mrr_at_1000_std
value: 24.16647047433768
- type: nauc_mrr_at_100_diff1
value: 31.467580423799163
- type: nauc_mrr_at_100_max
value: 39.50366461378037
- type: nauc_mrr_at_100_std
value: 24.17790086352623
- type: nauc_mrr_at_10_diff1
value: 31.45933060678348
- type: nauc_mrr_at_10_max
value: 39.7211569068441
- type: nauc_mrr_at_10_std
value: 24.398456960576272
- type: nauc_mrr_at_1_diff1
value: 35.22756819116212
- type: nauc_mrr_at_1_max
value: 35.39995365445277
- type: nauc_mrr_at_1_std
value: 17.176768576881987
- type: nauc_mrr_at_20_diff1
value: 31.43894468026687
- type: nauc_mrr_at_20_max
value: 39.58514381740837
- type: nauc_mrr_at_20_std
value: 24.229204696912763
- type: nauc_mrr_at_3_diff1
value: 30.34322716612186
- type: nauc_mrr_at_3_max
value: 38.11493133038243
- type: nauc_mrr_at_3_std
value: 23.195354121573196
- type: nauc_mrr_at_5_diff1
value: 30.978774210664533
- type: nauc_mrr_at_5_max
value: 39.20107683480328
- type: nauc_mrr_at_5_std
value: 24.16022688639071
- type: nauc_ndcg_at_1000_diff1
value: 28.966749941983483
- type: nauc_ndcg_at_1000_max
value: 43.94884315290988
- type: nauc_ndcg_at_1000_std
value: 30.013680691530347
- type: nauc_ndcg_at_100_diff1
value: 28.510858889585002
- type: nauc_ndcg_at_100_max
value: 43.961967588751584
- type: nauc_ndcg_at_100_std
value: 29.691366273678437
- type: nauc_ndcg_at_10_diff1
value: 28.29575867721997
- type: nauc_ndcg_at_10_max
value: 41.967654471182954
- type: nauc_ndcg_at_10_std
value: 25.698895263549552
- type: nauc_ndcg_at_1_diff1
value: 35.22756819116212
- type: nauc_ndcg_at_1_max
value: 35.39995365445277
- type: nauc_ndcg_at_1_std
value: 17.176768576881987
- type: nauc_ndcg_at_20_diff1
value: 27.774476349912202
- type: nauc_ndcg_at_20_max
value: 42.79292342302096
- type: nauc_ndcg_at_20_std
value: 27.521022252457016
- type: nauc_ndcg_at_3_diff1
value: 27.825944882654525
- type: nauc_ndcg_at_3_max
value: 37.2298293959292
- type: nauc_ndcg_at_3_std
value: 20.08915077852354
- type: nauc_ndcg_at_5_diff1
value: 27.819610550591108
- type: nauc_ndcg_at_5_max
value: 39.78372750504299
- type: nauc_ndcg_at_5_std
value: 22.591735790601717
- type: nauc_precision_at_1000_diff1
value: -4.50335714243064
- type: nauc_precision_at_1000_max
value: 8.251912026083913
- type: nauc_precision_at_1000_std
value: 25.069582587880397
- type: nauc_precision_at_100_diff1
value: 2.5806462972358646
- type: nauc_precision_at_100_max
value: 22.89207571209036
- type: nauc_precision_at_100_std
value: 33.53219498762396
- type: nauc_precision_at_10_diff1
value: 10.337042519080265
- type: nauc_precision_at_10_max
value: 33.219495576176485
- type: nauc_precision_at_10_std
value: 33.380734520375654
- type: nauc_precision_at_1_diff1
value: 35.22756819116212
- type: nauc_precision_at_1_max
value: 35.39995365445277
- type: nauc_precision_at_1_std
value: 17.176768576881987
- type: nauc_precision_at_20_diff1
value: 6.154629484583631
- type: nauc_precision_at_20_max
value: 29.97367729545834
- type: nauc_precision_at_20_std
value: 34.06376927516598
- type: nauc_precision_at_3_diff1
value: 14.847959988663446
- type: nauc_precision_at_3_max
value: 33.92508247202971
- type: nauc_precision_at_3_std
value: 27.418645475755994
- type: nauc_precision_at_5_diff1
value: 12.132570928439415
- type: nauc_precision_at_5_max
value: 34.06309599360566
- type: nauc_precision_at_5_std
value: 30.321616018987662
- type: nauc_recall_at_1000_diff1
value: 20.314982656928557
- type: nauc_recall_at_1000_max
value: 50.29152625089194
- type: nauc_recall_at_1000_std
value: 54.206815810043395
- type: nauc_recall_at_100_diff1
value: 19.108674599887934
- type: nauc_recall_at_100_max
value: 44.38802294958441
- type: nauc_recall_at_100_std
value: 39.826771664800155
- type: nauc_recall_at_10_diff1
value: 21.967797939910902
- type: nauc_recall_at_10_max
value: 40.93510726774966
- type: nauc_recall_at_10_std
value: 27.557171358431017
- type: nauc_recall_at_1_diff1
value: 38.080854298707045
- type: nauc_recall_at_1_max
value: 34.08902288212793
- type: nauc_recall_at_1_std
value: 9.84432929683795
- type: nauc_recall_at_20_diff1
value: 19.152237539756335
- type: nauc_recall_at_20_max
value: 41.15168395500831
- type: nauc_recall_at_20_std
value: 31.14319157208471
- type: nauc_recall_at_3_diff1
value: 23.723560256940054
- type: nauc_recall_at_3_max
value: 36.66374076935461
- type: nauc_recall_at_3_std
value: 19.394220430158605
- type: nauc_recall_at_5_diff1
value: 22.456881288343325
- type: nauc_recall_at_5_max
value: 39.11368648541473
- type: nauc_recall_at_5_std
value: 23.211447644138584
- type: ndcg_at_1
value: 42.085
- type: ndcg_at_10
value: 40.665
- type: ndcg_at_100
value: 47.947
- type: ndcg_at_1000
value: 50.827
- type: ndcg_at_20
value: 43.671
- type: ndcg_at_3
value: 34.743
- type: ndcg_at_5
value: 36.537
- type: precision_at_1
value: 42.085
- type: precision_at_10
value: 12.378
- type: precision_at_100
value: 2.023
- type: precision_at_1000
value: 0.257
- type: precision_at_20
value: 7.489
- type: precision_at_3
value: 25.798
- type: precision_at_5
value: 19.166
- type: recall_at_1
value: 18.331
- type: recall_at_10
value: 45.959
- type: recall_at_100
value: 70.463
- type: recall_at_1000
value: 86.275
- type: recall_at_20
value: 54.381
- type: recall_at_3
value: 30.742000000000004
- type: recall_at_5
value: 36.962
task:
type: Retrieval
- dataset:
config: default
name: MTEB CmedqaRetrieval (default)
revision: cd540c506dae1cf9e9a59c3e06f42030d54e7301
split: test
type: C-MTEB/CmedqaRetrieval
metrics:
- type: main_score
value: 39.474
- type: map_at_1
value: 22.913
- type: map_at_10
value: 33.42
- type: map_at_100
value: 35.187000000000005
- type: map_at_1000
value: 35.329
- type: map_at_20
value: 34.367999999999995
- type: map_at_3
value: 29.801
- type: map_at_5
value: 31.819999999999997
- type: mrr_at_1
value: 35.20880220055014
- type: mrr_at_10
value: 42.46370124277093
- type: mrr_at_100
value: 43.451892746304374
- type: mrr_at_1000
value: 43.515357148254616
- type: mrr_at_20
value: 43.01605074381842
- type: mrr_at_3
value: 40.18921397015911
- type: mrr_at_5
value: 41.442027173460005
- type: nauc_map_at_1000_diff1
value: 47.88109062782217
- type: nauc_map_at_1000_max
value: 47.30104864042583
- type: nauc_map_at_1000_std
value: -2.4422108261892004
- type: nauc_map_at_100_diff1
value: 47.840155550356286
- type: nauc_map_at_100_max
value: 47.26929656857804
- type: nauc_map_at_100_std
value: -2.4764234791086857
- type: nauc_map_at_10_diff1
value: 47.754314011595014
- type: nauc_map_at_10_max
value: 46.35945658400591
- type: nauc_map_at_10_std
value: -3.617190085107328
- type: nauc_map_at_1_diff1
value: 51.81726359874944
- type: nauc_map_at_1_max
value: 38.70534924597951
- type: nauc_map_at_1_std
value: -6.266993974109676
- type: nauc_map_at_20_diff1
value: 47.75255629516694
- type: nauc_map_at_20_max
value: 46.93628183798524
- type: nauc_map_at_20_std
value: -3.053187607599497
- type: nauc_map_at_3_diff1
value: 48.25302115458153
- type: nauc_map_at_3_max
value: 44.0710509539875
- type: nauc_map_at_3_std
value: -4.789110631158403
- type: nauc_map_at_5_diff1
value: 47.97922926523539
- type: nauc_map_at_5_max
value: 45.36522904658003
- type: nauc_map_at_5_std
value: -4.368174915583596
- type: nauc_mrr_at_1000_diff1
value: 54.37909929684462
- type: nauc_mrr_at_1000_max
value: 54.09320206810515
- type: nauc_mrr_at_1000_std
value: 1.9631808253767942
- type: nauc_mrr_at_100_diff1
value: 54.35701907810265
- type: nauc_mrr_at_100_max
value: 54.084515715106996
- type: nauc_mrr_at_100_std
value: 1.9744777479582167
- type: nauc_mrr_at_10_diff1
value: 54.25790229059854
- type: nauc_mrr_at_10_max
value: 53.93835824423914
- type: nauc_mrr_at_10_std
value: 1.7113656270743158
- type: nauc_mrr_at_1_diff1
value: 58.99251163531494
- type: nauc_mrr_at_1_max
value: 55.31168458378662
- type: nauc_mrr_at_1_std
value: 0.46067296570591204
- type: nauc_mrr_at_20_diff1
value: 54.31296677365124
- type: nauc_mrr_at_20_max
value: 54.04543558309209
- type: nauc_mrr_at_20_std
value: 1.8817185785511097
- type: nauc_mrr_at_3_diff1
value: 55.2604408399222
- type: nauc_mrr_at_3_max
value: 54.57860061961037
- type: nauc_mrr_at_3_std
value: 1.4687386437880332
- type: nauc_mrr_at_5_diff1
value: 54.61101931248141
- type: nauc_mrr_at_5_max
value: 54.15973329758553
- type: nauc_mrr_at_5_std
value: 1.5953274190295135
- type: nauc_ndcg_at_1000_diff1
value: 48.43475753849973
- type: nauc_ndcg_at_1000_max
value: 50.031126656407295
- type: nauc_ndcg_at_1000_std
value: 1.37629591827648
- type: nauc_ndcg_at_100_diff1
value: 47.589901287290814
- type: nauc_ndcg_at_100_max
value: 49.78923485645858
- type: nauc_ndcg_at_100_std
value: 1.7483960647728582
- type: nauc_ndcg_at_10_diff1
value: 47.24265676156858
- type: nauc_ndcg_at_10_max
value: 47.92982530292526
- type: nauc_ndcg_at_10_std
value: -1.6814141507368878
- type: nauc_ndcg_at_1_diff1
value: 58.99251163531494
- type: nauc_ndcg_at_1_max
value: 55.31168458378662
- type: nauc_ndcg_at_1_std
value: 0.46067296570591204
- type: nauc_ndcg_at_20_diff1
value: 47.186272426240436
- type: nauc_ndcg_at_20_max
value: 48.761525981057815
- type: nauc_ndcg_at_20_std
value: -0.5158662579547165
- type: nauc_ndcg_at_3_diff1
value: 48.994740589867256
- type: nauc_ndcg_at_3_max
value: 49.45848256380362
- type: nauc_ndcg_at_3_std
value: -1.1778055460063026
- type: nauc_ndcg_at_5_diff1
value: 48.03312755832304
- type: nauc_ndcg_at_5_max
value: 48.13971140429206
- type: nauc_ndcg_at_5_std
value: -1.9236881626706022
- type: nauc_precision_at_1000_diff1
value: 6.8796591359425685
- type: nauc_precision_at_1000_max
value: 26.485734999139986
- type: nauc_precision_at_1000_std
value: 21.378355818349448
- type: nauc_precision_at_100_diff1
value: 13.875906788437684
- type: nauc_precision_at_100_max
value: 36.69243065017309
- type: nauc_precision_at_100_std
value: 22.566174889409048
- type: nauc_precision_at_10_diff1
value: 27.483109850720673
- type: nauc_precision_at_10_max
value: 49.05252314758842
- type: nauc_precision_at_10_std
value: 10.81697625119965
- type: nauc_precision_at_1_diff1
value: 58.99251163531494
- type: nauc_precision_at_1_max
value: 55.31168458378662
- type: nauc_precision_at_1_std
value: 0.46067296570591204
- type: nauc_precision_at_20_diff1
value: 22.596360405586747
- type: nauc_precision_at_20_max
value: 45.898897052708584
- type: nauc_precision_at_20_std
value: 14.550759059636537
- type: nauc_precision_at_3_diff1
value: 38.434521839052124
- type: nauc_precision_at_3_max
value: 53.42967557108986
- type: nauc_precision_at_3_std
value: 5.637035190585405
- type: nauc_precision_at_5_diff1
value: 33.02740643862731
- type: nauc_precision_at_5_max
value: 51.55978819109881
- type: nauc_precision_at_5_std
value: 7.350865162796566
- type: nauc_recall_at_1000_diff1
value: 26.60961664665872
- type: nauc_recall_at_1000_max
value: 48.461012374664655
- type: nauc_recall_at_1000_std
value: 42.11129481075962
- type: nauc_recall_at_100_diff1
value: 30.56131256456399
- type: nauc_recall_at_100_max
value: 42.6335003846358
- type: nauc_recall_at_100_std
value: 14.65947561811711
- type: nauc_recall_at_10_diff1
value: 36.08925172792211
- type: nauc_recall_at_10_max
value: 39.549282848557404
- type: nauc_recall_at_10_std
value: -2.0637274570893176
- type: nauc_recall_at_1_diff1
value: 51.81726359874944
- type: nauc_recall_at_1_max
value: 38.70534924597951
- type: nauc_recall_at_1_std
value: -6.266993974109676
- type: nauc_recall_at_20_diff1
value: 34.19944302550159
- type: nauc_recall_at_20_max
value: 40.570502896763685
- type: nauc_recall_at_20_std
value: 1.6373957170773366
- type: nauc_recall_at_3_diff1
value: 42.661186351217
- type: nauc_recall_at_3_max
value: 41.07991726233878
- type: nauc_recall_at_3_std
value: -4.275756719649943
- type: nauc_recall_at_5_diff1
value: 39.824533760699666
- type: nauc_recall_at_5_max
value: 40.27289110879392
- type: nauc_recall_at_5_std
value: -3.7217848473578097
- type: ndcg_at_1
value: 35.209
- type: ndcg_at_10
value: 39.474
- type: ndcg_at_100
value: 46.664
- type: ndcg_at_1000
value: 49.375
- type: ndcg_at_20
value: 42.124
- type: ndcg_at_3
value: 34.855000000000004
- type: ndcg_at_5
value: 36.758
- type: precision_at_1
value: 35.209
- type: precision_at_10
value: 8.777
- type: precision_at_100
value: 1.464
- type: precision_at_1000
value: 0.18
- type: precision_at_20
value: 5.273
- type: precision_at_3
value: 19.688
- type: precision_at_5
value: 14.313999999999998
- type: recall_at_1
value: 22.913
- type: recall_at_10
value: 48.452
- type: recall_at_100
value: 78.459
- type: recall_at_1000
value: 97.02499999999999
- type: recall_at_20
value: 57.483
- type: recall_at_3
value: 34.365
- type: recall_at_5
value: 40.376
task:
type: Retrieval
- dataset:
config: default
name: MTEB CovidRetrieval (default)
revision: 1271c7809071a13532e05f25fb53511ffce77117
split: test
type: C-MTEB/CovidRetrieval
metrics:
- type: main_score
value: 86.248
- type: map_at_1
value: 76.238
- type: map_at_10
value: 83.121
- type: map_at_100
value: 83.30199999999999
- type: map_at_1000
value: 83.30499999999999
- type: map_at_20
value: 83.25
- type: map_at_3
value: 81.612
- type: map_at_5
value: 82.48
- type: mrr_at_1
value: 76.39620653319284
- type: mrr_at_10
value: 83.11798551524579
- type: mrr_at_100
value: 83.28539903804636
- type: mrr_at_1000
value: 83.28853918390932
- type: mrr_at_20
value: 83.23356831754137
- type: mrr_at_3
value: 81.6824727783632
- type: mrr_at_5
value: 82.51492799438009
- type: nauc_map_at_1000_diff1
value: 85.81070525508966
- type: nauc_map_at_1000_max
value: 45.965324844973
- type: nauc_map_at_1000_std
value: -52.78173904133118
- type: nauc_map_at_100_diff1
value: 85.81125460117724
- type: nauc_map_at_100_max
value: 45.97546414701782
- type: nauc_map_at_100_std
value: -52.75835778074522
- type: nauc_map_at_10_diff1
value: 85.7051290682998
- type: nauc_map_at_10_max
value: 46.23611656408647
- type: nauc_map_at_10_std
value: -52.6666709589737
- type: nauc_map_at_1_diff1
value: 88.26402080799741
- type: nauc_map_at_1_max
value: 44.050351515784264
- type: nauc_map_at_1_std
value: -50.56289445376376
- type: nauc_map_at_20_diff1
value: 85.77842644001255
- type: nauc_map_at_20_max
value: 46.04394867157703
- type: nauc_map_at_20_std
value: -52.78608349243351
- type: nauc_map_at_3_diff1
value: 85.30533143846657
- type: nauc_map_at_3_max
value: 44.887945606250355
- type: nauc_map_at_3_std
value: -54.935281949459714
- type: nauc_map_at_5_diff1
value: 85.50995878356277
- type: nauc_map_at_5_max
value: 45.46429726268937
- type: nauc_map_at_5_std
value: -54.088714368512626
- type: nauc_mrr_at_1000_diff1
value: 85.80217473988819
- type: nauc_mrr_at_1000_max
value: 45.86491819883026
- type: nauc_mrr_at_1000_std
value: -52.89623324635521
- type: nauc_mrr_at_100_diff1
value: 85.80272509797886
- type: nauc_mrr_at_100_max
value: 45.8750645714051
- type: nauc_mrr_at_100_std
value: -52.872859499782344
- type: nauc_mrr_at_10_diff1
value: 85.70145531650965
- type: nauc_mrr_at_10_max
value: 46.12911790236837
- type: nauc_mrr_at_10_std
value: -52.78082308094456
- type: nauc_mrr_at_1_diff1
value: 88.02729122831771
- type: nauc_mrr_at_1_max
value: 43.930769240632856
- type: nauc_mrr_at_1_std
value: -50.27087569264231
- type: nauc_mrr_at_20_diff1
value: 85.76996861021632
- type: nauc_mrr_at_20_max
value: 45.94379525383032
- type: nauc_mrr_at_20_std
value: -52.90016547379902
- type: nauc_mrr_at_3_diff1
value: 85.19995296250845
- type: nauc_mrr_at_3_max
value: 44.9761359296206
- type: nauc_mrr_at_3_std
value: -54.75325362660898
- type: nauc_mrr_at_5_diff1
value: 85.49765148042313
- type: nauc_mrr_at_5_max
value: 45.46879615637081
- type: nauc_mrr_at_5_std
value: -54.04590504125581
- type: nauc_ndcg_at_1000_diff1
value: 85.54750548221148
- type: nauc_ndcg_at_1000_max
value: 46.56757864726956
- type: nauc_ndcg_at_1000_std
value: -52.455376300829506
- type: nauc_ndcg_at_100_diff1
value: 85.58037530719729
- type: nauc_ndcg_at_100_max
value: 46.87344744069495
- type: nauc_ndcg_at_100_std
value: -51.80542993489906
- type: nauc_ndcg_at_10_diff1
value: 85.05501862955802
- type: nauc_ndcg_at_10_max
value: 48.21207972888072
- type: nauc_ndcg_at_10_std
value: -51.36375441528026
- type: nauc_ndcg_at_1_diff1
value: 88.02729122831771
- type: nauc_ndcg_at_1_max
value: 43.930769240632856
- type: nauc_ndcg_at_1_std
value: -50.27087569264231
- type: nauc_ndcg_at_20_diff1
value: 85.34336657375991
- type: nauc_ndcg_at_20_max
value: 47.435473586489564
- type: nauc_ndcg_at_20_std
value: -51.91008916005085
- type: nauc_ndcg_at_3_diff1
value: 84.13748027210943
- type: nauc_ndcg_at_3_max
value: 45.12567396232455
- type: nauc_ndcg_at_3_std
value: -56.47669761642026
- type: nauc_ndcg_at_5_diff1
value: 84.55448795921725
- type: nauc_ndcg_at_5_max
value: 46.18660401749204
- type: nauc_ndcg_at_5_std
value: -55.125758391684386
- type: nauc_precision_at_1000_diff1
value: -51.3307609221751
- type: nauc_precision_at_1000_max
value: -11.953105148922791
- type: nauc_precision_at_1000_std
value: 50.43685922371517
- type: nauc_precision_at_100_diff1
value: -14.801826549460248
- type: nauc_precision_at_100_max
value: 14.47045243643168
- type: nauc_precision_at_100_std
value: 47.50643511649529
- type: nauc_precision_at_10_diff1
value: 44.205996642681235
- type: nauc_precision_at_10_max
value: 51.92320132144985
- type: nauc_precision_at_10_std
value: -5.899575746257043
- type: nauc_precision_at_1_diff1
value: 88.02729122831771
- type: nauc_precision_at_1_max
value: 43.930769240632856
- type: nauc_precision_at_1_std
value: -50.27087569264231
- type: nauc_precision_at_20_diff1
value: 24.05279079986572
- type: nauc_precision_at_20_max
value: 36.65699450072386
- type: nauc_precision_at_20_std
value: 4.9968093048199655
- type: nauc_precision_at_3_diff1
value: 75.0765824270448
- type: nauc_precision_at_3_max
value: 43.9377907848161
- type: nauc_precision_at_3_std
value: -59.06007120970449
- type: nauc_precision_at_5_diff1
value: 66.45032329654471
- type: nauc_precision_at_5_max
value: 44.70325593801851
- type: nauc_precision_at_5_std
value: -48.45128571406001
- type: nauc_recall_at_1000_diff1
value: .nan
- type: nauc_recall_at_1000_max
value: .nan
- type: nauc_recall_at_1000_std
value: .nan
- type: nauc_recall_at_100_diff1
value: 91.01166071733385
- type: nauc_recall_at_100_max
value: 91.01166071733385
- type: nauc_recall_at_100_std
value: 39.017865732240296
- type: nauc_recall_at_10_diff1
value: 79.23395734932605
- type: nauc_recall_at_10_max
value: 72.33475719286457
- type: nauc_recall_at_10_std
value: -31.825871246522368
- type: nauc_recall_at_1_diff1
value: 88.26402080799741
- type: nauc_recall_at_1_max
value: 44.050351515784264
- type: nauc_recall_at_1_std
value: -50.56289445376376
- type: nauc_recall_at_20_diff1
value: 81.01725729918286
- type: nauc_recall_at_20_max
value: 73.38978739473507
- type: nauc_recall_at_20_std
value: -29.34049826695243
- type: nauc_recall_at_3_diff1
value: 79.42920887206496
- type: nauc_recall_at_3_max
value: 45.82669161901384
- type: nauc_recall_at_3_std
value: -62.97799126817177
- type: nauc_recall_at_5_diff1
value: 79.59149793640871
- type: nauc_recall_at_5_max
value: 49.931726479477284
- type: nauc_recall_at_5_std
value: -60.84730627583382
- type: ndcg_at_1
value: 76.396
- type: ndcg_at_10
value: 86.248
- type: ndcg_at_100
value: 87.024
- type: ndcg_at_1000
value: 87.105
- type: ndcg_at_20
value: 86.68
- type: ndcg_at_3
value: 83.248
- type: ndcg_at_5
value: 84.76700000000001
- type: precision_at_1
value: 76.396
- type: precision_at_10
value: 9.684
- type: precision_at_100
value: 1.004
- type: precision_at_1000
value: 0.101
- type: precision_at_20
value: 4.932
- type: precision_at_3
value: 29.398999999999997
- type: precision_at_5
value: 18.419
- type: recall_at_1
value: 76.238
- type: recall_at_10
value: 95.943
- type: recall_at_100
value: 99.368
- type: recall_at_1000
value: 100
- type: recall_at_20
value: 97.576
- type: recall_at_3
value: 87.882
- type: recall_at_5
value: 91.465
task:
type: Retrieval
- dataset:
config: default
name: MTEB DBPedia (default)
revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659
split: test
type: mteb/dbpedia
metrics:
- type: main_score
value: 45.591
- type: map_at_1
value: 8.742999999999999
- type: map_at_10
value: 20.905
- type: map_at_100
value: 30.684
- type: map_at_1000
value: 32.621
- type: map_at_20
value: 24.712999999999997
- type: map_at_3
value: 14.557
- type: map_at_5
value: 17.145
- type: mrr_at_1
value: 71.25
- type: mrr_at_10
value: 78.66547619047621
- type: mrr_at_100
value: 78.95338376428113
- type: mrr_at_1000
value: 78.95801669419396
- type: mrr_at_20
value: 78.86100898692813
- type: mrr_at_3
value: 77.29166666666669
- type: mrr_at_5
value: 78.22916666666667
- type: nauc_map_at_1000_diff1
value: 26.900440763739862
- type: nauc_map_at_1000_max
value: 27.41287499682588
- type: nauc_map_at_1000_std
value: 23.517716446354417
- type: nauc_map_at_100_diff1
value: 26.960847090712015
- type: nauc_map_at_100_max
value: 25.62780822806535
- type: nauc_map_at_100_std
value: 20.764583410794064
- type: nauc_map_at_10_diff1
value: 26.694350486905847
- type: nauc_map_at_10_max
value: 12.107317246864257
- type: nauc_map_at_10_std
value: -6.508802089734562
- type: nauc_map_at_1_diff1
value: 33.456445880493604
- type: nauc_map_at_1_max
value: 1.8493036801759046
- type: nauc_map_at_1_std
value: -20.84551283666329
- type: nauc_map_at_20_diff1
value: 26.507520810175777
- type: nauc_map_at_20_max
value: 17.43875958766562
- type: nauc_map_at_20_std
value: 4.266163091537051
- type: nauc_map_at_3_diff1
value: 28.002541708305444
- type: nauc_map_at_3_max
value: 4.0809964483517005
- type: nauc_map_at_3_std
value: -18.34367516143987
- type: nauc_map_at_5_diff1
value: 25.76395976603995
- type: nauc_map_at_5_max
value: 5.813153343229154
- type: nauc_map_at_5_std
value: -14.741693694003125
- type: nauc_mrr_at_1000_diff1
value: 50.942757165439
- type: nauc_mrr_at_1000_max
value: 56.366470724627945
- type: nauc_mrr_at_1000_std
value: 34.50507989901226
- type: nauc_mrr_at_100_diff1
value: 50.94308173621799
- type: nauc_mrr_at_100_max
value: 56.374579967289904
- type: nauc_mrr_at_100_std
value: 34.514547334865874
- type: nauc_mrr_at_10_diff1
value: 51.05277331154666
- type: nauc_mrr_at_10_max
value: 56.379238078204565
- type: nauc_mrr_at_10_std
value: 34.22592269786022
- type: nauc_mrr_at_1_diff1
value: 51.19992534005276
- type: nauc_mrr_at_1_max
value: 53.29320361167492
- type: nauc_mrr_at_1_std
value: 29.698327150556516
- type: nauc_mrr_at_20_diff1
value: 50.893948496369845
- type: nauc_mrr_at_20_max
value: 56.345711333735316
- type: nauc_mrr_at_20_std
value: 34.53797587141166
- type: nauc_mrr_at_3_diff1
value: 50.07564959982456
- type: nauc_mrr_at_3_max
value: 56.4377475772584
- type: nauc_mrr_at_3_std
value: 35.5047835143934
- type: nauc_mrr_at_5_diff1
value: 51.45075569502099
- type: nauc_mrr_at_5_max
value: 56.58093185835439
- type: nauc_mrr_at_5_std
value: 34.742060733649815
- type: nauc_ndcg_at_1000_diff1
value: 32.743004702251724
- type: nauc_ndcg_at_1000_max
value: 41.032846031207185
- type: nauc_ndcg_at_1000_std
value: 36.784535087650596
- type: nauc_ndcg_at_100_diff1
value: 33.46270808018173
- type: nauc_ndcg_at_100_max
value: 35.31907416778681
- type: nauc_ndcg_at_100_std
value: 30.076919254877726
- type: nauc_ndcg_at_10_diff1
value: 31.52784380637351
- type: nauc_ndcg_at_10_max
value: 33.51288766599305
- type: nauc_ndcg_at_10_std
value: 21.036204411544702
- type: nauc_ndcg_at_1_diff1
value: 44.24341788021985
- type: nauc_ndcg_at_1_max
value: 35.76352589449317
- type: nauc_ndcg_at_1_std
value: 20.47642009837009
- type: nauc_ndcg_at_20_diff1
value: 32.22677428049063
- type: nauc_ndcg_at_20_max
value: 31.522795481794386
- type: nauc_ndcg_at_20_std
value: 21.97922534275596
- type: nauc_ndcg_at_3_diff1
value: 34.55448406523195
- type: nauc_ndcg_at_3_max
value: 35.88713452555367
- type: nauc_ndcg_at_3_std
value: 20.680779175992637
- type: nauc_ndcg_at_5_diff1
value: 29.952345125400193
- type: nauc_ndcg_at_5_max
value: 33.204505749144225
- type: nauc_ndcg_at_5_std
value: 20.169179073327566
- type: nauc_precision_at_1000_diff1
value: -4.691637817960754
- type: nauc_precision_at_1000_max
value: 3.77960645220539
- type: nauc_precision_at_1000_std
value: 7.592604432200344
- type: nauc_precision_at_100_diff1
value: 0.7691018824540463
- type: nauc_precision_at_100_max
value: 25.09779364224492
- type: nauc_precision_at_100_std
value: 43.63510399036115
- type: nauc_precision_at_10_diff1
value: 8.16740121062683
- type: nauc_precision_at_10_max
value: 35.66102038839878
- type: nauc_precision_at_10_std
value: 43.307605079242116
- type: nauc_precision_at_1_diff1
value: 51.19992534005276
- type: nauc_precision_at_1_max
value: 53.29320361167492
- type: nauc_precision_at_1_std
value: 29.698327150556516
- type: nauc_precision_at_20_diff1
value: 3.6725937217003084
- type: nauc_precision_at_20_max
value: 31.556397294122657
- type: nauc_precision_at_20_std
value: 47.05790447768477
- type: nauc_precision_at_3_diff1
value: 19.107686617046717
- type: nauc_precision_at_3_max
value: 37.629662422617535
- type: nauc_precision_at_3_std
value: 30.667392858755104
- type: nauc_precision_at_5_diff1
value: 9.268155344857693
- type: nauc_precision_at_5_max
value: 33.73508193552941
- type: nauc_precision_at_5_std
value: 34.1577681720702
- type: nauc_recall_at_1000_diff1
value: 19.670215341306402
- type: nauc_recall_at_1000_max
value: 36.303227182727575
- type: nauc_recall_at_1000_std
value: 46.035472761074864
- type: nauc_recall_at_100_diff1
value: 22.891294916593708
- type: nauc_recall_at_100_max
value: 26.633595306280654
- type: nauc_recall_at_100_std
value: 27.810060155447008
- type: nauc_recall_at_10_diff1
value: 20.689635264248775
- type: nauc_recall_at_10_max
value: 6.44521719985839
- type: nauc_recall_at_10_std
value: -10.762652871248912
- type: nauc_recall_at_1_diff1
value: 33.456445880493604
- type: nauc_recall_at_1_max
value: 1.8493036801759046
- type: nauc_recall_at_1_std
value: -20.84551283666329
- type: nauc_recall_at_20_diff1
value: 19.968043212513955
- type: nauc_recall_at_20_max
value: 12.406516394986097
- type: nauc_recall_at_20_std
value: 1.9469308740605251
- type: nauc_recall_at_3_diff1
value: 23.86584939104936
- type: nauc_recall_at_3_max
value: 1.5263218674355084
- type: nauc_recall_at_3_std
value: -20.020404639715853
- type: nauc_recall_at_5_diff1
value: 20.963971034696304
- type: nauc_recall_at_5_max
value: 1.1659794377540564
- type: nauc_recall_at_5_std
value: -17.984037259129675
- type: ndcg_at_1
value: 58.75
- type: ndcg_at_10
value: 45.591
- type: ndcg_at_100
value: 50.32299999999999
- type: ndcg_at_1000
value: 57.339
- type: ndcg_at_20
value: 44.830999999999996
- type: ndcg_at_3
value: 50.348000000000006
- type: ndcg_at_5
value: 47.413
- type: precision_at_1
value: 71.25
- type: precision_at_10
value: 37
- type: precision_at_100
value: 12.275
- type: precision_at_1000
value: 2.388
- type: precision_at_20
value: 28.599999999999998
- type: precision_at_3
value: 54.167
- type: precision_at_5
value: 45.95
- type: recall_at_1
value: 8.742999999999999
- type: recall_at_10
value: 27.235
- type: recall_at_100
value: 56.87
- type: recall_at_1000
value: 79.616
- type: recall_at_20
value: 35.42
- type: recall_at_3
value: 15.899
- type: recall_at_5
value: 20.226
task:
type: Retrieval
- dataset:
config: default
name: MTEB DuRetrieval (default)
revision: a1a333e290fe30b10f3f56498e3a0d911a693ced
split: test
type: C-MTEB/DuRetrieval
metrics:
- type: main_score
value: 89.092
- type: map_at_1
value: 26.840999999999998
- type: map_at_10
value: 82.53399999999999
- type: map_at_100
value: 85.024
- type: map_at_1000
value: 85.06700000000001
- type: map_at_20
value: 84.468
- type: map_at_3
value: 57.318000000000005
- type: map_at_5
value: 72.45899999999999
- type: mrr_at_1
value: 91.8
- type: mrr_at_10
value: 94.22331349206348
- type: mrr_at_100
value: 94.29073099613512
- type: mrr_at_1000
value: 94.29345590354411
- type: mrr_at_20
value: 94.27236714158731
- type: mrr_at_3
value: 93.9583333333333
- type: mrr_at_5
value: 94.13083333333331
- type: nauc_map_at_1000_diff1
value: -0.5294847921357029
- type: nauc_map_at_1000_max
value: 48.537982451222256
- type: nauc_map_at_1000_std
value: 24.57134294695126
- type: nauc_map_at_100_diff1
value: -0.4972121954388926
- type: nauc_map_at_100_max
value: 48.54578906367702
- type: nauc_map_at_100_std
value: 24.553424486015988
- type: nauc_map_at_10_diff1
value: 4.540149880617989
- type: nauc_map_at_10_max
value: 45.010271398330936
- type: nauc_map_at_10_std
value: 11.342740975544137
- type: nauc_map_at_1_diff1
value: 46.671939051209534
- type: nauc_map_at_1_max
value: -10.343353582447454
- type: nauc_map_at_1_std
value: -37.000769458641614
- type: nauc_map_at_20_diff1
value: 0.7353948327885964
- type: nauc_map_at_20_max
value: 48.159473696425586
- type: nauc_map_at_20_std
value: 21.726623183516672
- type: nauc_map_at_3_diff1
value: 27.36273018992832
- type: nauc_map_at_3_max
value: 6.209222976162968
- type: nauc_map_at_3_std
value: -29.49937019740436
- type: nauc_map_at_5_diff1
value: 16.355176688788458
- type: nauc_map_at_5_max
value: 23.68912322491999
- type: nauc_map_at_5_std
value: -15.58756532067869
- type: nauc_mrr_at_1000_diff1
value: 29.106131814711762
- type: nauc_mrr_at_1000_max
value: 83.69793658177312
- type: nauc_mrr_at_1000_std
value: 53.466794583101674
- type: nauc_mrr_at_100_diff1
value: 29.099944594214218
- type: nauc_mrr_at_100_max
value: 83.7016063495492
- type: nauc_mrr_at_100_std
value: 53.46110370474347
- type: nauc_mrr_at_10_diff1
value: 29.177768115036866
- type: nauc_mrr_at_10_max
value: 83.85598578162819
- type: nauc_mrr_at_10_std
value: 53.721145169786546
- type: nauc_mrr_at_1_diff1
value: 28.92441529457308
- type: nauc_mrr_at_1_max
value: 80.10606681697051
- type: nauc_mrr_at_1_std
value: 49.52062125663288
- type: nauc_mrr_at_20_diff1
value: 29.05360668887872
- type: nauc_mrr_at_20_max
value: 83.74337903625239
- type: nauc_mrr_at_20_std
value: 53.560896726300456
- type: nauc_mrr_at_3_diff1
value: 29.381886087768592
- type: nauc_mrr_at_3_max
value: 83.79941401848109
- type: nauc_mrr_at_3_std
value: 53.206993142084336
- type: nauc_mrr_at_5_diff1
value: 29.490340317110018
- type: nauc_mrr_at_5_max
value: 83.9890625188502
- type: nauc_mrr_at_5_std
value: 53.88092195560588
- type: nauc_ndcg_at_1000_diff1
value: 2.2592818905716445
- type: nauc_ndcg_at_1000_max
value: 59.078754205314574
- type: nauc_ndcg_at_1000_std
value: 37.42083837752937
- type: nauc_ndcg_at_100_diff1
value: 1.910169845715913
- type: nauc_ndcg_at_100_max
value: 58.797074915831814
- type: nauc_ndcg_at_100_std
value: 37.641930982895865
- type: nauc_ndcg_at_10_diff1
value: 3.523547276319842
- type: nauc_ndcg_at_10_max
value: 54.362912834968824
- type: nauc_ndcg_at_10_std
value: 29.470303595440672
- type: nauc_ndcg_at_1_diff1
value: 28.92441529457308
- type: nauc_ndcg_at_1_max
value: 80.10606681697051
- type: nauc_ndcg_at_1_std
value: 49.52062125663288
- type: nauc_ndcg_at_20_diff1
value: 3.2099650914494147
- type: nauc_ndcg_at_20_max
value: 57.29392522446022
- type: nauc_ndcg_at_20_std
value: 33.29006591117316
- type: nauc_ndcg_at_3_diff1
value: -5.274180027153056
- type: nauc_ndcg_at_3_max
value: 54.63608285211238
- type: nauc_ndcg_at_3_std
value: 35.28328510031142
- type: nauc_ndcg_at_5_diff1
value: 1.6186355731863125
- type: nauc_ndcg_at_5_max
value: 47.9267534122155
- type: nauc_ndcg_at_5_std
value: 24.650267223938023
- type: nauc_precision_at_1000_diff1
value: -33.57623151624042
- type: nauc_precision_at_1000_max
value: 15.406833094801003
- type: nauc_precision_at_1000_std
value: 52.78230528681039
- type: nauc_precision_at_100_diff1
value: -34.188087848812856
- type: nauc_precision_at_100_max
value: 17.56114547834119
- type: nauc_precision_at_100_std
value: 54.99183244443311
- type: nauc_precision_at_10_diff1
value: -35.480881894318664
- type: nauc_precision_at_10_max
value: 29.832737480951206
- type: nauc_precision_at_10_std
value: 54.80988882122554
- type: nauc_precision_at_1_diff1
value: 28.92441529457308
- type: nauc_precision_at_1_max
value: 80.10606681697051
- type: nauc_precision_at_1_std
value: 49.52062125663288
- type: nauc_precision_at_20_diff1
value: -34.56876549624
- type: nauc_precision_at_20_max
value: 22.869127677560748
- type: nauc_precision_at_20_std
value: 56.12175175268179
- type: nauc_precision_at_3_diff1
value: -39.83230143856686
- type: nauc_precision_at_3_max
value: 49.37218197862474
- type: nauc_precision_at_3_std
value: 52.44508941596364
- type: nauc_precision_at_5_diff1
value: -38.90215955262043
- type: nauc_precision_at_5_max
value: 39.46762907461061
- type: nauc_precision_at_5_std
value: 52.15472117048514
- type: nauc_recall_at_1000_diff1
value: -2.390269278740673
- type: nauc_recall_at_1000_max
value: 66.71899600138589
- type: nauc_recall_at_1000_std
value: 72.38413969341725
- type: nauc_recall_at_100_diff1
value: -8.264412300123023
- type: nauc_recall_at_100_max
value: 57.09600612836173
- type: nauc_recall_at_100_std
value: 54.24820629311573
- type: nauc_recall_at_10_diff1
value: 9.033164187895029
- type: nauc_recall_at_10_max
value: 43.95235868275862
- type: nauc_recall_at_10_std
value: 7.664503177750303
- type: nauc_recall_at_1_diff1
value: 46.671939051209534
- type: nauc_recall_at_1_max
value: -10.343353582447454
- type: nauc_recall_at_1_std
value: -37.000769458641614
- type: nauc_recall_at_20_diff1
value: 1.9007241787060192
- type: nauc_recall_at_20_max
value: 51.258475649979495
- type: nauc_recall_at_20_std
value: 29.595747854574945
- type: nauc_recall_at_3_diff1
value: 27.588185692233928
- type: nauc_recall_at_3_max
value: 1.1715811977831723
- type: nauc_recall_at_3_std
value: -33.543961140123386
- type: nauc_recall_at_5_diff1
value: 20.108598449638585
- type: nauc_recall_at_5_max
value: 16.29392148870603
- type: nauc_recall_at_5_std
value: -23.22203273878015
- type: ndcg_at_1
value: 91.8
- type: ndcg_at_10
value: 89.092
- type: ndcg_at_100
value: 91.303
- type: ndcg_at_1000
value: 91.685
- type: ndcg_at_20
value: 90.295
- type: ndcg_at_3
value: 88.239
- type: ndcg_at_5
value: 87.17
- type: precision_at_1
value: 91.8
- type: precision_at_10
value: 42.445
- type: precision_at_100
value: 4.796
- type: precision_at_1000
value: 0.48900000000000005
- type: precision_at_20
value: 22.865
- type: precision_at_3
value: 79.133
- type: precision_at_5
value: 66.83
- type: recall_at_1
value: 26.840999999999998
- type: recall_at_10
value: 89.911
- type: recall_at_100
value: 97.467
- type: recall_at_1000
value: 99.432
- type: recall_at_20
value: 94.21300000000001
- type: recall_at_3
value: 59.136
- type: recall_at_5
value: 76.42999999999999
task:
type: Retrieval
- dataset:
config: default
name: MTEB EcomRetrieval (default)
revision: 687de13dc7294d6fd9be10c6945f9e8fec8166b9
split: test
type: C-MTEB/EcomRetrieval
metrics:
- type: main_score
value: 66.321
- type: map_at_1
value: 52
- type: map_at_10
value: 61.38400000000001
- type: map_at_100
value: 61.936
- type: map_at_1000
value: 61.948
- type: map_at_20
value: 61.748999999999995
- type: map_at_3
value: 58.95
- type: map_at_5
value: 60.33500000000001
- type: mrr_at_1
value: 52
- type: mrr_at_10
value: 61.383730158730145
- type: mrr_at_100
value: 61.93550577147795
- type: mrr_at_1000
value: 61.94847623773313
- type: mrr_at_20
value: 61.74938091707053
- type: mrr_at_3
value: 58.94999999999999
- type: mrr_at_5
value: 60.33499999999998
- type: nauc_map_at_1000_diff1
value: 69.53815055633056
- type: nauc_map_at_1000_max
value: 30.458633171853865
- type: nauc_map_at_1000_std
value: -14.190281565834498
- type: nauc_map_at_100_diff1
value: 69.52712172331074
- type: nauc_map_at_100_max
value: 30.463227838546892
- type: nauc_map_at_100_std
value: -14.184140431293416
- type: nauc_map_at_10_diff1
value: 69.32352246595482
- type: nauc_map_at_10_max
value: 30.546566692369453
- type: nauc_map_at_10_std
value: -14.315486822308543
- type: nauc_map_at_1_diff1
value: 74.32463681375484
- type: nauc_map_at_1_max
value: 26.609037488437888
- type: nauc_map_at_1_std
value: -17.98411230208391
- type: nauc_map_at_20_diff1
value: 69.48043886595383
- type: nauc_map_at_20_max
value: 30.46331735615358
- type: nauc_map_at_20_std
value: -14.250220216410959
- type: nauc_map_at_3_diff1
value: 69.6166158921783
- type: nauc_map_at_3_max
value: 29.289000321401637
- type: nauc_map_at_3_std
value: -15.631322693470228
- type: nauc_map_at_5_diff1
value: 69.15996765075326
- type: nauc_map_at_5_max
value: 29.83809759513448
- type: nauc_map_at_5_std
value: -14.639993334641666
- type: nauc_mrr_at_1000_diff1
value: 69.53815055633056
- type: nauc_mrr_at_1000_max
value: 30.458633171853865
- type: nauc_mrr_at_1000_std
value: -14.190281565834498
- type: nauc_mrr_at_100_diff1
value: 69.52712172331074
- type: nauc_mrr_at_100_max
value: 30.463227838546892
- type: nauc_mrr_at_100_std
value: -14.184140431293416
- type: nauc_mrr_at_10_diff1
value: 69.32352246595482
- type: nauc_mrr_at_10_max
value: 30.546566692369453
- type: nauc_mrr_at_10_std
value: -14.315486822308543
- type: nauc_mrr_at_1_diff1
value: 74.32463681375484
- type: nauc_mrr_at_1_max
value: 26.609037488437888
- type: nauc_mrr_at_1_std
value: -17.98411230208391
- type: nauc_mrr_at_20_diff1
value: 69.48043886595383
- type: nauc_mrr_at_20_max
value: 30.46331735615358
- type: nauc_mrr_at_20_std
value: -14.250220216410959
- type: nauc_mrr_at_3_diff1
value: 69.6166158921783
- type: nauc_mrr_at_3_max
value: 29.289000321401637
- type: nauc_mrr_at_3_std
value: -15.631322693470228
- type: nauc_mrr_at_5_diff1
value: 69.15996765075326
- type: nauc_mrr_at_5_max
value: 29.83809759513448
- type: nauc_mrr_at_5_std
value: -14.639993334641666
- type: nauc_ndcg_at_1000_diff1
value: 68.6001470182718
- type: nauc_ndcg_at_1000_max
value: 32.862663771581296
- type: nauc_ndcg_at_1000_std
value: -10.904093603734804
- type: nauc_ndcg_at_100_diff1
value: 68.31379960767023
- type: nauc_ndcg_at_100_max
value: 33.2277234522222
- type: nauc_ndcg_at_100_std
value: -10.42791128471343
- type: nauc_ndcg_at_10_diff1
value: 67.37575260754308
- type: nauc_ndcg_at_10_max
value: 33.34555324598902
- type: nauc_ndcg_at_10_std
value: -11.541389056105334
- type: nauc_ndcg_at_1_diff1
value: 74.32463681375484
- type: nauc_ndcg_at_1_max
value: 26.609037488437888
- type: nauc_ndcg_at_1_std
value: -17.98411230208391
- type: nauc_ndcg_at_20_diff1
value: 67.91992957951669
- type: nauc_ndcg_at_20_max
value: 33.15985844275247
- type: nauc_ndcg_at_20_std
value: -10.959778392529978
- type: nauc_ndcg_at_3_diff1
value: 68.0758452291291
- type: nauc_ndcg_at_3_max
value: 30.278684617897234
- type: nauc_ndcg_at_3_std
value: -14.52465256831778
- type: nauc_ndcg_at_5_diff1
value: 67.11997361829751
- type: nauc_ndcg_at_5_max
value: 31.395042783624017
- type: nauc_ndcg_at_5_std
value: -12.570510995741401
- type: nauc_precision_at_1000_diff1
value: 63.28042328042302
- type: nauc_precision_at_1000_max
value: 92.55524431994967
- type: nauc_precision_at_1000_std
value: 80.93370681605869
- type: nauc_precision_at_100_diff1
value: 57.286803610333095
- type: nauc_precision_at_100_max
value: 76.19514472455643
- type: nauc_precision_at_100_std
value: 52.6011515717397
- type: nauc_precision_at_10_diff1
value: 57.55534750523178
- type: nauc_precision_at_10_max
value: 48.96602048683773
- type: nauc_precision_at_10_std
value: 3.7548188126445106
- type: nauc_precision_at_1_diff1
value: 74.32463681375484
- type: nauc_precision_at_1_max
value: 26.609037488437888
- type: nauc_precision_at_1_std
value: -17.98411230208391
- type: nauc_precision_at_20_diff1
value: 58.225383213472746
- type: nauc_precision_at_20_max
value: 53.45024463652369
- type: nauc_precision_at_20_std
value: 14.984081611046667
- type: nauc_precision_at_3_diff1
value: 62.9982742182692
- type: nauc_precision_at_3_max
value: 33.58360903888962
- type: nauc_precision_at_3_std
value: -10.74777553458719
- type: nauc_precision_at_5_diff1
value: 59.22737266435213
- type: nauc_precision_at_5_max
value: 37.562770377873
- type: nauc_precision_at_5_std
value: -4.205437353688412
- type: nauc_recall_at_1000_diff1
value: 63.280423280423626
- type: nauc_recall_at_1000_max
value: 92.55524431995006
- type: nauc_recall_at_1000_std
value: 80.9337068160594
- type: nauc_recall_at_100_diff1
value: 57.28680361033288
- type: nauc_recall_at_100_max
value: 76.19514472455649
- type: nauc_recall_at_100_std
value: 52.60115157173969
- type: nauc_recall_at_10_diff1
value: 57.55534750523186
- type: nauc_recall_at_10_max
value: 48.9660204868377
- type: nauc_recall_at_10_std
value: 3.754818812644508
- type: nauc_recall_at_1_diff1
value: 74.32463681375484
- type: nauc_recall_at_1_max
value: 26.609037488437888
- type: nauc_recall_at_1_std
value: -17.98411230208391
- type: nauc_recall_at_20_diff1
value: 58.22538321347268
- type: nauc_recall_at_20_max
value: 53.450244636523806
- type: nauc_recall_at_20_std
value: 14.984081611046815
- type: nauc_recall_at_3_diff1
value: 62.99827421826917
- type: nauc_recall_at_3_max
value: 33.58360903888954
- type: nauc_recall_at_3_std
value: -10.747775534587227
- type: nauc_recall_at_5_diff1
value: 59.22737266435212
- type: nauc_recall_at_5_max
value: 37.562770377873136
- type: nauc_recall_at_5_std
value: -4.205437353688367
- type: ndcg_at_1
value: 52
- type: ndcg_at_10
value: 66.321
- type: ndcg_at_100
value: 68.928
- type: ndcg_at_1000
value: 69.303
- type: ndcg_at_20
value: 67.659
- type: ndcg_at_3
value: 61.246
- type: ndcg_at_5
value: 63.772
- type: precision_at_1
value: 52
- type: precision_at_10
value: 8.200000000000001
- type: precision_at_100
value: 0.9400000000000001
- type: precision_at_1000
value: 0.097
- type: precision_at_20
value: 4.365
- type: precision_at_3
value: 22.633
- type: precision_at_5
value: 14.82
- type: recall_at_1
value: 52
- type: recall_at_10
value: 82
- type: recall_at_100
value: 94
- type: recall_at_1000
value: 97
- type: recall_at_20
value: 87.3
- type: recall_at_3
value: 67.9
- type: recall_at_5
value: 74.1
task:
type: Retrieval
- dataset:
config: default
name: MTEB FEVER (default)
revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
split: test
type: mteb/fever
metrics:
- type: main_score
value: 90.51
- type: map_at_1
value: 79.957
- type: map_at_10
value: 87.42
- type: map_at_100
value: 87.572
- type: map_at_1000
value: 87.583
- type: map_at_20
value: 87.509
- type: map_at_3
value: 86.5
- type: map_at_5
value: 87.101
- type: mrr_at_1
value: 86.15361536153615
- type: mrr_at_10
value: 91.73240538339532
- type: mrr_at_100
value: 91.75016165436173
- type: mrr_at_1000
value: 91.75072714764477
- type: mrr_at_20
value: 91.74440753331851
- type: mrr_at_3
value: 91.29162916291617
- type: mrr_at_5
value: 91.61341134113393
- type: nauc_map_at_1000_diff1
value: 55.62747866315302
- type: nauc_map_at_1000_max
value: 23.544790223399655
- type: nauc_map_at_1000_std
value: -5.579174609060041
- type: nauc_map_at_100_diff1
value: 55.589929194255774
- type: nauc_map_at_100_max
value: 23.54032170624126
- type: nauc_map_at_100_std
value: -5.560250864299321
- type: nauc_map_at_10_diff1
value: 55.20193146039033
- type: nauc_map_at_10_max
value: 23.46449956896367
- type: nauc_map_at_10_std
value: -5.479355256959771
- type: nauc_map_at_1_diff1
value: 60.81437797444696
- type: nauc_map_at_1_max
value: 16.21293503927129
- type: nauc_map_at_1_std
value: -11.371544003488534
- type: nauc_map_at_20_diff1
value: 55.39988137889343
- type: nauc_map_at_20_max
value: 23.479788953316085
- type: nauc_map_at_20_std
value: -5.558018257831432
- type: nauc_map_at_3_diff1
value: 55.20272249508823
- type: nauc_map_at_3_max
value: 23.437623610288373
- type: nauc_map_at_3_std
value: -6.446259728275601
- type: nauc_map_at_5_diff1
value: 54.811705968136174
- type: nauc_map_at_5_max
value: 23.53527198979852
- type: nauc_map_at_5_std
value: -5.519017712375851
- type: nauc_mrr_at_1000_diff1
value: 77.09388384236254
- type: nauc_mrr_at_1000_max
value: 24.041983462826703
- type: nauc_mrr_at_1000_std
value: -17.51208378082453
- type: nauc_mrr_at_100_diff1
value: 77.09379302405928
- type: nauc_mrr_at_100_max
value: 24.04676368680233
- type: nauc_mrr_at_100_std
value: -17.50584210817013
- type: nauc_mrr_at_10_diff1
value: 77.07416710284788
- type: nauc_mrr_at_10_max
value: 24.16468273355217
- type: nauc_mrr_at_10_std
value: -17.41320387711184
- type: nauc_mrr_at_1_diff1
value: 77.80612385762478
- type: nauc_mrr_at_1_max
value: 18.798274746696624
- type: nauc_mrr_at_1_std
value: -19.812320477907612
- type: nauc_mrr_at_20_diff1
value: 77.08185932106402
- type: nauc_mrr_at_20_max
value: 24.09328840875767
- type: nauc_mrr_at_20_std
value: -17.45879512580195
- type: nauc_mrr_at_3_diff1
value: 77.01593602357202
- type: nauc_mrr_at_3_max
value: 25.190875004233217
- type: nauc_mrr_at_3_std
value: -17.72250952117334
- type: nauc_mrr_at_5_diff1
value: 77.01761563326805
- type: nauc_mrr_at_5_max
value: 24.757066673211142
- type: nauc_mrr_at_5_std
value: -17.19951314225811
- type: nauc_ndcg_at_1000_diff1
value: 57.8494260162157
- type: nauc_ndcg_at_1000_max
value: 25.34017867497563
- type: nauc_ndcg_at_1000_std
value: -4.9971835073125295
- type: nauc_ndcg_at_100_diff1
value: 56.88886047109567
- type: nauc_ndcg_at_100_max
value: 25.34585159509186
- type: nauc_ndcg_at_100_std
value: -4.396088754530463
- type: nauc_ndcg_at_10_diff1
value: 55.39706710366615
- type: nauc_ndcg_at_10_max
value: 25.23366894651518
- type: nauc_ndcg_at_10_std
value: -4.02051245272735
- type: nauc_ndcg_at_1_diff1
value: 77.80612385762478
- type: nauc_ndcg_at_1_max
value: 18.798274746696624
- type: nauc_ndcg_at_1_std
value: -19.812320477907612
- type: nauc_ndcg_at_20_diff1
value: 55.90944611273084
- type: nauc_ndcg_at_20_max
value: 25.218359231514103
- type: nauc_ndcg_at_20_std
value: -4.195454512225829
- type: nauc_ndcg_at_3_diff1
value: 56.900264881158826
- type: nauc_ndcg_at_3_max
value: 26.11057832808819
- type: nauc_ndcg_at_3_std
value: -6.429746036887465
- type: nauc_ndcg_at_5_diff1
value: 55.20032432540728
- type: nauc_ndcg_at_5_max
value: 25.905242282953328
- type: nauc_ndcg_at_5_std
value: -4.237294390432071
- type: nauc_precision_at_1000_diff1
value: -10.72073093313023
- type: nauc_precision_at_1000_max
value: -1.3010305287914616
- type: nauc_precision_at_1000_std
value: 1.7882934955205159
- type: nauc_precision_at_100_diff1
value: -12.173285712757355
- type: nauc_precision_at_100_max
value: 1.4279807975435195
- type: nauc_precision_at_100_std
value: 5.202388521512146
- type: nauc_precision_at_10_diff1
value: -13.072275371151196
- type: nauc_precision_at_10_max
value: 6.229597015121936
- type: nauc_precision_at_10_std
value: 7.9298978727659355
- type: nauc_precision_at_1_diff1
value: 77.80612385762478
- type: nauc_precision_at_1_max
value: 18.798274746696624
- type: nauc_precision_at_1_std
value: -19.812320477907612
- type: nauc_precision_at_20_diff1
value: -13.19212302010048
- type: nauc_precision_at_20_max
value: 3.809358294404054
- type: nauc_precision_at_20_std
value: 6.136483332491836
- type: nauc_precision_at_3_diff1
value: 6.88040425238454
- type: nauc_precision_at_3_max
value: 21.230050942241586
- type: nauc_precision_at_3_std
value: 3.6113059937245726
- type: nauc_precision_at_5_diff1
value: -7.926656002659219
- type: nauc_precision_at_5_max
value: 13.728985807916457
- type: nauc_precision_at_5_std
value: 9.104406648204018
- type: nauc_recall_at_1000_diff1
value: 6.826634892317323
- type: nauc_recall_at_1000_max
value: 47.461957880242785
- type: nauc_recall_at_1000_std
value: 51.6613432996783
- type: nauc_recall_at_100_diff1
value: 11.236324687996389
- type: nauc_recall_at_100_max
value: 38.127691674159934
- type: nauc_recall_at_100_std
value: 38.353731224893984
- type: nauc_recall_at_10_diff1
value: 17.925743881123733
- type: nauc_recall_at_10_max
value: 33.185408322658816
- type: nauc_recall_at_10_std
value: 25.085133063767085
- type: nauc_recall_at_1_diff1
value: 60.81437797444696
- type: nauc_recall_at_1_max
value: 16.21293503927129
- type: nauc_recall_at_1_std
value: -11.371544003488534
- type: nauc_recall_at_20_diff1
value: 15.381672516888978
- type: nauc_recall_at_20_max
value: 34.01502923316828
- type: nauc_recall_at_20_std
value: 28.67331265878902
- type: nauc_recall_at_3_diff1
value: 35.65247847623485
- type: nauc_recall_at_3_max
value: 32.105152773899285
- type: nauc_recall_at_3_std
value: 5.912621293073229
- type: nauc_recall_at_5_diff1
value: 25.114222723712746
- type: nauc_recall_at_5_max
value: 33.665783755502844
- type: nauc_recall_at_5_std
value: 16.756592637190636
- type: ndcg_at_1
value: 86.154
- type: ndcg_at_10
value: 90.51
- type: ndcg_at_100
value: 90.981
- type: ndcg_at_1000
value: 91.178
- type: ndcg_at_20
value: 90.70700000000001
- type: ndcg_at_3
value: 89.25699999999999
- type: ndcg_at_5
value: 89.988
- type: precision_at_1
value: 86.154
- type: precision_at_10
value: 10.695
- type: precision_at_100
value: 1.115
- type: precision_at_1000
value: 0.11499999999999999
- type: precision_at_20
value: 5.426
- type: precision_at_3
value: 33.788000000000004
- type: precision_at_5
value: 20.846
- type: recall_at_1
value: 79.957
- type: recall_at_10
value: 95.724
- type: recall_at_100
value: 97.444
- type: recall_at_1000
value: 98.64699999999999
- type: recall_at_20
value: 96.324
- type: recall_at_3
value: 92.281
- type: recall_at_5
value: 94.204
task:
type: Retrieval
- dataset:
config: default
name: MTEB FiQA2018 (default)
revision: 27a168819829fe9bcd655c2df245fb19452e8e06
split: test
type: mteb/fiqa
metrics:
- type: main_score
value: 47.514
- type: map_at_1
value: 24.645
- type: map_at_10
value: 39.352
- type: map_at_100
value: 41.22
- type: map_at_1000
value: 41.377
- type: map_at_20
value: 40.355000000000004
- type: map_at_3
value: 34.075
- type: map_at_5
value: 37.108000000000004
- type: mrr_at_1
value: 47.0679012345679
- type: mrr_at_10
value: 55.40882814030964
- type: mrr_at_100
value: 56.12125106952315
- type: mrr_at_1000
value: 56.15387132263201
- type: mrr_at_20
value: 55.8948061625507
- type: mrr_at_3
value: 52.95781893004117
- type: mrr_at_5
value: 54.28497942386829
- type: nauc_map_at_1000_diff1
value: 44.77272346288289
- type: nauc_map_at_1000_max
value: 36.09407349481673
- type: nauc_map_at_1000_std
value: -1.387935841299599
- type: nauc_map_at_100_diff1
value: 44.723556943431234
- type: nauc_map_at_100_max
value: 35.99093730191303
- type: nauc_map_at_100_std
value: -1.4074999846573228
- type: nauc_map_at_10_diff1
value: 44.391340020713855
- type: nauc_map_at_10_max
value: 34.602517259403506
- type: nauc_map_at_10_std
value: -2.3790851307330625
- type: nauc_map_at_1_diff1
value: 50.105829266519244
- type: nauc_map_at_1_max
value: 24.152912506739803
- type: nauc_map_at_1_std
value: -5.27506525468281
- type: nauc_map_at_20_diff1
value: 44.50379935527974
- type: nauc_map_at_20_max
value: 35.348069309685776
- type: nauc_map_at_20_std
value: -1.709571532659604
- type: nauc_map_at_3_diff1
value: 45.670369127791915
- type: nauc_map_at_3_max
value: 30.831452903193163
- type: nauc_map_at_3_std
value: -4.436677865346682
- type: nauc_map_at_5_diff1
value: 45.17911631909283
- type: nauc_map_at_5_max
value: 33.306423021308134
- type: nauc_map_at_5_std
value: -4.031363774995068
- type: nauc_mrr_at_1000_diff1
value: 52.61628096939053
- type: nauc_mrr_at_1000_max
value: 45.71319446022708
- type: nauc_mrr_at_1000_std
value: -0.20437591300743238
- type: nauc_mrr_at_100_diff1
value: 52.59036374639475
- type: nauc_mrr_at_100_max
value: 45.72743514492137
- type: nauc_mrr_at_100_std
value: -0.18457663920869963
- type: nauc_mrr_at_10_diff1
value: 52.45956212546747
- type: nauc_mrr_at_10_max
value: 45.61538981845574
- type: nauc_mrr_at_10_std
value: -0.45293760316300996
- type: nauc_mrr_at_1_diff1
value: 56.626080862506356
- type: nauc_mrr_at_1_max
value: 45.034721250577874
- type: nauc_mrr_at_1_std
value: -2.6493217245134795
- type: nauc_mrr_at_20_diff1
value: 52.52240442485512
- type: nauc_mrr_at_20_max
value: 45.67194818657121
- type: nauc_mrr_at_20_std
value: -0.19023089137780325
- type: nauc_mrr_at_3_diff1
value: 53.018265225297235
- type: nauc_mrr_at_3_max
value: 45.99369691850936
- type: nauc_mrr_at_3_std
value: -0.5705271234961398
- type: nauc_mrr_at_5_diff1
value: 52.778126086680864
- type: nauc_mrr_at_5_max
value: 45.63643961349299
- type: nauc_mrr_at_5_std
value: -0.8259675580429905
- type: nauc_ndcg_at_1000_diff1
value: 45.90373831741528
- type: nauc_ndcg_at_1000_max
value: 41.35017558325094
- type: nauc_ndcg_at_1000_std
value: 2.096012035567072
- type: nauc_ndcg_at_100_diff1
value: 45.027446346957284
- type: nauc_ndcg_at_100_max
value: 40.605465687346175
- type: nauc_ndcg_at_100_std
value: 2.4038882149653276
- type: nauc_ndcg_at_10_diff1
value: 44.074735693463325
- type: nauc_ndcg_at_10_max
value: 37.46233357867317
- type: nauc_ndcg_at_10_std
value: -0.4475709952502303
- type: nauc_ndcg_at_1_diff1
value: 56.626080862506356
- type: nauc_ndcg_at_1_max
value: 45.034721250577874
- type: nauc_ndcg_at_1_std
value: -2.6493217245134795
- type: nauc_ndcg_at_20_diff1
value: 44.31817600347284
- type: nauc_ndcg_at_20_max
value: 38.476688485181185
- type: nauc_ndcg_at_20_std
value: 1.117991976066639
- type: nauc_ndcg_at_3_diff1
value: 46.20974876099122
- type: nauc_ndcg_at_3_max
value: 39.67931470812708
- type: nauc_ndcg_at_3_std
value: -2.5706495888606065
- type: nauc_ndcg_at_5_diff1
value: 45.325044209858376
- type: nauc_ndcg_at_5_max
value: 37.91199035928747
- type: nauc_ndcg_at_5_std
value: -2.6517106051896415
- type: nauc_precision_at_1000_diff1
value: -0.9537663597844245
- type: nauc_precision_at_1000_max
value: 27.761641710876805
- type: nauc_precision_at_1000_std
value: 10.06076340740224
- type: nauc_precision_at_100_diff1
value: 4.898859142232583
- type: nauc_precision_at_100_max
value: 33.62244081851389
- type: nauc_precision_at_100_std
value: 12.958070332981025
- type: nauc_precision_at_10_diff1
value: 13.911881718581084
- type: nauc_precision_at_10_max
value: 39.06687486946156
- type: nauc_precision_at_10_std
value: 8.891111414659303
- type: nauc_precision_at_1_diff1
value: 56.626080862506356
- type: nauc_precision_at_1_max
value: 45.034721250577874
- type: nauc_precision_at_1_std
value: -2.6493217245134795
- type: nauc_precision_at_20_diff1
value: 10.775097513378254
- type: nauc_precision_at_20_max
value: 37.65460968883382
- type: nauc_precision_at_20_std
value: 11.889915451399668
- type: nauc_precision_at_3_diff1
value: 28.30096121403731
- type: nauc_precision_at_3_max
value: 42.40913904542612
- type: nauc_precision_at_3_std
value: 1.2961939760329662
- type: nauc_precision_at_5_diff1
value: 22.29912026851874
- type: nauc_precision_at_5_max
value: 41.214493600606104
- type: nauc_precision_at_5_std
value: 3.265899775196082
- type: nauc_recall_at_1000_diff1
value: 30.334475310610348
- type: nauc_recall_at_1000_max
value: 46.44256605093095
- type: nauc_recall_at_1000_std
value: 37.4195772516604
- type: nauc_recall_at_100_diff1
value: 26.949261821988298
- type: nauc_recall_at_100_max
value: 35.73869214813791
- type: nauc_recall_at_100_std
value: 18.21976935233426
- type: nauc_recall_at_10_diff1
value: 31.804710246288064
- type: nauc_recall_at_10_max
value: 28.494462228237317
- type: nauc_recall_at_10_std
value: 2.0918304228972238
- type: nauc_recall_at_1_diff1
value: 50.105829266519244
- type: nauc_recall_at_1_max
value: 24.152912506739803
- type: nauc_recall_at_1_std
value: -5.27506525468281
- type: nauc_recall_at_20_diff1
value: 29.604091813286402
- type: nauc_recall_at_20_max
value: 28.97789764511831
- type: nauc_recall_at_20_std
value: 7.039137680276673
- type: nauc_recall_at_3_diff1
value: 40.10999098029199
- type: nauc_recall_at_3_max
value: 28.584637753129964
- type: nauc_recall_at_3_std
value: -3.135936497604714
- type: nauc_recall_at_5_diff1
value: 36.70967721515207
- type: nauc_recall_at_5_max
value: 28.719731279382298
- type: nauc_recall_at_5_std
value: -3.039387759442582
- type: ndcg_at_1
value: 47.068
- type: ndcg_at_10
value: 47.514
- type: ndcg_at_100
value: 53.991
- type: ndcg_at_1000
value: 56.513000000000005
- type: ndcg_at_20
value: 50.092000000000006
- type: ndcg_at_3
value: 43.147000000000006
- type: ndcg_at_5
value: 44.675
- type: precision_at_1
value: 47.068
- type: precision_at_10
value: 13.148000000000001
- type: precision_at_100
value: 1.9980000000000002
- type: precision_at_1000
value: 0.244
- type: precision_at_20
value: 7.6770000000000005
- type: precision_at_3
value: 28.395
- type: precision_at_5
value: 21.204
- type: recall_at_1
value: 24.645
- type: recall_at_10
value: 54.715
- type: recall_at_100
value: 78.674
- type: recall_at_1000
value: 93.60199999999999
- type: recall_at_20
value: 62.81099999999999
- type: recall_at_3
value: 38.580999999999996
- type: recall_at_5
value: 45.727000000000004
task:
type: Retrieval
- dataset:
config: default
name: MTEB HotpotQA (default)
revision: ab518f4d6fcca38d87c25209f94beba119d02014
split: test
type: mteb/hotpotqa
metrics:
- type: main_score
value: 75.32600000000001
- type: map_at_1
value: 42.093
- type: map_at_10
value: 67.487
- type: map_at_100
value: 68.24900000000001
- type: map_at_1000
value: 68.29899999999999
- type: map_at_20
value: 67.93900000000001
- type: map_at_3
value: 64.193
- type: map_at_5
value: 66.328
- type: mrr_at_1
value: 84.18636056718434
- type: mrr_at_10
value: 88.96446523691603
- type: mrr_at_100
value: 89.07749107714154
- type: mrr_at_1000
value: 89.0802400764323
- type: mrr_at_20
value: 89.03693404881906
- type: mrr_at_3
value: 88.35021381949115
- type: mrr_at_5
value: 88.76817465676321
- type: nauc_map_at_1000_diff1
value: 9.995319980546128
- type: nauc_map_at_1000_max
value: 15.843115548360199
- type: nauc_map_at_1000_std
value: 5.286741865550291
- type: nauc_map_at_100_diff1
value: 9.965997966056612
- type: nauc_map_at_100_max
value: 15.832340410378018
- type: nauc_map_at_100_std
value: 5.31003490395493
- type: nauc_map_at_10_diff1
value: 9.75529136863581
- type: nauc_map_at_10_max
value: 15.5004959623361
- type: nauc_map_at_10_std
value: 4.734637872591846
- type: nauc_map_at_1_diff1
value: 73.30294723315882
- type: nauc_map_at_1_max
value: 43.66329775084331
- type: nauc_map_at_1_std
value: -7.67788750420626
- type: nauc_map_at_20_diff1
value: 9.779985336264675
- type: nauc_map_at_20_max
value: 15.673614982634346
- type: nauc_map_at_20_std
value: 5.129436201234171
- type: nauc_map_at_3_diff1
value: 9.893652511517288
- type: nauc_map_at_3_max
value: 14.582702430192729
- type: nauc_map_at_3_std
value: 2.035001467791114
- type: nauc_map_at_5_diff1
value: 9.697760587575981
- type: nauc_map_at_5_max
value: 15.21430699967373
- type: nauc_map_at_5_std
value: 3.5955115009020933
- type: nauc_mrr_at_1000_diff1
value: 73.11748626397122
- type: nauc_mrr_at_1000_max
value: 46.73528373640579
- type: nauc_mrr_at_1000_std
value: -5.970370775969118
- type: nauc_mrr_at_100_diff1
value: 73.11880113030185
- type: nauc_mrr_at_100_max
value: 46.74380443058361
- type: nauc_mrr_at_100_std
value: -5.9578304093828205
- type: nauc_mrr_at_10_diff1
value: 73.12886006500067
- type: nauc_mrr_at_10_max
value: 46.767758336745416
- type: nauc_mrr_at_10_std
value: -5.991825690006404
- type: nauc_mrr_at_1_diff1
value: 73.30294723315882
- type: nauc_mrr_at_1_max
value: 43.66329775084331
- type: nauc_mrr_at_1_std
value: -7.67788750420626
- type: nauc_mrr_at_20_diff1
value: 73.13899139910129
- type: nauc_mrr_at_20_max
value: 46.787791460797514
- type: nauc_mrr_at_20_std
value: -5.935002393245029
- type: nauc_mrr_at_3_diff1
value: 72.98715491258669
- type: nauc_mrr_at_3_max
value: 47.21249528414659
- type: nauc_mrr_at_3_std
value: -6.247995601535651
- type: nauc_mrr_at_5_diff1
value: 73.06864822495608
- type: nauc_mrr_at_5_max
value: 46.88005515702291
- type: nauc_mrr_at_5_std
value: -6.032634039363137
- type: nauc_ndcg_at_1000_diff1
value: 16.60411769086399
- type: nauc_ndcg_at_1000_max
value: 20.75198311077535
- type: nauc_ndcg_at_1000_std
value: 8.13237736475886
- type: nauc_ndcg_at_100_diff1
value: 15.72911678935216
- type: nauc_ndcg_at_100_max
value: 20.47072334762776
- type: nauc_ndcg_at_100_std
value: 8.782193310979013
- type: nauc_ndcg_at_10_diff1
value: 14.772128790148878
- type: nauc_ndcg_at_10_max
value: 19.077954955081136
- type: nauc_ndcg_at_10_std
value: 6.284782593325895
- type: nauc_ndcg_at_1_diff1
value: 73.30294723315882
- type: nauc_ndcg_at_1_max
value: 43.66329775084331
- type: nauc_ndcg_at_1_std
value: -7.67788750420626
- type: nauc_ndcg_at_20_diff1
value: 14.752399154683435
- type: nauc_ndcg_at_20_max
value: 19.55715062684391
- type: nauc_ndcg_at_20_std
value: 7.573384824307518
- type: nauc_ndcg_at_3_diff1
value: 15.56449232913491
- type: nauc_ndcg_at_3_max
value: 18.182333510364554
- type: nauc_ndcg_at_3_std
value: 1.9272410761288006
- type: nauc_ndcg_at_5_diff1
value: 14.884941140343921
- type: nauc_ndcg_at_5_max
value: 18.68777570103015
- type: nauc_ndcg_at_5_std
value: 4.0935157857334135
- type: nauc_precision_at_1000_diff1
value: -10.878725068314541
- type: nauc_precision_at_1000_max
value: 18.544493538408346
- type: nauc_precision_at_1000_std
value: 44.73746132705706
- type: nauc_precision_at_100_diff1
value: -6.782374787932728
- type: nauc_precision_at_100_max
value: 16.721047366440693
- type: nauc_precision_at_100_std
value: 34.190413143432174
- type: nauc_precision_at_10_diff1
value: -2.58056116443468
- type: nauc_precision_at_10_max
value: 12.545200150514491
- type: nauc_precision_at_10_std
value: 15.18383050468422
- type: nauc_precision_at_1_diff1
value: 73.30294723315882
- type: nauc_precision_at_1_max
value: 43.66329775084331
- type: nauc_precision_at_1_std
value: -7.67788750420626
- type: nauc_precision_at_20_diff1
value: -4.900789620284776
- type: nauc_precision_at_20_max
value: 13.422571163066484
- type: nauc_precision_at_20_std
value: 21.035079923229627
- type: nauc_precision_at_3_diff1
value: 2.3940138426298976
- type: nauc_precision_at_3_max
value: 12.500482474979865
- type: nauc_precision_at_3_std
value: 4.925125401965679
- type: nauc_precision_at_5_diff1
value: -0.20414348059352502
- type: nauc_precision_at_5_max
value: 12.617438217714346
- type: nauc_precision_at_5_std
value: 9.093888696691772
- type: nauc_recall_at_1000_diff1
value: -10.878725068314326
- type: nauc_recall_at_1000_max
value: 18.544493538408393
- type: nauc_recall_at_1000_std
value: 44.73746132705682
- type: nauc_recall_at_100_diff1
value: -6.782374787932829
- type: nauc_recall_at_100_max
value: 16.721047366440786
- type: nauc_recall_at_100_std
value: 34.190413143432046
- type: nauc_recall_at_10_diff1
value: -2.5805611644346587
- type: nauc_recall_at_10_max
value: 12.545200150514441
- type: nauc_recall_at_10_std
value: 15.183830504684334
- type: nauc_recall_at_1_diff1
value: 73.30294723315882
- type: nauc_recall_at_1_max
value: 43.66329775084331
- type: nauc_recall_at_1_std
value: -7.67788750420626
- type: nauc_recall_at_20_diff1
value: -4.9007896202848125
- type: nauc_recall_at_20_max
value: 13.422571163066511
- type: nauc_recall_at_20_std
value: 21.035079923229684
- type: nauc_recall_at_3_diff1
value: 2.394013842629748
- type: nauc_recall_at_3_max
value: 12.500482474979735
- type: nauc_recall_at_3_std
value: 4.925125401965642
- type: nauc_recall_at_5_diff1
value: -0.2041434805933923
- type: nauc_recall_at_5_max
value: 12.61743821771437
- type: nauc_recall_at_5_std
value: 9.09388869669173
- type: ndcg_at_1
value: 84.186
- type: ndcg_at_10
value: 75.32600000000001
- type: ndcg_at_100
value: 77.874
- type: ndcg_at_1000
value: 78.806
- type: ndcg_at_20
value: 76.40400000000001
- type: ndcg_at_3
value: 70.909
- type: ndcg_at_5
value: 73.479
- type: precision_at_1
value: 84.186
- type: precision_at_10
value: 15.587000000000002
- type: precision_at_100
value: 1.757
- type: precision_at_1000
value: 0.188
- type: precision_at_20
value: 8.139000000000001
- type: precision_at_3
value: 45.51
- type: precision_at_5
value: 29.326
- type: recall_at_1
value: 42.093
- type: recall_at_10
value: 77.934
- type: recall_at_100
value: 87.86
- type: recall_at_1000
value: 94.024
- type: recall_at_20
value: 81.391
- type: recall_at_3
value: 68.265
- type: recall_at_5
value: 73.315
task:
type: Retrieval
- dataset:
config: default
name: MTEB MMarcoRetrieval (default)
revision: 539bbde593d947e2a124ba72651aafc09eb33fc2
split: test
type: C-MTEB/MMarcoRetrieval
metrics:
- type: main_score
value: 81.326
- type: map_at_1
value: 68.989
- type: map_at_10
value: 77.847
- type: map_at_100
value: 78.156
- type: map_at_1000
value: 78.164
- type: map_at_20
value: 78.061
- type: map_at_3
value: 76.13799999999999
- type: map_at_5
value: 77.229
- type: mrr_at_1
value: 71.21776504297995
- type: mrr_at_10
value: 78.32120684950182
- type: mrr_at_100
value: 78.58936042309043
- type: mrr_at_1000
value: 78.59610252917135
- type: mrr_at_20
value: 78.50335098226165
- type: mrr_at_3
value: 76.86007640878704
- type: mrr_at_5
value: 77.7698185291306
- type: nauc_map_at_1000_diff1
value: 79.16029739870804
- type: nauc_map_at_1000_max
value: 43.78891562548924
- type: nauc_map_at_1000_std
value: -13.246437799707838
- type: nauc_map_at_100_diff1
value: 79.15596023648153
- type: nauc_map_at_100_max
value: 43.79965621272441
- type: nauc_map_at_100_std
value: -13.227701618120562
- type: nauc_map_at_10_diff1
value: 79.05928504713239
- type: nauc_map_at_10_max
value: 43.94264840364276
- type: nauc_map_at_10_std
value: -13.161277666070722
- type: nauc_map_at_1_diff1
value: 81.1405902848429
- type: nauc_map_at_1_max
value: 35.44331128528461
- type: nauc_map_at_1_std
value: -20.100450066560107
- type: nauc_map_at_20_diff1
value: 79.11902099348794
- type: nauc_map_at_20_max
value: 43.87508609629538
- type: nauc_map_at_20_std
value: -13.151424214062782
- type: nauc_map_at_3_diff1
value: 79.04969209762379
- type: nauc_map_at_3_max
value: 42.60060629924232
- type: nauc_map_at_3_std
value: -15.308728754602544
- type: nauc_map_at_5_diff1
value: 79.03276369575103
- type: nauc_map_at_5_max
value: 43.751838034034435
- type: nauc_map_at_5_std
value: -13.970955760273384
- type: nauc_mrr_at_1000_diff1
value: 79.61979726371646
- type: nauc_mrr_at_1000_max
value: 44.66719436723058
- type: nauc_mrr_at_1000_std
value: -12.325730966118812
- type: nauc_mrr_at_100_diff1
value: 79.61632982477367
- type: nauc_mrr_at_100_max
value: 44.678855951697926
- type: nauc_mrr_at_100_std
value: -12.306704634602726
- type: nauc_mrr_at_10_diff1
value: 79.51956393477862
- type: nauc_mrr_at_10_max
value: 44.84408594441904
- type: nauc_mrr_at_10_std
value: -12.127272545651286
- type: nauc_mrr_at_1_diff1
value: 81.90501303330416
- type: nauc_mrr_at_1_max
value: 39.97432077076156
- type: nauc_mrr_at_1_std
value: -18.389127950793718
- type: nauc_mrr_at_20_diff1
value: 79.58270388752608
- type: nauc_mrr_at_20_max
value: 44.764311380347536
- type: nauc_mrr_at_20_std
value: -12.19496193710737
- type: nauc_mrr_at_3_diff1
value: 79.60316497236226
- type: nauc_mrr_at_3_max
value: 43.988585000681866
- type: nauc_mrr_at_3_std
value: -13.850905471978848
- type: nauc_mrr_at_5_diff1
value: 79.47331529124322
- type: nauc_mrr_at_5_max
value: 44.71348924741831
- type: nauc_mrr_at_5_std
value: -12.783865115616067
- type: nauc_ndcg_at_1000_diff1
value: 78.78969082437384
- type: nauc_ndcg_at_1000_max
value: 45.81926213521159
- type: nauc_ndcg_at_1000_std
value: -10.022967495470796
- type: nauc_ndcg_at_100_diff1
value: 78.67711818680986
- type: nauc_ndcg_at_100_max
value: 46.25026629853773
- type: nauc_ndcg_at_100_std
value: -9.261589155598827
- type: nauc_ndcg_at_10_diff1
value: 78.22360676709333
- type: nauc_ndcg_at_10_max
value: 47.09475980422957
- type: nauc_ndcg_at_10_std
value: -8.547356995014225
- type: nauc_ndcg_at_1_diff1
value: 81.90501303330416
- type: nauc_ndcg_at_1_max
value: 39.97432077076156
- type: nauc_ndcg_at_1_std
value: -18.389127950793718
- type: nauc_ndcg_at_20_diff1
value: 78.42651077972808
- type: nauc_ndcg_at_20_max
value: 46.854979960167384
- type: nauc_ndcg_at_20_std
value: -8.573604161908804
- type: nauc_ndcg_at_3_diff1
value: 78.35604774445503
- type: nauc_ndcg_at_3_max
value: 44.45828113811858
- type: nauc_ndcg_at_3_std
value: -13.309019113602183
- type: nauc_ndcg_at_5_diff1
value: 78.16411679491445
- type: nauc_ndcg_at_5_max
value: 46.49660148233588
- type: nauc_ndcg_at_5_std
value: -10.808600667753756
- type: nauc_precision_at_1000_diff1
value: -20.82705844414604
- type: nauc_precision_at_1000_max
value: 16.03948538489189
- type: nauc_precision_at_1000_std
value: 27.567999530944252
- type: nauc_precision_at_100_diff1
value: -9.819051662848384
- type: nauc_precision_at_100_max
value: 24.780047792084588
- type: nauc_precision_at_100_std
value: 31.421518287384515
- type: nauc_precision_at_10_diff1
value: 18.177203300248284
- type: nauc_precision_at_10_max
value: 39.22103562502051
- type: nauc_precision_at_10_std
value: 23.546566193049625
- type: nauc_precision_at_1_diff1
value: 81.90501303330416
- type: nauc_precision_at_1_max
value: 39.97432077076156
- type: nauc_precision_at_1_std
value: -18.389127950793718
- type: nauc_precision_at_20_diff1
value: 6.381318855094538
- type: nauc_precision_at_20_max
value: 33.91673607084499
- type: nauc_precision_at_20_std
value: 28.25672869116022
- type: nauc_precision_at_3_diff1
value: 44.27538712834998
- type: nauc_precision_at_3_max
value: 41.809017183415975
- type: nauc_precision_at_3_std
value: 1.9207478473598019
- type: nauc_precision_at_5_diff1
value: 32.00478806248665
- type: nauc_precision_at_5_max
value: 42.07709798215479
- type: nauc_precision_at_5_std
value: 11.409280224561886
- type: nauc_recall_at_1000_diff1
value: 68.67828128136055
- type: nauc_recall_at_1000_max
value: 91.34477778650434
- type: nauc_recall_at_1000_std
value: 78.06367095124621
- type: nauc_recall_at_100_diff1
value: 69.56542856305472
- type: nauc_recall_at_100_max
value: 83.9231178271008
- type: nauc_recall_at_100_std
value: 61.0259051303632
- type: nauc_recall_at_10_diff1
value: 71.27617186532059
- type: nauc_recall_at_10_max
value: 66.52481809588132
- type: nauc_recall_at_10_std
value: 21.855896508801056
- type: nauc_recall_at_1_diff1
value: 81.1405902848429
- type: nauc_recall_at_1_max
value: 35.44331128528461
- type: nauc_recall_at_1_std
value: -20.100450066560107
- type: nauc_recall_at_20_diff1
value: 70.52557301925908
- type: nauc_recall_at_20_max
value: 72.93606618693053
- type: nauc_recall_at_20_std
value: 34.72268440514996
- type: nauc_recall_at_3_diff1
value: 74.80190815622731
- type: nauc_recall_at_3_max
value: 49.32547885793892
- type: nauc_recall_at_3_std
value: -7.8209259710161545
- type: nauc_recall_at_5_diff1
value: 73.14792922324158
- type: nauc_recall_at_5_max
value: 58.00913922758409
- type: nauc_recall_at_5_std
value: 2.5058994355597797
- type: ndcg_at_1
value: 71.218
- type: ndcg_at_10
value: 81.326
- type: ndcg_at_100
value: 82.64
- type: ndcg_at_1000
value: 82.84700000000001
- type: ndcg_at_20
value: 82.04
- type: ndcg_at_3
value: 78.09899999999999
- type: ndcg_at_5
value: 79.919
- type: precision_at_1
value: 71.218
- type: precision_at_10
value: 9.739
- type: precision_at_100
value: 1.0370000000000001
- type: precision_at_1000
value: 0.106
- type: precision_at_20
value: 5.021
- type: precision_at_3
value: 29.274
- type: precision_at_5
value: 18.55
- type: recall_at_1
value: 68.989
- type: recall_at_10
value: 91.54299999999999
- type: recall_at_100
value: 97.33800000000001
- type: recall_at_1000
value: 98.978
- type: recall_at_20
value: 94.272
- type: recall_at_3
value: 82.997
- type: recall_at_5
value: 87.332
task:
type: Retrieval
- dataset:
config: default
name: MTEB MSMARCO (default)
revision: c5a29a104738b98a9e76336939199e264163d4a0
split: dev
type: mteb/msmarco
metrics:
- type: main_score
value: 41.730000000000004
- type: map_at_1
value: 22.247
- type: map_at_10
value: 34.707
- type: map_at_100
value: 35.942
- type: map_at_1000
value: 35.985
- type: map_at_20
value: 35.487
- type: map_at_3
value: 30.681000000000004
- type: map_at_5
value: 32.999
- type: mrr_at_1
value: 22.851002865329512
- type: mrr_at_10
value: 35.31956951835174
- type: mrr_at_100
value: 36.483110345028855
- type: mrr_at_1000
value: 36.52100396629788
- type: mrr_at_20
value: 36.06021818764369
- type: mrr_at_3
value: 31.387297039159375
- type: mrr_at_5
value: 33.65663801337153
- type: nauc_map_at_1000_diff1
value: 32.0636034497493
- type: nauc_map_at_1000_max
value: -0.10762638411612366
- type: nauc_map_at_1000_std
value: -22.783093674680906
- type: nauc_map_at_100_diff1
value: 32.056943193976636
- type: nauc_map_at_100_max
value: -0.10716522768388786
- type: nauc_map_at_100_std
value: -22.76716334129472
- type: nauc_map_at_10_diff1
value: 31.881216789203926
- type: nauc_map_at_10_max
value: -0.26405277769440294
- type: nauc_map_at_10_std
value: -23.34575128505554
- type: nauc_map_at_1_diff1
value: 35.9596501126688
- type: nauc_map_at_1_max
value: -0.8091904424462875
- type: nauc_map_at_1_std
value: -20.527774904206694
- type: nauc_map_at_20_diff1
value: 31.98673561195359
- type: nauc_map_at_20_max
value: -0.16723938267691815
- type: nauc_map_at_20_std
value: -23.07658691561266
- type: nauc_map_at_3_diff1
value: 32.077835294709836
- type: nauc_map_at_3_max
value: -0.31351965109909036
- type: nauc_map_at_3_std
value: -22.975671553829898
- type: nauc_map_at_5_diff1
value: 31.7691847510221
- type: nauc_map_at_5_max
value: -0.41491147447451765
- type: nauc_map_at_5_std
value: -23.42837462137769
- type: nauc_mrr_at_1000_diff1
value: 31.853823797444946
- type: nauc_mrr_at_1000_max
value: -0.0795653698187692
- type: nauc_mrr_at_1000_std
value: -22.461855418444497
- type: nauc_mrr_at_100_diff1
value: 31.849866586080957
- type: nauc_mrr_at_100_max
value: -0.07482091919541356
- type: nauc_mrr_at_100_std
value: -22.443743562722524
- type: nauc_mrr_at_10_diff1
value: 31.643111549575337
- type: nauc_mrr_at_10_max
value: -0.1788859143460348
- type: nauc_mrr_at_10_std
value: -22.954992949686492
- type: nauc_mrr_at_1_diff1
value: 35.7178151921562
- type: nauc_mrr_at_1_max
value: -0.8193741570534676
- type: nauc_mrr_at_1_std
value: -20.3433031561671
- type: nauc_mrr_at_20_diff1
value: 31.776530547723524
- type: nauc_mrr_at_20_max
value: -0.10542019020456578
- type: nauc_mrr_at_20_std
value: -22.71080549252751
- type: nauc_mrr_at_3_diff1
value: 31.80756010049482
- type: nauc_mrr_at_3_max
value: -0.3435740120558125
- type: nauc_mrr_at_3_std
value: -22.697104396706045
- type: nauc_mrr_at_5_diff1
value: 31.54341690236841
- type: nauc_mrr_at_5_max
value: -0.4024431073551746
- type: nauc_mrr_at_5_std
value: -23.12417136090685
- type: nauc_ndcg_at_1000_diff1
value: 31.430857835400268
- type: nauc_ndcg_at_1000_max
value: 0.6319498920296351
- type: nauc_ndcg_at_1000_std
value: -21.712146797190552
- type: nauc_ndcg_at_100_diff1
value: 31.32357884299405
- type: nauc_ndcg_at_100_max
value: 0.6780605760661018
- type: nauc_ndcg_at_100_std
value: -21.076881792417474
- type: nauc_ndcg_at_10_diff1
value: 30.53957684629431
- type: nauc_ndcg_at_10_max
value: -0.06521885345644524
- type: nauc_ndcg_at_10_std
value: -24.232178998766035
- type: nauc_ndcg_at_1_diff1
value: 35.7178151921562
- type: nauc_ndcg_at_1_max
value: -0.8193741570534676
- type: nauc_ndcg_at_1_std
value: -20.3433031561671
- type: nauc_ndcg_at_20_diff1
value: 30.94619961764288
- type: nauc_ndcg_at_20_max
value: 0.26559255443441493
- type: nauc_ndcg_at_20_std
value: -23.282194745484635
- type: nauc_ndcg_at_3_diff1
value: 30.848425100267956
- type: nauc_ndcg_at_3_max
value: -0.19327880155683758
- type: nauc_ndcg_at_3_std
value: -23.64217533635991
- type: nauc_ndcg_at_5_diff1
value: 30.341358607759684
- type: nauc_ndcg_at_5_max
value: -0.377993201232737
- type: nauc_ndcg_at_5_std
value: -24.432188906664475
- type: nauc_precision_at_1000_diff1
value: -1.6627268212806197
- type: nauc_precision_at_1000_max
value: 15.576434562098331
- type: nauc_precision_at_1000_std
value: 16.889453027471074
- type: nauc_precision_at_100_diff1
value: 12.457837392454724
- type: nauc_precision_at_100_max
value: 10.697083247910326
- type: nauc_precision_at_100_std
value: 9.850479135599352
- type: nauc_precision_at_10_diff1
value: 23.76307833863195
- type: nauc_precision_at_10_max
value: 1.0200495511494787
- type: nauc_precision_at_10_std
value: -25.095511952597043
- type: nauc_precision_at_1_diff1
value: 35.7178151921562
- type: nauc_precision_at_1_max
value: -0.8193741570534676
- type: nauc_precision_at_1_std
value: -20.3433031561671
- type: nauc_precision_at_20_diff1
value: 22.153410773797738
- type: nauc_precision_at_20_max
value: 3.66764745625043
- type: nauc_precision_at_20_std
value: -18.644800930016796
- type: nauc_precision_at_3_diff1
value: 27.07667204046241
- type: nauc_precision_at_3_max
value: 0.05921216427990492
- type: nauc_precision_at_3_std
value: -25.37077013195443
- type: nauc_precision_at_5_diff1
value: 25.2422760258942
- type: nauc_precision_at_5_max
value: -0.038410292400458575
- type: nauc_precision_at_5_std
value: -26.572596379973678
- type: nauc_recall_at_1000_diff1
value: 24.93945079613093
- type: nauc_recall_at_1000_max
value: 36.0272340013387
- type: nauc_recall_at_1000_std
value: 49.87019053879185
- type: nauc_recall_at_100_diff1
value: 27.806311570451737
- type: nauc_recall_at_100_max
value: 9.186682294186838
- type: nauc_recall_at_100_std
value: 7.454897789626523
- type: nauc_recall_at_10_diff1
value: 26.294054520577564
- type: nauc_recall_at_10_max
value: 0.3138271945251296
- type: nauc_recall_at_10_std
value: -27.178178304458882
- type: nauc_recall_at_1_diff1
value: 35.9596501126688
- type: nauc_recall_at_1_max
value: -0.8091904424462875
- type: nauc_recall_at_1_std
value: -20.527774904206694
- type: nauc_recall_at_20_diff1
value: 27.30304739410091
- type: nauc_recall_at_20_max
value: 1.8065387859063573
- type: nauc_recall_at_20_std
value: -23.36769420764751
- type: nauc_recall_at_3_diff1
value: 27.564414984037743
- type: nauc_recall_at_3_max
value: 0.14988748329158136
- type: nauc_recall_at_3_std
value: -25.42373674008892
- type: nauc_recall_at_5_diff1
value: 26.26961147212282
- type: nauc_recall_at_5_max
value: -0.3843219880375397
- type: nauc_recall_at_5_std
value: -27.256288072895167
- type: ndcg_at_1
value: 22.851
- type: ndcg_at_10
value: 41.730000000000004
- type: ndcg_at_100
value: 47.58
- type: ndcg_at_1000
value: 48.623
- type: ndcg_at_20
value: 44.483
- type: ndcg_at_3
value: 33.577
- type: ndcg_at_5
value: 37.685
- type: precision_at_1
value: 22.851
- type: precision_at_10
value: 6.622999999999999
- type: precision_at_100
value: 0.954
- type: precision_at_1000
value: 0.104
- type: precision_at_20
value: 3.8859999999999997
- type: precision_at_3
value: 14.26
- type: precision_at_5
value: 10.636
- type: recall_at_1
value: 22.247
- type: recall_at_10
value: 63.339
- type: recall_at_100
value: 90.21000000000001
- type: recall_at_1000
value: 98.112
- type: recall_at_20
value: 74.01299999999999
- type: recall_at_3
value: 41.248000000000005
- type: recall_at_5
value: 51.073
task:
type: Retrieval
- dataset:
config: default
name: MTEB MedicalRetrieval (default)
revision: 2039188fb5800a9803ba5048df7b76e6fb151fc6
split: test
type: C-MTEB/MedicalRetrieval
metrics:
- type: main_score
value: 60.26200000000001
- type: map_at_1
value: 52.400000000000006
- type: map_at_10
value: 57.68299999999999
- type: map_at_100
value: 58.17099999999999
- type: map_at_1000
value: 58.221999999999994
- type: map_at_20
value: 57.922
- type: map_at_3
value: 56.483000000000004
- type: map_at_5
value: 57.158
- type: mrr_at_1
value: 52.6
- type: mrr_at_10
value: 57.783095238095264
- type: mrr_at_100
value: 58.271291861365846
- type: mrr_at_1000
value: 58.32232037265995
- type: mrr_at_20
value: 58.022672106874914
- type: mrr_at_3
value: 56.58333333333334
- type: mrr_at_5
value: 57.25833333333333
- type: nauc_map_at_1000_diff1
value: 80.25186413093654
- type: nauc_map_at_1000_max
value: 68.6589463683237
- type: nauc_map_at_1000_std
value: 23.516042280692524
- type: nauc_map_at_100_diff1
value: 80.23486691799475
- type: nauc_map_at_100_max
value: 68.65365107940102
- type: nauc_map_at_100_std
value: 23.50821753358528
- type: nauc_map_at_10_diff1
value: 80.31312526535031
- type: nauc_map_at_10_max
value: 68.72433332315319
- type: nauc_map_at_10_std
value: 23.430105304951933
- type: nauc_map_at_1_diff1
value: 83.93425962857124
- type: nauc_map_at_1_max
value: 68.43607087633792
- type: nauc_map_at_1_std
value: 20.35659452637097
- type: nauc_map_at_20_diff1
value: 80.24816008810332
- type: nauc_map_at_20_max
value: 68.64396105945949
- type: nauc_map_at_20_std
value: 23.44583464712498
- type: nauc_map_at_3_diff1
value: 80.80315735472321
- type: nauc_map_at_3_max
value: 69.21085450395991
- type: nauc_map_at_3_std
value: 22.90928162214152
- type: nauc_map_at_5_diff1
value: 80.54059981371853
- type: nauc_map_at_5_max
value: 69.0416952214985
- type: nauc_map_at_5_std
value: 23.007008655536996
- type: nauc_mrr_at_1000_diff1
value: 80.02598222484747
- type: nauc_mrr_at_1000_max
value: 69.00867672896285
- type: nauc_mrr_at_1000_std
value: 23.813776518995162
- type: nauc_mrr_at_100_diff1
value: 80.00929182010866
- type: nauc_mrr_at_100_max
value: 69.0028714499467
- type: nauc_mrr_at_100_std
value: 23.80551909810323
- type: nauc_mrr_at_10_diff1
value: 80.0899623623251
- type: nauc_mrr_at_10_max
value: 69.07006299955447
- type: nauc_mrr_at_10_std
value: 23.72503001475042
- type: nauc_mrr_at_1_diff1
value: 83.52410075586056
- type: nauc_mrr_at_1_max
value: 69.0826737918139
- type: nauc_mrr_at_1_std
value: 20.90224184801206
- type: nauc_mrr_at_20_diff1
value: 80.02389922820662
- type: nauc_mrr_at_20_max
value: 68.99115920463171
- type: nauc_mrr_at_20_std
value: 23.741353036951455
- type: nauc_mrr_at_3_diff1
value: 80.58508574532472
- type: nauc_mrr_at_3_max
value: 69.55006536103767
- type: nauc_mrr_at_3_std
value: 23.19761715742159
- type: nauc_mrr_at_5_diff1
value: 80.31970424627885
- type: nauc_mrr_at_5_max
value: 69.38465676127115
- type: nauc_mrr_at_5_std
value: 23.29880079508209
- type: nauc_ndcg_at_1000_diff1
value: 78.69533160203528
- type: nauc_ndcg_at_1000_max
value: 68.03229826865802
- type: nauc_ndcg_at_1000_std
value: 25.297260209746632
- type: nauc_ndcg_at_100_diff1
value: 78.11195926870373
- type: nauc_ndcg_at_100_max
value: 67.82474214228908
- type: nauc_ndcg_at_100_std
value: 25.37474683279944
- type: nauc_ndcg_at_10_diff1
value: 78.63003479922878
- type: nauc_ndcg_at_10_max
value: 68.0928496345068
- type: nauc_ndcg_at_10_std
value: 24.759961085744468
- type: nauc_ndcg_at_1_diff1
value: 83.93425962857124
- type: nauc_ndcg_at_1_max
value: 68.43607087633792
- type: nauc_ndcg_at_1_std
value: 20.35659452637097
- type: nauc_ndcg_at_20_diff1
value: 78.32020502913969
- type: nauc_ndcg_at_20_max
value: 67.75412241017483
- type: nauc_ndcg_at_20_std
value: 24.825076899725687
- type: nauc_ndcg_at_3_diff1
value: 79.74959098483932
- type: nauc_ndcg_at_3_max
value: 69.23742202876441
- type: nauc_ndcg_at_3_std
value: 23.585024714097848
- type: nauc_ndcg_at_5_diff1
value: 79.23101922231892
- type: nauc_ndcg_at_5_max
value: 68.9115828141736
- type: nauc_ndcg_at_5_std
value: 23.740463950499
- type: nauc_precision_at_1000_diff1
value: 61.54570919276784
- type: nauc_precision_at_1000_max
value: 59.570494864612414
- type: nauc_precision_at_1000_std
value: 57.00534759358268
- type: nauc_precision_at_100_diff1
value: 64.76869699306094
- type: nauc_precision_at_100_max
value: 62.30339244410177
- type: nauc_precision_at_100_std
value: 37.98628703601719
- type: nauc_precision_at_10_diff1
value: 72.45633104101779
- type: nauc_precision_at_10_max
value: 65.35363669934428
- type: nauc_precision_at_10_std
value: 29.658486793057513
- type: nauc_precision_at_1_diff1
value: 83.93425962857124
- type: nauc_precision_at_1_max
value: 68.43607087633792
- type: nauc_precision_at_1_std
value: 20.35659452637097
- type: nauc_precision_at_20_diff1
value: 70.30608304625963
- type: nauc_precision_at_20_max
value: 63.407183088678664
- type: nauc_precision_at_20_std
value: 30.37490465221116
- type: nauc_precision_at_3_diff1
value: 76.51822478400202
- type: nauc_precision_at_3_max
value: 69.24170168994287
- type: nauc_precision_at_3_std
value: 25.597170951121594
- type: nauc_precision_at_5_diff1
value: 74.94208233318108
- type: nauc_precision_at_5_max
value: 68.35040831955108
- type: nauc_precision_at_5_std
value: 26.022568584170465
- type: nauc_recall_at_1000_diff1
value: 61.545709192768214
- type: nauc_recall_at_1000_max
value: 59.570494864612776
- type: nauc_recall_at_1000_std
value: 57.00534759358318
- type: nauc_recall_at_100_diff1
value: 64.76869699306086
- type: nauc_recall_at_100_max
value: 62.303392444101746
- type: nauc_recall_at_100_std
value: 37.98628703601717
- type: nauc_recall_at_10_diff1
value: 72.45633104101782
- type: nauc_recall_at_10_max
value: 65.35363669934435
- type: nauc_recall_at_10_std
value: 29.65848679305753
- type: nauc_recall_at_1_diff1
value: 83.93425962857124
- type: nauc_recall_at_1_max
value: 68.43607087633792
- type: nauc_recall_at_1_std
value: 20.35659452637097
- type: nauc_recall_at_20_diff1
value: 70.30608304625963
- type: nauc_recall_at_20_max
value: 63.407183088678664
- type: nauc_recall_at_20_std
value: 30.374904652211303
- type: nauc_recall_at_3_diff1
value: 76.51822478400197
- type: nauc_recall_at_3_max
value: 69.24170168994279
- type: nauc_recall_at_3_std
value: 25.597170951121612
- type: nauc_recall_at_5_diff1
value: 74.94208233318102
- type: nauc_recall_at_5_max
value: 68.35040831955102
- type: nauc_recall_at_5_std
value: 26.022568584170475
- type: ndcg_at_1
value: 52.400000000000006
- type: ndcg_at_10
value: 60.26200000000001
- type: ndcg_at_100
value: 62.993
- type: ndcg_at_1000
value: 64.535
- type: ndcg_at_20
value: 61.141999999999996
- type: ndcg_at_3
value: 57.751
- type: ndcg_at_5
value: 58.977000000000004
- type: precision_at_1
value: 52.400000000000006
- type: precision_at_10
value: 6.84
- type: precision_at_100
value: 0.8200000000000001
- type: precision_at_1000
value: 0.095
- type: precision_at_20
value: 3.595
- type: precision_at_3
value: 20.467
- type: precision_at_5
value: 12.879999999999999
- type: recall_at_1
value: 52.400000000000006
- type: recall_at_10
value: 68.4
- type: recall_at_100
value: 82
- type: recall_at_1000
value: 94.5
- type: recall_at_20
value: 71.89999999999999
- type: recall_at_3
value: 61.4
- type: recall_at_5
value: 64.4
task:
type: Retrieval
- dataset:
config: default
name: MTEB NFCorpus (default)
revision: ec0fa4fe99da2ff19ca1214b7966684033a58814
split: test
type: mteb/nfcorpus
metrics:
- type: main_score
value: 37.891000000000005
- type: map_at_1
value: 6.4399999999999995
- type: map_at_10
value: 14.719
- type: map_at_100
value: 18.747
- type: map_at_1000
value: 20.357
- type: map_at_20
value: 16.326
- type: map_at_3
value: 11.040999999999999
- type: map_at_5
value: 12.642000000000001
- type: mrr_at_1
value: 49.22600619195047
- type: mrr_at_10
value: 58.40680131701803
- type: mrr_at_100
value: 58.87911539017606
- type: mrr_at_1000
value: 58.9172513415915
- type: mrr_at_20
value: 58.679349975789606
- type: mrr_at_3
value: 56.449948400412794
- type: mrr_at_5
value: 57.719298245614034
- type: nauc_map_at_1000_diff1
value: 33.230722382872365
- type: nauc_map_at_1000_max
value: 29.958254502666996
- type: nauc_map_at_1000_std
value: 12.351734798263454
- type: nauc_map_at_100_diff1
value: 35.19923237757157
- type: nauc_map_at_100_max
value: 29.329030443305648
- type: nauc_map_at_100_std
value: 8.900436204621524
- type: nauc_map_at_10_diff1
value: 39.93995003547164
- type: nauc_map_at_10_max
value: 23.56047272296223
- type: nauc_map_at_10_std
value: -3.0673642943053134
- type: nauc_map_at_1_diff1
value: 54.28005668530579
- type: nauc_map_at_1_max
value: 10.578462013666147
- type: nauc_map_at_1_std
value: -17.523933026613282
- type: nauc_map_at_20_diff1
value: 37.534174369462114
- type: nauc_map_at_20_max
value: 26.10226085362098
- type: nauc_map_at_20_std
value: 1.64213157637494
- type: nauc_map_at_3_diff1
value: 44.669764973127776
- type: nauc_map_at_3_max
value: 16.064917234606437
- type: nauc_map_at_3_std
value: -10.406776750194295
- type: nauc_map_at_5_diff1
value: 42.77441524934012
- type: nauc_map_at_5_max
value: 19.558923561089774
- type: nauc_map_at_5_std
value: -8.098884326439757
- type: nauc_mrr_at_1000_diff1
value: 33.29112194380436
- type: nauc_mrr_at_1000_max
value: 50.81292385326501
- type: nauc_mrr_at_1000_std
value: 27.62985206778606
- type: nauc_mrr_at_100_diff1
value: 33.276475091311255
- type: nauc_mrr_at_100_max
value: 50.85370004566169
- type: nauc_mrr_at_100_std
value: 27.685134876913843
- type: nauc_mrr_at_10_diff1
value: 33.15759779165778
- type: nauc_mrr_at_10_max
value: 50.65755287933926
- type: nauc_mrr_at_10_std
value: 27.562950062694558
- type: nauc_mrr_at_1_diff1
value: 36.37601545284129
- type: nauc_mrr_at_1_max
value: 45.72722868673481
- type: nauc_mrr_at_1_std
value: 18.55225595921063
- type: nauc_mrr_at_20_diff1
value: 33.440227211273005
- type: nauc_mrr_at_20_max
value: 50.86793658499095
- type: nauc_mrr_at_20_std
value: 27.609176719637706
- type: nauc_mrr_at_3_diff1
value: 33.47002885119429
- type: nauc_mrr_at_3_max
value: 49.8157891072848
- type: nauc_mrr_at_3_std
value: 26.689624887422404
- type: nauc_mrr_at_5_diff1
value: 33.153134678779764
- type: nauc_mrr_at_5_max
value: 50.383762857355194
- type: nauc_mrr_at_5_std
value: 27.068173550184316
- type: nauc_ndcg_at_1000_diff1
value: 28.35603452869833
- type: nauc_ndcg_at_1000_max
value: 45.41761040447669
- type: nauc_ndcg_at_1000_std
value: 30.9338935123294
- type: nauc_ndcg_at_100_diff1
value: 28.2916752904402
- type: nauc_ndcg_at_100_max
value: 40.30182084762403
- type: nauc_ndcg_at_100_std
value: 26.28495251583047
- type: nauc_ndcg_at_10_diff1
value: 24.788071641487708
- type: nauc_ndcg_at_10_max
value: 40.12743958245296
- type: nauc_ndcg_at_10_std
value: 26.58469476507991
- type: nauc_ndcg_at_1_diff1
value: 36.12595606627444
- type: nauc_ndcg_at_1_max
value: 43.30675377860411
- type: nauc_ndcg_at_1_std
value: 17.630710476320917
- type: nauc_ndcg_at_20_diff1
value: 24.787941274982103
- type: nauc_ndcg_at_20_max
value: 39.026577013109794
- type: nauc_ndcg_at_20_std
value: 27.060661974731133
- type: nauc_ndcg_at_3_diff1
value: 27.791041290881815
- type: nauc_ndcg_at_3_max
value: 43.237441431102205
- type: nauc_ndcg_at_3_std
value: 23.300542721018694
- type: nauc_ndcg_at_5_diff1
value: 26.05331825533656
- type: nauc_ndcg_at_5_max
value: 42.2459176598355
- type: nauc_ndcg_at_5_std
value: 24.515545611337718
- type: nauc_precision_at_1000_diff1
value: -19.008829403142347
- type: nauc_precision_at_1000_max
value: 8.162315941038813
- type: nauc_precision_at_1000_std
value: 36.8803602556575
- type: nauc_precision_at_100_diff1
value: -14.186664088951117
- type: nauc_precision_at_100_max
value: 20.74568414768601
- type: nauc_precision_at_100_std
value: 46.11245904471494
- type: nauc_precision_at_10_diff1
value: 2.544985600795211
- type: nauc_precision_at_10_max
value: 38.012859132070545
- type: nauc_precision_at_10_std
value: 39.72677966147808
- type: nauc_precision_at_1_diff1
value: 36.37601545284129
- type: nauc_precision_at_1_max
value: 45.72722868673481
- type: nauc_precision_at_1_std
value: 18.55225595921063
- type: nauc_precision_at_20_diff1
value: -3.753006427110271
- type: nauc_precision_at_20_max
value: 33.72153145480959
- type: nauc_precision_at_20_std
value: 44.63783536567636
- type: nauc_precision_at_3_diff1
value: 16.628508929858125
- type: nauc_precision_at_3_max
value: 44.0066986115904
- type: nauc_precision_at_3_std
value: 30.290565597925568
- type: nauc_precision_at_5_diff1
value: 9.930397054770475
- type: nauc_precision_at_5_max
value: 41.41134797150698
- type: nauc_precision_at_5_std
value: 33.73100330814152
- type: nauc_recall_at_1000_diff1
value: 12.072299973396554
- type: nauc_recall_at_1000_max
value: 22.732366240050446
- type: nauc_recall_at_1000_std
value: 17.86341797097261
- type: nauc_recall_at_100_diff1
value: 20.263994031645876
- type: nauc_recall_at_100_max
value: 26.14243476135425
- type: nauc_recall_at_100_std
value: 15.522503389903141
- type: nauc_recall_at_10_diff1
value: 32.25623112992171
- type: nauc_recall_at_10_max
value: 21.682862433450175
- type: nauc_recall_at_10_std
value: -0.9148057523522605
- type: nauc_recall_at_1_diff1
value: 54.28005668530579
- type: nauc_recall_at_1_max
value: 10.578462013666147
- type: nauc_recall_at_1_std
value: -17.523933026613282
- type: nauc_recall_at_20_diff1
value: 28.093221355343694
- type: nauc_recall_at_20_max
value: 23.921334945420412
- type: nauc_recall_at_20_std
value: 3.965629365931312
- type: nauc_recall_at_3_diff1
value: 41.38330659934456
- type: nauc_recall_at_3_max
value: 16.023503457855647
- type: nauc_recall_at_3_std
value: -8.013087104222292
- type: nauc_recall_at_5_diff1
value: 37.83725114335421
- type: nauc_recall_at_5_max
value: 19.441966766336197
- type: nauc_recall_at_5_std
value: -6.153515011798168
- type: ndcg_at_1
value: 47.678
- type: ndcg_at_10
value: 37.891000000000005
- type: ndcg_at_100
value: 35.333
- type: ndcg_at_1000
value: 44.043
- type: ndcg_at_20
value: 35.321999999999996
- type: ndcg_at_3
value: 44.233
- type: ndcg_at_5
value: 41.323
- type: precision_at_1
value: 49.226
- type: precision_at_10
value: 27.647
- type: precision_at_100
value: 8.966000000000001
- type: precision_at_1000
value: 2.1999999999999997
- type: precision_at_20
value: 20.372
- type: precision_at_3
value: 41.589
- type: precision_at_5
value: 35.356
- type: recall_at_1
value: 6.4399999999999995
- type: recall_at_10
value: 18.682000000000002
- type: recall_at_100
value: 35.839
- type: recall_at_1000
value: 67.216
- type: recall_at_20
value: 22.61
- type: recall_at_3
value: 12.199
- type: recall_at_5
value: 14.641000000000002
task:
type: Retrieval
- dataset:
config: default
name: MTEB NQ (default)
revision: b774495ed302d8c44a3a7ea25c90dbce03968f31
split: test
type: mteb/nq
metrics:
- type: main_score
value: 64.688
- type: map_at_1
value: 41.066
- type: map_at_10
value: 57.418
- type: map_at_100
value: 58.159000000000006
- type: map_at_1000
value: 58.177
- type: map_at_20
value: 57.919
- type: map_at_3
value: 53.358000000000004
- type: map_at_5
value: 56.005
- type: mrr_at_1
value: 46.00231749710313
- type: mrr_at_10
value: 59.84124685022698
- type: mrr_at_100
value: 60.34280926553718
- type: mrr_at_1000
value: 60.35412522607222
- type: mrr_at_20
value: 60.18970234136678
- type: mrr_at_3
value: 56.79316338354554
- type: mrr_at_5
value: 58.780417149478446
- type: nauc_map_at_1000_diff1
value: 38.78776819131969
- type: nauc_map_at_1000_max
value: 24.390829165753804
- type: nauc_map_at_1000_std
value: -0.8959882120637628
- type: nauc_map_at_100_diff1
value: 38.78070841818844
- type: nauc_map_at_100_max
value: 24.405013334075484
- type: nauc_map_at_100_std
value: -0.8799516222708522
- type: nauc_map_at_10_diff1
value: 38.72437116926775
- type: nauc_map_at_10_max
value: 24.398806749097698
- type: nauc_map_at_10_std
value: -1.0592266396250514
- type: nauc_map_at_1_diff1
value: 42.179461753096895
- type: nauc_map_at_1_max
value: 19.349773592901936
- type: nauc_map_at_1_std
value: -4.524397921564787
- type: nauc_map_at_20_diff1
value: 38.74566928627237
- type: nauc_map_at_20_max
value: 24.468786939668597
- type: nauc_map_at_20_std
value: -0.8568795871183598
- type: nauc_map_at_3_diff1
value: 38.73170462814454
- type: nauc_map_at_3_max
value: 22.851756023909196
- type: nauc_map_at_3_std
value: -2.5060248302567025
- type: nauc_map_at_5_diff1
value: 38.62186383188218
- type: nauc_map_at_5_max
value: 24.35582662505046
- type: nauc_map_at_5_std
value: -1.4319997886566798
- type: nauc_mrr_at_1000_diff1
value: 38.71889349776728
- type: nauc_mrr_at_1000_max
value: 24.51713049124131
- type: nauc_mrr_at_1000_std
value: 0.09666909988322456
- type: nauc_mrr_at_100_diff1
value: 38.714226019044276
- type: nauc_mrr_at_100_max
value: 24.52890092559232
- type: nauc_mrr_at_100_std
value: 0.1096986768089403
- type: nauc_mrr_at_10_diff1
value: 38.643356854659665
- type: nauc_mrr_at_10_max
value: 24.600084444873232
- type: nauc_mrr_at_10_std
value: 0.13602746715836517
- type: nauc_mrr_at_1_diff1
value: 42.158230484920594
- type: nauc_mrr_at_1_max
value: 21.627378828477532
- type: nauc_mrr_at_1_std
value: -1.9065076224382262
- type: nauc_mrr_at_20_diff1
value: 38.679698906833885
- type: nauc_mrr_at_20_max
value: 24.632607091840622
- type: nauc_mrr_at_20_std
value: 0.20084937172893197
- type: nauc_mrr_at_3_diff1
value: 38.39925142686701
- type: nauc_mrr_at_3_max
value: 23.66152747304512
- type: nauc_mrr_at_3_std
value: -0.628519594378832
- type: nauc_mrr_at_5_diff1
value: 38.371246750856606
- type: nauc_mrr_at_5_max
value: 24.682577377101104
- type: nauc_mrr_at_5_std
value: 0.060689534605535604
- type: nauc_ndcg_at_1000_diff1
value: 38.17628275755128
- type: nauc_ndcg_at_1000_max
value: 25.80539838814419
- type: nauc_ndcg_at_1000_std
value: 0.8586445473741267
- type: nauc_ndcg_at_100_diff1
value: 38.01695858087552
- type: nauc_ndcg_at_100_max
value: 26.26810029116518
- type: nauc_ndcg_at_100_std
value: 1.3954995431536397
- type: nauc_ndcg_at_10_diff1
value: 37.714599237914975
- type: nauc_ndcg_at_10_max
value: 26.527887523630056
- type: nauc_ndcg_at_10_std
value: 0.9303338049009193
- type: nauc_ndcg_at_1_diff1
value: 42.23553827569472
- type: nauc_ndcg_at_1_max
value: 21.6544058258776
- type: nauc_ndcg_at_1_std
value: -1.8247895107064274
- type: nauc_ndcg_at_20_diff1
value: 37.79188917035569
- type: nauc_ndcg_at_20_max
value: 26.83835500851581
- type: nauc_ndcg_at_20_std
value: 1.6532937246137365
- type: nauc_ndcg_at_3_diff1
value: 37.61343042280347
- type: nauc_ndcg_at_3_max
value: 23.917437039390137
- type: nauc_ndcg_at_3_std
value: -1.5800740410055774
- type: nauc_ndcg_at_5_diff1
value: 37.336065614777795
- type: nauc_ndcg_at_5_max
value: 26.401880079796037
- type: nauc_ndcg_at_5_std
value: 0.19202305501782874
- type: nauc_precision_at_1000_diff1
value: -11.144872061510371
- type: nauc_precision_at_1000_max
value: 5.6241070221486185
- type: nauc_precision_at_1000_std
value: 14.747372144618382
- type: nauc_precision_at_100_diff1
value: -8.369408543560592
- type: nauc_precision_at_100_max
value: 11.023240256125085
- type: nauc_precision_at_100_std
value: 18.045874227710854
- type: nauc_precision_at_10_diff1
value: 6.584951316095451
- type: nauc_precision_at_10_max
value: 21.658298046875455
- type: nauc_precision_at_10_std
value: 14.797525192336455
- type: nauc_precision_at_1_diff1
value: 42.23553827569472
- type: nauc_precision_at_1_max
value: 21.6544058258776
- type: nauc_precision_at_1_std
value: -1.8247895107064274
- type: nauc_precision_at_20_diff1
value: 0.33534981268579644
- type: nauc_precision_at_20_max
value: 19.146808861474966
- type: nauc_precision_at_20_std
value: 18.320685032020453
- type: nauc_precision_at_3_diff1
value: 22.218212561230644
- type: nauc_precision_at_3_max
value: 24.292474434607307
- type: nauc_precision_at_3_std
value: 6.154065316865717
- type: nauc_precision_at_5_diff1
value: 13.630700977184757
- type: nauc_precision_at_5_max
value: 25.323202132346257
- type: nauc_precision_at_5_std
value: 11.253374996425851
- type: nauc_recall_at_1000_diff1
value: 14.520278571679423
- type: nauc_recall_at_1000_max
value: 77.34603733204753
- type: nauc_recall_at_1000_std
value: 81.57477743790244
- type: nauc_recall_at_100_diff1
value: 25.31452271948367
- type: nauc_recall_at_100_max
value: 59.73388798773907
- type: nauc_recall_at_100_std
value: 45.159605231791645
- type: nauc_recall_at_10_diff1
value: 31.50932770496877
- type: nauc_recall_at_10_max
value: 36.52278383994803
- type: nauc_recall_at_10_std
value: 9.297753632006579
- type: nauc_recall_at_1_diff1
value: 42.179461753096895
- type: nauc_recall_at_1_max
value: 19.349773592901936
- type: nauc_recall_at_1_std
value: -4.524397921564787
- type: nauc_recall_at_20_diff1
value: 29.71090965503412
- type: nauc_recall_at_20_max
value: 44.5403524954586
- type: nauc_recall_at_20_std
value: 19.84516215680696
- type: nauc_recall_at_3_diff1
value: 33.18557182144929
- type: nauc_recall_at_3_max
value: 25.44098884889825
- type: nauc_recall_at_3_std
value: -0.714874800098336
- type: nauc_recall_at_5_diff1
value: 31.408995216208442
- type: nauc_recall_at_5_max
value: 32.959847941704965
- type: nauc_recall_at_5_std
value: 4.396048200488618
- type: ndcg_at_1
value: 45.973000000000006
- type: ndcg_at_10
value: 64.688
- type: ndcg_at_100
value: 67.649
- type: ndcg_at_1000
value: 68.024
- type: ndcg_at_20
value: 66.267
- type: ndcg_at_3
value: 57.497
- type: ndcg_at_5
value: 61.695
- type: precision_at_1
value: 45.973000000000006
- type: precision_at_10
value: 10.046
- type: precision_at_100
value: 1.174
- type: precision_at_1000
value: 0.121
- type: precision_at_20
value: 5.41
- type: precision_at_3
value: 25.657000000000004
- type: precision_at_5
value: 17.868000000000002
- type: recall_at_1
value: 41.066
- type: recall_at_10
value: 84.135
- type: recall_at_100
value: 96.67099999999999
- type: recall_at_1000
value: 99.387
- type: recall_at_20
value: 89.938
- type: recall_at_3
value: 66.01700000000001
- type: recall_at_5
value: 75.53800000000001
task:
type: Retrieval
- dataset:
config: default
name: MTEB QuoraRetrieval (default)
revision: e4e08e0b7dbe3c8700f0daef558ff32256715259
split: test
type: mteb/quora
metrics:
- type: main_score
value: 89.16
- type: map_at_1
value: 71.393
- type: map_at_10
value: 85.431
- type: map_at_100
value: 86.035
- type: map_at_1000
value: 86.05000000000001
- type: map_at_20
value: 85.83
- type: map_at_3
value: 82.44200000000001
- type: map_at_5
value: 84.30499999999999
- type: mrr_at_1
value: 82.24000000000001
- type: mrr_at_10
value: 88.35231349206327
- type: mrr_at_100
value: 88.42772088028269
- type: mrr_at_1000
value: 88.42827680568509
- type: mrr_at_20
value: 88.41080736639496
- type: mrr_at_3
value: 87.44333333333314
- type: mrr_at_5
value: 88.06383333333302
- type: nauc_map_at_1000_diff1
value: 75.90832808026184
- type: nauc_map_at_1000_max
value: 23.53900071578855
- type: nauc_map_at_1000_std
value: -39.927827265219854
- type: nauc_map_at_100_diff1
value: 75.91666852843873
- type: nauc_map_at_100_max
value: 23.50731886205444
- type: nauc_map_at_100_std
value: -39.97767565405775
- type: nauc_map_at_10_diff1
value: 76.19643319669306
- type: nauc_map_at_10_max
value: 22.746085654201355
- type: nauc_map_at_10_std
value: -41.844605134704715
- type: nauc_map_at_1_diff1
value: 78.82668992548645
- type: nauc_map_at_1_max
value: 17.523297297435224
- type: nauc_map_at_1_std
value: -36.057027609352616
- type: nauc_map_at_20_diff1
value: 76.05783399854896
- type: nauc_map_at_20_max
value: 23.20072488797186
- type: nauc_map_at_20_std
value: -40.81478369191535
- type: nauc_map_at_3_diff1
value: 76.58496438269384
- type: nauc_map_at_3_max
value: 20.4701448032309
- type: nauc_map_at_3_std
value: -43.521383471579895
- type: nauc_map_at_5_diff1
value: 76.40363254040322
- type: nauc_map_at_5_max
value: 21.90377180304047
- type: nauc_map_at_5_std
value: -43.03599301443475
- type: nauc_mrr_at_1000_diff1
value: 76.1837291208273
- type: nauc_mrr_at_1000_max
value: 25.456231389943472
- type: nauc_mrr_at_1000_std
value: -36.78952655269047
- type: nauc_mrr_at_100_diff1
value: 76.18357466163647
- type: nauc_mrr_at_100_max
value: 25.45740950566372
- type: nauc_mrr_at_100_std
value: -36.78988094252611
- type: nauc_mrr_at_10_diff1
value: 76.19419770729094
- type: nauc_mrr_at_10_max
value: 25.46351564930667
- type: nauc_mrr_at_10_std
value: -36.87723141573333
- type: nauc_mrr_at_1_diff1
value: 76.88658099222951
- type: nauc_mrr_at_1_max
value: 25.16206140546511
- type: nauc_mrr_at_1_std
value: -34.38213705828303
- type: nauc_mrr_at_20_diff1
value: 76.18498132927036
- type: nauc_mrr_at_20_max
value: 25.452636746160334
- type: nauc_mrr_at_20_std
value: -36.8262960671442
- type: nauc_mrr_at_3_diff1
value: 75.91316743884869
- type: nauc_mrr_at_3_max
value: 25.192903416947154
- type: nauc_mrr_at_3_std
value: -37.59082081169092
- type: nauc_mrr_at_5_diff1
value: 76.10631727710869
- type: nauc_mrr_at_5_max
value: 25.452802894102184
- type: nauc_mrr_at_5_std
value: -37.127478331481385
- type: nauc_ndcg_at_1000_diff1
value: 75.67111455136272
- type: nauc_ndcg_at_1000_max
value: 24.59412813548136
- type: nauc_ndcg_at_1000_std
value: -38.16233070595817
- type: nauc_ndcg_at_100_diff1
value: 75.6966192214638
- type: nauc_ndcg_at_100_max
value: 24.482618498152718
- type: nauc_ndcg_at_100_std
value: -38.2972249197645
- type: nauc_ndcg_at_10_diff1
value: 75.91456179347648
- type: nauc_ndcg_at_10_max
value: 23.909997093518882
- type: nauc_ndcg_at_10_std
value: -41.117367550775775
- type: nauc_ndcg_at_1_diff1
value: 76.90637526910301
- type: nauc_ndcg_at_1_max
value: 25.208692439256314
- type: nauc_ndcg_at_1_std
value: -34.31241421164998
- type: nauc_ndcg_at_20_diff1
value: 75.91752313318823
- type: nauc_ndcg_at_20_max
value: 23.97959910582008
- type: nauc_ndcg_at_20_std
value: -40.19894918594622
- type: nauc_ndcg_at_3_diff1
value: 75.29898448319304
- type: nauc_ndcg_at_3_max
value: 22.551888973893174
- type: nauc_ndcg_at_3_std
value: -41.849345286714914
- type: nauc_ndcg_at_5_diff1
value: 75.72092321502765
- type: nauc_ndcg_at_5_max
value: 23.60519691021222
- type: nauc_ndcg_at_5_std
value: -41.91376015869966
- type: nauc_precision_at_1000_diff1
value: -44.127398455168155
- type: nauc_precision_at_1000_max
value: 0.4863503339606656
- type: nauc_precision_at_1000_std
value: 35.71349627810218
- type: nauc_precision_at_100_diff1
value: -43.8110631364381
- type: nauc_precision_at_100_max
value: 0.18511273986329388
- type: nauc_precision_at_100_std
value: 35.0337532338386
- type: nauc_precision_at_10_diff1
value: -39.01785144655778
- type: nauc_precision_at_10_max
value: 0.6971259033469296
- type: nauc_precision_at_10_std
value: 23.987789294860296
- type: nauc_precision_at_1_diff1
value: 76.90637526910301
- type: nauc_precision_at_1_max
value: 25.208692439256314
- type: nauc_precision_at_1_std
value: -34.31241421164998
- type: nauc_precision_at_20_diff1
value: -41.847410261778215
- type: nauc_precision_at_20_max
value: 0.2097497139050107
- type: nauc_precision_at_20_std
value: 29.570215363845254
- type: nauc_precision_at_3_diff1
value: -19.357706112240898
- type: nauc_precision_at_3_max
value: 4.657210761338343
- type: nauc_precision_at_3_std
value: 4.66414866378942
- type: nauc_precision_at_5_diff1
value: -31.450105276959246
- type: nauc_precision_at_5_max
value: 2.530729198663426
- type: nauc_precision_at_5_std
value: 15.214488796180778
- type: nauc_recall_at_1000_diff1
value: 63.77714603284074
- type: nauc_recall_at_1000_max
value: -19.834348468005338
- type: nauc_recall_at_1000_std
value: -31.970501378815406
- type: nauc_recall_at_100_diff1
value: 72.64699541352437
- type: nauc_recall_at_100_max
value: 22.534774000300914
- type: nauc_recall_at_100_std
value: -34.14821906643992
- type: nauc_recall_at_10_diff1
value: 73.69851010248087
- type: nauc_recall_at_10_max
value: 17.587267645521084
- type: nauc_recall_at_10_std
value: -60.086028161269255
- type: nauc_recall_at_1_diff1
value: 78.82668992548645
- type: nauc_recall_at_1_max
value: 17.523297297435224
- type: nauc_recall_at_1_std
value: -36.057027609352616
- type: nauc_recall_at_20_diff1
value: 75.18327167336511
- type: nauc_recall_at_20_max
value: 15.224312707593581
- type: nauc_recall_at_20_std
value: -64.29673756030971
- type: nauc_recall_at_3_diff1
value: 72.82339198565903
- type: nauc_recall_at_3_max
value: 16.48401326031195
- type: nauc_recall_at_3_std
value: -51.34959867868339
- type: nauc_recall_at_5_diff1
value: 72.06607870924402
- type: nauc_recall_at_5_max
value: 17.79100454091683
- type: nauc_recall_at_5_std
value: -55.141969303723236
- type: ndcg_at_1
value: 82.23
- type: ndcg_at_10
value: 89.16
- type: ndcg_at_100
value: 90.211
- type: ndcg_at_1000
value: 90.288
- type: ndcg_at_20
value: 89.75
- type: ndcg_at_3
value: 86.33999999999999
- type: ndcg_at_5
value: 87.878
- type: precision_at_1
value: 82.23
- type: precision_at_10
value: 13.539000000000001
- type: precision_at_100
value: 1.5350000000000001
- type: precision_at_1000
value: 0.157
- type: precision_at_20
value: 7.173
- type: precision_at_3
value: 37.769999999999996
- type: precision_at_5
value: 24.842
- type: recall_at_1
value: 71.393
- type: recall_at_10
value: 96.277
- type: recall_at_100
value: 99.682
- type: recall_at_1000
value: 99.991
- type: recall_at_20
value: 98.119
- type: recall_at_3
value: 88.113
- type: recall_at_5
value: 92.487
task:
type: Retrieval
- dataset:
config: default
name: MTEB SCIDOCS (default)
revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88
split: test
type: mteb/scidocs
metrics:
- type: main_score
value: 21.768
- type: map_at_1
value: 5.288
- type: map_at_10
value: 13.211999999999998
- type: map_at_100
value: 15.516
- type: map_at_1000
value: 15.855
- type: map_at_20
value: 14.398
- type: map_at_3
value: 9.446
- type: map_at_5
value: 11.358
- type: mrr_at_1
value: 26.1
- type: mrr_at_10
value: 37.05285714285715
- type: mrr_at_100
value: 38.139536530570474
- type: mrr_at_1000
value: 38.188891914502385
- type: mrr_at_20
value: 37.72580983687793
- type: mrr_at_3
value: 33.766666666666666
- type: mrr_at_5
value: 35.61166666666664
- type: nauc_map_at_1000_diff1
value: 12.656513374145106
- type: nauc_map_at_1000_max
value: 24.783730165079767
- type: nauc_map_at_1000_std
value: 14.273105528075881
- type: nauc_map_at_100_diff1
value: 12.645274279253234
- type: nauc_map_at_100_max
value: 24.66916730283569
- type: nauc_map_at_100_std
value: 14.01388256685681
- type: nauc_map_at_10_diff1
value: 12.875184612114158
- type: nauc_map_at_10_max
value: 22.99248747261523
- type: nauc_map_at_10_std
value: 11.322421142235303
- type: nauc_map_at_1_diff1
value: 19.91934001257035
- type: nauc_map_at_1_max
value: 14.489385038441368
- type: nauc_map_at_1_std
value: 4.41741897722446
- type: nauc_map_at_20_diff1
value: 12.551493516097029
- type: nauc_map_at_20_max
value: 23.98638784186677
- type: nauc_map_at_20_std
value: 12.137119810139337
- type: nauc_map_at_3_diff1
value: 13.304199659006906
- type: nauc_map_at_3_max
value: 19.206662818297556
- type: nauc_map_at_3_std
value: 5.979854013734774
- type: nauc_map_at_5_diff1
value: 12.592685371640899
- type: nauc_map_at_5_max
value: 21.401755502623857
- type: nauc_map_at_5_std
value: 8.616019144052176
- type: nauc_mrr_at_1000_diff1
value: 17.106981265941144
- type: nauc_mrr_at_1000_max
value: 19.09531398933099
- type: nauc_mrr_at_1000_std
value: 8.169540394403022
- type: nauc_mrr_at_100_diff1
value: 17.095540709474268
- type: nauc_mrr_at_100_max
value: 19.13061833258963
- type: nauc_mrr_at_100_std
value: 8.200152670895841
- type: nauc_mrr_at_10_diff1
value: 17.12427440233127
- type: nauc_mrr_at_10_max
value: 19.182348996793223
- type: nauc_mrr_at_10_std
value: 8.192298588949605
- type: nauc_mrr_at_1_diff1
value: 19.658541442108323
- type: nauc_mrr_at_1_max
value: 14.086175574689
- type: nauc_mrr_at_1_std
value: 4.294204987684459
- type: nauc_mrr_at_20_diff1
value: 17.11841058621361
- type: nauc_mrr_at_20_max
value: 19.08060621099804
- type: nauc_mrr_at_20_std
value: 8.207663272417248
- type: nauc_mrr_at_3_diff1
value: 16.817515599428003
- type: nauc_mrr_at_3_max
value: 18.25748916506445
- type: nauc_mrr_at_3_std
value: 6.846952621522001
- type: nauc_mrr_at_5_diff1
value: 16.88400369902854
- type: nauc_mrr_at_5_max
value: 19.010732907567835
- type: nauc_mrr_at_5_std
value: 7.620205914556857
- type: nauc_ndcg_at_1000_diff1
value: 13.929366629395965
- type: nauc_ndcg_at_1000_max
value: 27.928821890027788
- type: nauc_ndcg_at_1000_std
value: 20.248730694486596
- type: nauc_ndcg_at_100_diff1
value: 13.373097949946548
- type: nauc_ndcg_at_100_max
value: 27.7569179558014
- type: nauc_ndcg_at_100_std
value: 19.430518808099066
- type: nauc_ndcg_at_10_diff1
value: 13.903268298479643
- type: nauc_ndcg_at_10_max
value: 24.42902762317196
- type: nauc_ndcg_at_10_std
value: 13.000787883946375
- type: nauc_ndcg_at_1_diff1
value: 19.658541442108323
- type: nauc_ndcg_at_1_max
value: 14.086175574689
- type: nauc_ndcg_at_1_std
value: 4.294204987684459
- type: nauc_ndcg_at_20_diff1
value: 13.225178086875712
- type: nauc_ndcg_at_20_max
value: 25.593917892162366
- type: nauc_ndcg_at_20_std
value: 14.329359732472307
- type: nauc_ndcg_at_3_diff1
value: 13.623063832385599
- type: nauc_ndcg_at_3_max
value: 19.573786303546317
- type: nauc_ndcg_at_3_std
value: 7.004315969675634
- type: nauc_ndcg_at_5_diff1
value: 13.348473328008254
- type: nauc_ndcg_at_5_max
value: 22.590877194048893
- type: nauc_ndcg_at_5_std
value: 10.042511434054045
- type: nauc_precision_at_1000_diff1
value: 8.645904039123652
- type: nauc_precision_at_1000_max
value: 28.109231756883812
- type: nauc_precision_at_1000_std
value: 33.04671080070988
- type: nauc_precision_at_100_diff1
value: 8.412663272083252
- type: nauc_precision_at_100_max
value: 29.49914081195902
- type: nauc_precision_at_100_std
value: 29.36943289684785
- type: nauc_precision_at_10_diff1
value: 10.979668084988889
- type: nauc_precision_at_10_max
value: 26.299619084817234
- type: nauc_precision_at_10_std
value: 17.125846012423636
- type: nauc_precision_at_1_diff1
value: 19.658541442108323
- type: nauc_precision_at_1_max
value: 14.086175574689
- type: nauc_precision_at_1_std
value: 4.294204987684459
- type: nauc_precision_at_20_diff1
value: 8.742228574214412
- type: nauc_precision_at_20_max
value: 26.79019694882003
- type: nauc_precision_at_20_std
value: 18.511025182455885
- type: nauc_precision_at_3_diff1
value: 10.931190244994063
- type: nauc_precision_at_3_max
value: 21.54838184676272
- type: nauc_precision_at_3_std
value: 8.227969431101453
- type: nauc_precision_at_5_diff1
value: 10.118127450316793
- type: nauc_precision_at_5_max
value: 24.563204070643497
- type: nauc_precision_at_5_std
value: 12.512813757935984
- type: nauc_recall_at_1000_diff1
value: 8.496583562854939
- type: nauc_recall_at_1000_max
value: 28.54428524766185
- type: nauc_recall_at_1000_std
value: 34.29952091855355
- type: nauc_recall_at_100_diff1
value: 8.253267171086184
- type: nauc_recall_at_100_max
value: 29.45889349600719
- type: nauc_recall_at_100_std
value: 29.660576554498974
- type: nauc_recall_at_10_diff1
value: 10.769625058011071
- type: nauc_recall_at_10_max
value: 26.366548779182192
- type: nauc_recall_at_10_std
value: 17.1742171893101
- type: nauc_recall_at_1_diff1
value: 19.91934001257035
- type: nauc_recall_at_1_max
value: 14.489385038441368
- type: nauc_recall_at_1_std
value: 4.41741897722446
- type: nauc_recall_at_20_diff1
value: 8.614356719100252
- type: nauc_recall_at_20_max
value: 26.785434059132506
- type: nauc_recall_at_20_std
value: 18.55022579096539
- type: nauc_recall_at_3_diff1
value: 11.063544704398076
- type: nauc_recall_at_3_max
value: 21.827384423462075
- type: nauc_recall_at_3_std
value: 8.20600294063751
- type: nauc_recall_at_5_diff1
value: 10.122455882369156
- type: nauc_recall_at_5_max
value: 24.706979708938757
- type: nauc_recall_at_5_std
value: 12.54515446765516
- type: ndcg_at_1
value: 26.1
- type: ndcg_at_10
value: 21.768
- type: ndcg_at_100
value: 30.355999999999998
- type: ndcg_at_1000
value: 35.9
- type: ndcg_at_20
value: 24.924
- type: ndcg_at_3
value: 20.903
- type: ndcg_at_5
value: 18.199
- type: precision_at_1
value: 26.1
- type: precision_at_10
value: 11.16
- type: precision_at_100
value: 2.349
- type: precision_at_1000
value: 0.367
- type: precision_at_20
value: 7.41
- type: precision_at_3
value: 19.433
- type: precision_at_5
value: 15.9
- type: recall_at_1
value: 5.288
- type: recall_at_10
value: 22.634999999999998
- type: recall_at_100
value: 47.698
- type: recall_at_1000
value: 74.593
- type: recall_at_20
value: 30.020000000000003
- type: recall_at_3
value: 11.793
- type: recall_at_5
value: 16.097
task:
type: Retrieval
- dataset:
config: default
name: MTEB SciFact (default)
revision: 0228b52cf27578f30900b9e5271d331663a030d7
split: test
type: mteb/scifact
metrics:
- type: main_score
value: 77.754
- type: map_at_1
value: 62.761
- type: map_at_10
value: 73.463
- type: map_at_100
value: 73.839
- type: map_at_1000
value: 73.854
- type: map_at_20
value: 73.69
- type: map_at_3
value: 70.87400000000001
- type: map_at_5
value: 72.249
- type: mrr_at_1
value: 65.66666666666666
- type: mrr_at_10
value: 74.43624338624336
- type: mrr_at_100
value: 74.71904205166682
- type: mrr_at_1000
value: 74.73399734402419
- type: mrr_at_20
value: 74.589783486494
- type: mrr_at_3
value: 72.72222222222223
- type: mrr_at_5
value: 73.65555555555555
- type: nauc_map_at_1000_diff1
value: 70.00892547294436
- type: nauc_map_at_1000_max
value: 51.41960577804691
- type: nauc_map_at_1000_std
value: 5.978829433471654
- type: nauc_map_at_100_diff1
value: 70.00836052867537
- type: nauc_map_at_100_max
value: 51.42861464065368
- type: nauc_map_at_100_std
value: 5.992256135151107
- type: nauc_map_at_10_diff1
value: 70.15073307409155
- type: nauc_map_at_10_max
value: 51.65496907162298
- type: nauc_map_at_10_std
value: 5.810737915329279
- type: nauc_map_at_1_diff1
value: 71.18129532179381
- type: nauc_map_at_1_max
value: 40.69935323263643
- type: nauc_map_at_1_std
value: -4.136348859696338
- type: nauc_map_at_20_diff1
value: 69.9842505973294
- type: nauc_map_at_20_max
value: 51.57855182212502
- type: nauc_map_at_20_std
value: 6.125238609836613
- type: nauc_map_at_3_diff1
value: 69.69232815430708
- type: nauc_map_at_3_max
value: 49.17028508238125
- type: nauc_map_at_3_std
value: 1.7604096941226137
- type: nauc_map_at_5_diff1
value: 70.90530312747431
- type: nauc_map_at_5_max
value: 50.877509633660644
- type: nauc_map_at_5_std
value: 4.25683085790038
- type: nauc_mrr_at_1000_diff1
value: 69.69945212403626
- type: nauc_mrr_at_1000_max
value: 53.53857495659493
- type: nauc_mrr_at_1000_std
value: 9.537046545129654
- type: nauc_mrr_at_100_diff1
value: 69.69908652848497
- type: nauc_mrr_at_100_max
value: 53.54634044384429
- type: nauc_mrr_at_100_std
value: 9.548345115227077
- type: nauc_mrr_at_10_diff1
value: 69.68940108332458
- type: nauc_mrr_at_10_max
value: 53.78009841857116
- type: nauc_mrr_at_10_std
value: 9.516233608830541
- type: nauc_mrr_at_1_diff1
value: 70.18866680222685
- type: nauc_mrr_at_1_max
value: 46.98369430812099
- type: nauc_mrr_at_1_std
value: 4.2833354400119
- type: nauc_mrr_at_20_diff1
value: 69.64750758771528
- type: nauc_mrr_at_20_max
value: 53.65682728187867
- type: nauc_mrr_at_20_std
value: 9.622798995087647
- type: nauc_mrr_at_3_diff1
value: 69.21638398716401
- type: nauc_mrr_at_3_max
value: 53.60114695403701
- type: nauc_mrr_at_3_std
value: 8.777041524188228
- type: nauc_mrr_at_5_diff1
value: 70.01733919189228
- type: nauc_mrr_at_5_max
value: 54.4843359484408
- type: nauc_mrr_at_5_std
value: 10.26973902416146
- type: nauc_ndcg_at_1000_diff1
value: 69.62702580021256
- type: nauc_ndcg_at_1000_max
value: 53.54352649889886
- type: nauc_ndcg_at_1000_std
value: 8.880318494884355
- type: nauc_ndcg_at_100_diff1
value: 69.46493737196539
- type: nauc_ndcg_at_100_max
value: 53.740641082071804
- type: nauc_ndcg_at_100_std
value: 9.32034924214471
- type: nauc_ndcg_at_10_diff1
value: 69.77719788572776
- type: nauc_ndcg_at_10_max
value: 55.08725100434386
- type: nauc_ndcg_at_10_std
value: 9.17127987828087
- type: nauc_ndcg_at_1_diff1
value: 70.18866680222685
- type: nauc_ndcg_at_1_max
value: 46.98369430812099
- type: nauc_ndcg_at_1_std
value: 4.2833354400119
- type: nauc_ndcg_at_20_diff1
value: 69.3592801336012
- type: nauc_ndcg_at_20_max
value: 54.71325998091494
- type: nauc_ndcg_at_20_std
value: 10.066345792662997
- type: nauc_ndcg_at_3_diff1
value: 68.54848316527975
- type: nauc_ndcg_at_3_max
value: 53.37666510182388
- type: nauc_ndcg_at_3_std
value: 5.813919678224535
- type: nauc_ndcg_at_5_diff1
value: 71.09231090269157
- type: nauc_ndcg_at_5_max
value: 54.763118697264105
- type: nauc_ndcg_at_5_std
value: 8.077244939348736
- type: nauc_precision_at_1000_diff1
value: -35.65939471493537
- type: nauc_precision_at_1000_max
value: 16.222575385441036
- type: nauc_precision_at_1000_std
value: 48.01552083548329
- type: nauc_precision_at_100_diff1
value: -23.189605031989597
- type: nauc_precision_at_100_max
value: 22.043736096370566
- type: nauc_precision_at_100_std
value: 45.47258848854428
- type: nauc_precision_at_10_diff1
value: -0.7615073397022429
- type: nauc_precision_at_10_max
value: 37.45472261645026
- type: nauc_precision_at_10_std
value: 42.330911478743374
- type: nauc_precision_at_1_diff1
value: 70.18866680222685
- type: nauc_precision_at_1_max
value: 46.98369430812099
- type: nauc_precision_at_1_std
value: 4.2833354400119
- type: nauc_precision_at_20_diff1
value: -9.800943666463139
- type: nauc_precision_at_20_max
value: 31.89387265367008
- type: nauc_precision_at_20_std
value: 45.291666441469495
- type: nauc_precision_at_3_diff1
value: 32.87510487623137
- type: nauc_precision_at_3_max
value: 51.426610106841764
- type: nauc_precision_at_3_std
value: 25.24363555289574
- type: nauc_precision_at_5_diff1
value: 22.32745586276285
- type: nauc_precision_at_5_max
value: 48.179145752651515
- type: nauc_precision_at_5_std
value: 35.49639988533587
- type: nauc_recall_at_1000_diff1
value: .nan
- type: nauc_recall_at_1000_max
value: .nan
- type: nauc_recall_at_1000_std
value: .nan
- type: nauc_recall_at_100_diff1
value: 61.42857142857152
- type: nauc_recall_at_100_max
value: 61.545284780578854
- type: nauc_recall_at_100_std
value: 28.216619981325497
- type: nauc_recall_at_10_diff1
value: 68.14227843217638
- type: nauc_recall_at_10_max
value: 68.0438674752541
- type: nauc_recall_at_10_std
value: 17.216039931036985
- type: nauc_recall_at_1_diff1
value: 71.18129532179381
- type: nauc_recall_at_1_max
value: 40.69935323263643
- type: nauc_recall_at_1_std
value: -4.136348859696338
- type: nauc_recall_at_20_diff1
value: 64.65615231599898
- type: nauc_recall_at_20_max
value: 68.82332643202204
- type: nauc_recall_at_20_std
value: 26.45029026103195
- type: nauc_recall_at_3_diff1
value: 66.66056229757102
- type: nauc_recall_at_3_max
value: 57.5874024986744
- type: nauc_recall_at_3_std
value: 5.89101784676189
- type: nauc_recall_at_5_diff1
value: 72.34813575456495
- type: nauc_recall_at_5_max
value: 63.8277779755477
- type: nauc_recall_at_5_std
value: 13.862091061194013
- type: ndcg_at_1
value: 65.667
- type: ndcg_at_10
value: 77.754
- type: ndcg_at_100
value: 79.342
- type: ndcg_at_1000
value: 79.76299999999999
- type: ndcg_at_20
value: 78.468
- type: ndcg_at_3
value: 73.687
- type: ndcg_at_5
value: 75.50099999999999
- type: precision_at_1
value: 65.667
- type: precision_at_10
value: 10.167
- type: precision_at_100
value: 1.097
- type: precision_at_1000
value: 0.11299999999999999
- type: precision_at_20
value: 5.25
- type: precision_at_3
value: 28.666999999999998
- type: precision_at_5
value: 18.533
- type: recall_at_1
value: 62.761
- type: recall_at_10
value: 89.60000000000001
- type: recall_at_100
value: 96.667
- type: recall_at_1000
value: 100
- type: recall_at_20
value: 92.333
- type: recall_at_3
value: 78.822
- type: recall_at_5
value: 83.39999999999999
task:
type: Retrieval
- dataset:
config: default
name: MTEB T2Retrieval (default)
revision: 8731a845f1bf500a4f111cf1070785c793d10e64
split: test
type: C-MTEB/T2Retrieval
metrics:
- type: main_score
value: 85.408
- type: map_at_1
value: 27.974
- type: map_at_10
value: 78.107
- type: map_at_100
value: 81.606
- type: map_at_1000
value: 81.669
- type: map_at_20
value: 80.784
- type: map_at_3
value: 55.283
- type: map_at_5
value: 67.72399999999999
- type: mrr_at_1
value: 91.070489216202
- type: mrr_at_10
value: 93.20924798699666
- type: mrr_at_100
value: 93.2897391797075
- type: mrr_at_1000
value: 93.29251314252166
- type: mrr_at_20
value: 93.26295707737242
- type: mrr_at_3
value: 92.86778888304391
- type: mrr_at_5
value: 93.08609503769956
- type: nauc_map_at_1000_diff1
value: 14.02161421364639
- type: nauc_map_at_1000_max
value: 47.186571672677054
- type: nauc_map_at_1000_std
value: 20.616875530058905
- type: nauc_map_at_100_diff1
value: 14.015808279665501
- type: nauc_map_at_100_max
value: 47.064027752077116
- type: nauc_map_at_100_std
value: 20.510779071993177
- type: nauc_map_at_10_diff1
value: 17.859573746844152
- type: nauc_map_at_10_max
value: 33.99524263062824
- type: nauc_map_at_10_std
value: 2.9750580972472327
- type: nauc_map_at_1_diff1
value: 52.955344727933515
- type: nauc_map_at_1_max
value: -25.84268384364316
- type: nauc_map_at_1_std
value: -41.01798171453465
- type: nauc_map_at_20_diff1
value: 14.590278001943377
- type: nauc_map_at_20_max
value: 44.555133043897854
- type: nauc_map_at_20_std
value: 16.847807075388115
- type: nauc_map_at_3_diff1
value: 38.01293816018024
- type: nauc_map_at_3_max
value: -13.32573886546727
- type: nauc_map_at_3_std
value: -39.139010666587886
- type: nauc_map_at_5_diff1
value: 29.93221195702576
- type: nauc_map_at_5_max
value: 3.7008458479169537
- type: nauc_map_at_5_std
value: -27.496871606312183
- type: nauc_mrr_at_1000_diff1
value: 52.21254036979167
- type: nauc_mrr_at_1000_max
value: 81.88050840815251
- type: nauc_mrr_at_1000_std
value: 47.9830708984396
- type: nauc_mrr_at_100_diff1
value: 52.213514352113656
- type: nauc_mrr_at_100_max
value: 81.8865599907679
- type: nauc_mrr_at_100_std
value: 47.99621895256341
- type: nauc_mrr_at_10_diff1
value: 52.2488647184239
- type: nauc_mrr_at_10_max
value: 81.98121309685763
- type: nauc_mrr_at_10_std
value: 48.01382808881993
- type: nauc_mrr_at_1_diff1
value: 52.18512162818067
- type: nauc_mrr_at_1_max
value: 78.17497748037101
- type: nauc_mrr_at_1_std
value: 42.12866110712485
- type: nauc_mrr_at_20_diff1
value: 52.2244682517749
- type: nauc_mrr_at_20_max
value: 81.92327282809062
- type: nauc_mrr_at_20_std
value: 48.031963018453574
- type: nauc_mrr_at_3_diff1
value: 52.22511084737672
- type: nauc_mrr_at_3_max
value: 81.86052316487371
- type: nauc_mrr_at_3_std
value: 47.71493909281435
- type: nauc_mrr_at_5_diff1
value: 52.21838855843384
- type: nauc_mrr_at_5_max
value: 82.09279480475385
- type: nauc_mrr_at_5_std
value: 48.15370258286536
- type: nauc_ndcg_at_1000_diff1
value: 19.082798798848845
- type: nauc_ndcg_at_1000_max
value: 59.979802333634815
- type: nauc_ndcg_at_1000_std
value: 34.690664674450716
- type: nauc_ndcg_at_100_diff1
value: 18.539024016306367
- type: nauc_ndcg_at_100_max
value: 58.65448008268863
- type: nauc_ndcg_at_100_std
value: 34.04253795370821
- type: nauc_ndcg_at_10_diff1
value: 18.172957278807868
- type: nauc_ndcg_at_10_max
value: 49.2049503163979
- type: nauc_ndcg_at_10_std
value: 21.46955810647442
- type: nauc_ndcg_at_1_diff1
value: 52.18512162818067
- type: nauc_ndcg_at_1_max
value: 78.17497748037101
- type: nauc_ndcg_at_1_std
value: 42.12866110712485
- type: nauc_ndcg_at_20_diff1
value: 18.379027735832178
- type: nauc_ndcg_at_20_max
value: 53.23513838907849
- type: nauc_ndcg_at_20_std
value: 26.7601010446773
- type: nauc_ndcg_at_3_diff1
value: 14.679608635115319
- type: nauc_ndcg_at_3_max
value: 66.67606724204819
- type: nauc_ndcg_at_3_std
value: 37.824882125811904
- type: nauc_ndcg_at_5_diff1
value: 14.713093689867252
- type: nauc_ndcg_at_5_max
value: 58.614360345482154
- type: nauc_ndcg_at_5_std
value: 31.12595430971059
- type: nauc_precision_at_1000_diff1
value: -33.13227819808231
- type: nauc_precision_at_1000_max
value: 48.59634908517778
- type: nauc_precision_at_1000_std
value: 66.00782368965828
- type: nauc_precision_at_100_diff1
value: -33.173707177959336
- type: nauc_precision_at_100_max
value: 50.03358290470462
- type: nauc_precision_at_100_std
value: 67.05786873559285
- type: nauc_precision_at_10_diff1
value: -33.78943458913884
- type: nauc_precision_at_10_max
value: 54.76809833047381
- type: nauc_precision_at_10_std
value: 62.590134814916986
- type: nauc_precision_at_1_diff1
value: 52.18512162818067
- type: nauc_precision_at_1_max
value: 78.17497748037101
- type: nauc_precision_at_1_std
value: 42.12866110712485
- type: nauc_precision_at_20_diff1
value: -33.3169918994365
- type: nauc_precision_at_20_max
value: 52.62730418313208
- type: nauc_precision_at_20_std
value: 66.07927210284211
- type: nauc_precision_at_3_diff1
value: -31.74995396944038
- type: nauc_precision_at_3_max
value: 65.95961370994449
- type: nauc_precision_at_3_std
value: 55.08242524554448
- type: nauc_precision_at_5_diff1
value: -35.01473556966971
- type: nauc_precision_at_5_max
value: 60.369530978392824
- type: nauc_precision_at_5_std
value: 58.55729742429819
- type: nauc_recall_at_1000_diff1
value: 7.369474595497491
- type: nauc_recall_at_1000_max
value: 62.759393411790974
- type: nauc_recall_at_1000_std
value: 64.15678271186357
- type: nauc_recall_at_100_diff1
value: 8.204319572435734
- type: nauc_recall_at_100_max
value: 50.39692582229378
- type: nauc_recall_at_100_std
value: 44.669032653919814
- type: nauc_recall_at_10_diff1
value: 16.65018200107064
- type: nauc_recall_at_10_max
value: 23.77056248647486
- type: nauc_recall_at_10_std
value: -4.164976890234266
- type: nauc_recall_at_1_diff1
value: 52.955344727933515
- type: nauc_recall_at_1_max
value: -25.84268384364316
- type: nauc_recall_at_1_std
value: -41.01798171453465
- type: nauc_recall_at_20_diff1
value: 11.026989567642152
- type: nauc_recall_at_20_max
value: 38.82346627365128
- type: nauc_recall_at_20_std
value: 19.261774770548087
- type: nauc_recall_at_3_diff1
value: 36.44902805248335
- type: nauc_recall_at_3_max
value: -17.66694139657555
- type: nauc_recall_at_3_std
value: -41.82560813117898
- type: nauc_recall_at_5_diff1
value: 29.001950616630435
- type: nauc_recall_at_5_max
value: -4.623916723824471
- type: nauc_recall_at_5_std
value: -33.60086136077999
- type: ndcg_at_1
value: 91.07
- type: ndcg_at_10
value: 85.408
- type: ndcg_at_100
value: 88.767
- type: ndcg_at_1000
value: 89.384
- type: ndcg_at_20
value: 87.03
- type: ndcg_at_3
value: 87.168
- type: ndcg_at_5
value: 85.622
- type: precision_at_1
value: 91.07
- type: precision_at_10
value: 42.219
- type: precision_at_100
value: 5.002
- type: precision_at_1000
value: 0.515
- type: precision_at_20
value: 23.344
- type: precision_at_3
value: 76.175
- type: precision_at_5
value: 63.632999999999996
- type: recall_at_1
value: 27.974
- type: recall_at_10
value: 83.974
- type: recall_at_100
value: 95.06400000000001
- type: recall_at_1000
value: 98.207
- type: recall_at_20
value: 89.566
- type: recall_at_3
value: 56.830999999999996
- type: recall_at_5
value: 70.769
task:
type: Retrieval
- dataset:
config: default
name: MTEB TRECCOVID (default)
revision: bb9466bac8153a0349341eb1b22e06409e78ef4e
split: test
type: mteb/trec-covid
metrics:
- type: main_score
value: 80.46
- type: map_at_1
value: 0.22899999999999998
- type: map_at_10
value: 2.0140000000000002
- type: map_at_100
value: 13.531
- type: map_at_1000
value: 32.993
- type: map_at_20
value: 3.846
- type: map_at_3
value: 0.654
- type: map_at_5
value: 1.0619999999999998
- type: mrr_at_1
value: 88
- type: mrr_at_10
value: 94
- type: mrr_at_100
value: 94
- type: mrr_at_1000
value: 94
- type: mrr_at_20
value: 94
- type: mrr_at_3
value: 94
- type: mrr_at_5
value: 94
- type: nauc_map_at_1000_diff1
value: -3.3698499398528785
- type: nauc_map_at_1000_max
value: 45.445709400181414
- type: nauc_map_at_1000_std
value: 74.92334681323416
- type: nauc_map_at_100_diff1
value: -0.9733247625101976
- type: nauc_map_at_100_max
value: 13.09104596763804
- type: nauc_map_at_100_std
value: 48.19460362265014
- type: nauc_map_at_10_diff1
value: -6.228259203343852
- type: nauc_map_at_10_max
value: -11.979703047874356
- type: nauc_map_at_10_std
value: 7.951041219330085
- type: nauc_map_at_1_diff1
value: 17.419304474427516
- type: nauc_map_at_1_max
value: -17.074884091442755
- type: nauc_map_at_1_std
value: -1.9267809596597902
- type: nauc_map_at_20_diff1
value: -2.6496424260949722
- type: nauc_map_at_20_max
value: -8.158038255243799
- type: nauc_map_at_20_std
value: 15.206187928855147
- type: nauc_map_at_3_diff1
value: -0.4999760890617131
- type: nauc_map_at_3_max
value: -18.019567439367997
- type: nauc_map_at_3_std
value: 3.360209032763635
- type: nauc_map_at_5_diff1
value: -3.7052862954357835
- type: nauc_map_at_5_max
value: -16.74492876700498
- type: nauc_map_at_5_std
value: -0.5357800964930773
- type: nauc_mrr_at_1000_diff1
value: -11.979000954502055
- type: nauc_mrr_at_1000_max
value: 13.434616608336228
- type: nauc_mrr_at_1000_std
value: 51.48743238943712
- type: nauc_mrr_at_100_diff1
value: -11.979000954502055
- type: nauc_mrr_at_100_max
value: 13.434616608336228
- type: nauc_mrr_at_100_std
value: 51.48743238943712
- type: nauc_mrr_at_10_diff1
value: -11.979000954502055
- type: nauc_mrr_at_10_max
value: 13.434616608336228
- type: nauc_mrr_at_10_std
value: 51.48743238943712
- type: nauc_mrr_at_1_diff1
value: -11.979000954502169
- type: nauc_mrr_at_1_max
value: 13.434616608336004
- type: nauc_mrr_at_1_std
value: 51.48743238943678
- type: nauc_mrr_at_20_diff1
value: -11.979000954502055
- type: nauc_mrr_at_20_max
value: 13.434616608336228
- type: nauc_mrr_at_20_std
value: 51.48743238943712
- type: nauc_mrr_at_3_diff1
value: -11.979000954502055
- type: nauc_mrr_at_3_max
value: 13.434616608336228
- type: nauc_mrr_at_3_std
value: 51.48743238943712
- type: nauc_mrr_at_5_diff1
value: -11.979000954502055
- type: nauc_mrr_at_5_max
value: 13.434616608336228
- type: nauc_mrr_at_5_std
value: 51.48743238943712
- type: nauc_ndcg_at_1000_diff1
value: -8.71892178242702
- type: nauc_ndcg_at_1000_max
value: 43.18986143410146
- type: nauc_ndcg_at_1000_std
value: 73.79391175021726
- type: nauc_ndcg_at_100_diff1
value: -20.94835546481927
- type: nauc_ndcg_at_100_max
value: 43.26793350899254
- type: nauc_ndcg_at_100_std
value: 73.36033827657992
- type: nauc_ndcg_at_10_diff1
value: -45.88213133678636
- type: nauc_ndcg_at_10_max
value: 25.13057874192747
- type: nauc_ndcg_at_10_std
value: 59.133806456446194
- type: nauc_ndcg_at_1_diff1
value: -33.88545395379104
- type: nauc_ndcg_at_1_max
value: 8.330621542466726
- type: nauc_ndcg_at_1_std
value: 54.98861047836
- type: nauc_ndcg_at_20_diff1
value: -40.815265043494634
- type: nauc_ndcg_at_20_max
value: 32.4919156896144
- type: nauc_ndcg_at_20_std
value: 66.80157913099653
- type: nauc_ndcg_at_3_diff1
value: -43.89517591868999
- type: nauc_ndcg_at_3_max
value: 3.1046688999223795
- type: nauc_ndcg_at_3_std
value: 49.264714822436495
- type: nauc_ndcg_at_5_diff1
value: -48.47646523003602
- type: nauc_ndcg_at_5_max
value: 13.063986525977134
- type: nauc_ndcg_at_5_std
value: 47.217180495399
- type: nauc_precision_at_1000_diff1
value: -4.274567070500386
- type: nauc_precision_at_1000_max
value: 55.59099307815999
- type: nauc_precision_at_1000_std
value: 43.903773352951184
- type: nauc_precision_at_100_diff1
value: -17.698958966671068
- type: nauc_precision_at_100_max
value: 48.857497250056916
- type: nauc_precision_at_100_std
value: 76.64422233464362
- type: nauc_precision_at_10_diff1
value: -45.993800160046064
- type: nauc_precision_at_10_max
value: 37.681508472337676
- type: nauc_precision_at_10_std
value: 63.205736794263245
- type: nauc_precision_at_1_diff1
value: -11.979000954502169
- type: nauc_precision_at_1_max
value: 13.434616608336004
- type: nauc_precision_at_1_std
value: 51.48743238943678
- type: nauc_precision_at_20_diff1
value: -36.731647449437105
- type: nauc_precision_at_20_max
value: 40.23960942279067
- type: nauc_precision_at_20_std
value: 73.5602466884422
- type: nauc_precision_at_3_diff1
value: -40.0823995755172
- type: nauc_precision_at_3_max
value: 17.756484284777965
- type: nauc_precision_at_3_std
value: 55.28574549767477
- type: nauc_precision_at_5_diff1
value: -44.943344455547994
- type: nauc_precision_at_5_max
value: 24.72519096898761
- type: nauc_precision_at_5_std
value: 46.23397300181218
- type: nauc_recall_at_1000_diff1
value: 0.7802131475089158
- type: nauc_recall_at_1000_max
value: 39.995727295561565
- type: nauc_recall_at_1000_std
value: 62.98335386556941
- type: nauc_recall_at_100_diff1
value: 2.880922928015335
- type: nauc_recall_at_100_max
value: 3.0979967654662546
- type: nauc_recall_at_100_std
value: 33.62736021544352
- type: nauc_recall_at_10_diff1
value: -2.1458012939001376
- type: nauc_recall_at_10_max
value: -16.3216625701991
- type: nauc_recall_at_10_std
value: -0.3582401567691279
- type: nauc_recall_at_1_diff1
value: 17.419304474427516
- type: nauc_recall_at_1_max
value: -17.074884091442755
- type: nauc_recall_at_1_std
value: -1.9267809596597902
- type: nauc_recall_at_20_diff1
value: 4.064773126987082
- type: nauc_recall_at_20_max
value: -14.464337876692632
- type: nauc_recall_at_20_std
value: 3.6886303476512303
- type: nauc_recall_at_3_diff1
value: -2.400164192124634
- type: nauc_recall_at_3_max
value: -20.616624939897328
- type: nauc_recall_at_3_std
value: -3.107132696710774
- type: nauc_recall_at_5_diff1
value: -0.2723579989577546
- type: nauc_recall_at_5_max
value: -19.575511541653118
- type: nauc_recall_at_5_std
value: -8.616141856565983
- type: ndcg_at_1
value: 85
- type: ndcg_at_10
value: 80.46
- type: ndcg_at_100
value: 65.167
- type: ndcg_at_1000
value: 58.385
- type: ndcg_at_20
value: 78.226
- type: ndcg_at_3
value: 84.776
- type: ndcg_at_5
value: 82.841
- type: precision_at_1
value: 88
- type: precision_at_10
value: 85
- type: precision_at_100
value: 67.4
- type: precision_at_1000
value: 25.872
- type: precision_at_20
value: 83.2
- type: precision_at_3
value: 88.667
- type: precision_at_5
value: 86.8
- type: recall_at_1
value: 0.22899999999999998
- type: recall_at_10
value: 2.198
- type: recall_at_100
value: 16.436
- type: recall_at_1000
value: 55.028
- type: recall_at_20
value: 4.311
- type: recall_at_3
value: 0.687
- type: recall_at_5
value: 1.139
task:
type: Retrieval
- dataset:
config: default
name: MTEB Touche2020 (default)
revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f
split: test
type: mteb/touche2020
metrics:
- type: main_score
value: 32.391
- type: map_at_1
value: 3.11
- type: map_at_10
value: 13.324
- type: map_at_100
value: 20.766000000000002
- type: map_at_1000
value: 22.358
- type: map_at_20
value: 16.604
- type: map_at_3
value: 7.256
- type: map_at_5
value: 9.682
- type: mrr_at_1
value: 44.89795918367347
- type: mrr_at_10
value: 59.203109815354715
- type: mrr_at_100
value: 59.650478274342866
- type: mrr_at_1000
value: 59.650478274342866
- type: mrr_at_20
value: 59.49322586177328
- type: mrr_at_3
value: 55.78231292517006
- type: mrr_at_5
value: 58.02721088435374
- type: nauc_map_at_1000_diff1
value: -0.6118420298552262
- type: nauc_map_at_1000_max
value: -10.459268048435016
- type: nauc_map_at_1000_std
value: 0.12875167804448237
- type: nauc_map_at_100_diff1
value: 0.034899904451799156
- type: nauc_map_at_100_max
value: -11.410978367647376
- type: nauc_map_at_100_std
value: -1.8187273407892068
- type: nauc_map_at_10_diff1
value: 3.798983081181108
- type: nauc_map_at_10_max
value: -12.81890421998731
- type: nauc_map_at_10_std
value: -16.10507951565588
- type: nauc_map_at_1_diff1
value: 14.97446890875931
- type: nauc_map_at_1_max
value: -12.221096391517113
- type: nauc_map_at_1_std
value: -5.911340708079087
- type: nauc_map_at_20_diff1
value: -0.39358655498774253
- type: nauc_map_at_20_max
value: -15.29885453445688
- type: nauc_map_at_20_std
value: -12.961858056841008
- type: nauc_map_at_3_diff1
value: 9.380618934451833
- type: nauc_map_at_3_max
value: -7.205679861124378
- type: nauc_map_at_3_std
value: -15.013412162167935
- type: nauc_map_at_5_diff1
value: 4.088519896755799
- type: nauc_map_at_5_max
value: -11.02285555747699
- type: nauc_map_at_5_std
value: -13.175770655883321
- type: nauc_mrr_at_1000_diff1
value: 25.891530000669285
- type: nauc_mrr_at_1000_max
value: -7.362255923471247
- type: nauc_mrr_at_1000_std
value: 12.349814765006057
- type: nauc_mrr_at_100_diff1
value: 25.891530000669285
- type: nauc_mrr_at_100_max
value: -7.362255923471247
- type: nauc_mrr_at_100_std
value: 12.349814765006057
- type: nauc_mrr_at_10_diff1
value: 25.905118677615924
- type: nauc_mrr_at_10_max
value: -7.379407993095785
- type: nauc_mrr_at_10_std
value: 12.678211902714907
- type: nauc_mrr_at_1_diff1
value: 25.264967101595392
- type: nauc_mrr_at_1_max
value: -8.802783567339088
- type: nauc_mrr_at_1_std
value: 6.8981595305490275
- type: nauc_mrr_at_20_diff1
value: 26.130098215553716
- type: nauc_mrr_at_20_max
value: -7.124888191805823
- type: nauc_mrr_at_20_std
value: 12.831151805386995
- type: nauc_mrr_at_3_diff1
value: 21.463610246401245
- type: nauc_mrr_at_3_max
value: -8.246508754437377
- type: nauc_mrr_at_3_std
value: 7.541159898957048
- type: nauc_mrr_at_5_diff1
value: 25.356160853384996
- type: nauc_mrr_at_5_max
value: -8.025890546642223
- type: nauc_mrr_at_5_std
value: 12.516569298584516
- type: nauc_ndcg_at_1000_diff1
value: 10.562268378470806
- type: nauc_ndcg_at_1000_max
value: 1.8176645853292939
- type: nauc_ndcg_at_1000_std
value: 25.447937824378407
- type: nauc_ndcg_at_100_diff1
value: 13.737510141791622
- type: nauc_ndcg_at_100_max
value: -6.293379340626089
- type: nauc_ndcg_at_100_std
value: 21.19589523378754
- type: nauc_ndcg_at_10_diff1
value: 12.226992779324847
- type: nauc_ndcg_at_10_max
value: -10.729165359819161
- type: nauc_ndcg_at_10_std
value: 2.152554877119559
- type: nauc_ndcg_at_1_diff1
value: 22.271721472718987
- type: nauc_ndcg_at_1_max
value: -9.589388388269104
- type: nauc_ndcg_at_1_std
value: 10.040277845055297
- type: nauc_ndcg_at_20_diff1
value: 10.259537135397967
- type: nauc_ndcg_at_20_max
value: -14.92156316466216
- type: nauc_ndcg_at_20_std
value: -0.40555881794020615
- type: nauc_ndcg_at_3_diff1
value: 16.530086684928193
- type: nauc_ndcg_at_3_max
value: -3.7008246097133624
- type: nauc_ndcg_at_3_std
value: 2.700263509520849
- type: nauc_ndcg_at_5_diff1
value: 9.415775332356283
- type: nauc_ndcg_at_5_max
value: -8.885134891124261
- type: nauc_ndcg_at_5_std
value: 8.723001465752747
- type: nauc_precision_at_1000_diff1
value: -19.141537060734514
- type: nauc_precision_at_1000_max
value: 43.24279932811497
- type: nauc_precision_at_1000_std
value: 33.807943139065955
- type: nauc_precision_at_100_diff1
value: 0.8205778736819969
- type: nauc_precision_at_100_max
value: 26.564541682215147
- type: nauc_precision_at_100_std
value: 68.16585654468182
- type: nauc_precision_at_10_diff1
value: 5.338469479012317
- type: nauc_precision_at_10_max
value: -6.987141914817638
- type: nauc_precision_at_10_std
value: 8.991544272328849
- type: nauc_precision_at_1_diff1
value: 25.264967101595392
- type: nauc_precision_at_1_max
value: -8.802783567339088
- type: nauc_precision_at_1_std
value: 6.8981595305490275
- type: nauc_precision_at_20_diff1
value: -1.3199014118668273
- type: nauc_precision_at_20_max
value: -8.369989085582542
- type: nauc_precision_at_20_std
value: 17.18610407974423
- type: nauc_precision_at_3_diff1
value: 14.51501551852763
- type: nauc_precision_at_3_max
value: 1.4412777565815174
- type: nauc_precision_at_3_std
value: -3.5849467506167354
- type: nauc_precision_at_5_diff1
value: 3.8592251130664836
- type: nauc_precision_at_5_max
value: -9.031948602868964
- type: nauc_precision_at_5_std
value: 7.956916525964915
- type: nauc_recall_at_1000_diff1
value: 8.425702252781758
- type: nauc_recall_at_1000_max
value: 29.06584945450092
- type: nauc_recall_at_1000_std
value: 66.77406431605769
- type: nauc_recall_at_100_diff1
value: 13.179777935665497
- type: nauc_recall_at_100_max
value: -8.215791450563307
- type: nauc_recall_at_100_std
value: 29.691010455649458
- type: nauc_recall_at_10_diff1
value: 6.674660601983981
- type: nauc_recall_at_10_max
value: -17.337651319937034
- type: nauc_recall_at_10_std
value: -13.353609112306215
- type: nauc_recall_at_1_diff1
value: 14.97446890875931
- type: nauc_recall_at_1_max
value: -12.221096391517113
- type: nauc_recall_at_1_std
value: -5.911340708079087
- type: nauc_recall_at_20_diff1
value: 3.0314480639365335
- type: nauc_recall_at_20_max
value: -21.890125966228695
- type: nauc_recall_at_20_std
value: -8.320962736647811
- type: nauc_recall_at_3_diff1
value: 8.044938435901074
- type: nauc_recall_at_3_max
value: -8.943571362493486
- type: nauc_recall_at_3_std
value: -16.597925007178382
- type: nauc_recall_at_5_diff1
value: 2.2653542126136887
- type: nauc_recall_at_5_max
value: -16.759671951395465
- type: nauc_recall_at_5_std
value: -10.852252699482321
- type: ndcg_at_1
value: 41.837
- type: ndcg_at_10
value: 32.391
- type: ndcg_at_100
value: 44.536
- type: ndcg_at_1000
value: 55.162
- type: ndcg_at_20
value: 32.759
- type: ndcg_at_3
value: 38.673
- type: ndcg_at_5
value: 36.299
- type: precision_at_1
value: 44.897999999999996
- type: precision_at_10
value: 27.551
- type: precision_at_100
value: 9.061
- type: precision_at_1000
value: 1.637
- type: precision_at_20
value: 20.918
- type: precision_at_3
value: 40.136
- type: precision_at_5
value: 35.918
- type: recall_at_1
value: 3.11
- type: recall_at_10
value: 19.712
- type: recall_at_100
value: 55.24
- type: recall_at_1000
value: 88.009
- type: recall_at_20
value: 29.076999999999998
- type: recall_at_3
value: 8.444
- type: recall_at_5
value: 12.424
task:
type: Retrieval
- dataset:
config: default
name: MTEB VideoRetrieval (default)
revision: 58c2597a5943a2ba48f4668c3b90d796283c5639
split: test
type: C-MTEB/VideoRetrieval
metrics:
- type: main_score
value: 76.889
- type: map_at_1
value: 63.9
- type: map_at_10
value: 72.841
- type: map_at_100
value: 73.2
- type: map_at_1000
value: 73.208
- type: map_at_20
value: 73.105
- type: map_at_3
value: 71.15
- type: map_at_5
value: 72.14500000000001
- type: mrr_at_1
value: 63.9
- type: mrr_at_10
value: 72.84071428571434
- type: mrr_at_100
value: 73.19985132359601
- type: mrr_at_1000
value: 73.20847798600303
- type: mrr_at_20
value: 73.10518399685974
- type: mrr_at_3
value: 71.15000000000006
- type: mrr_at_5
value: 72.14500000000005
- type: nauc_map_at_1000_diff1
value: 73.8925912463481
- type: nauc_map_at_1000_max
value: 26.00690333807939
- type: nauc_map_at_1000_std
value: -40.624829088300054
- type: nauc_map_at_100_diff1
value: 73.88530931992535
- type: nauc_map_at_100_max
value: 26.026860107313766
- type: nauc_map_at_100_std
value: -40.6028828700579
- type: nauc_map_at_10_diff1
value: 73.89360151928338
- type: nauc_map_at_10_max
value: 26.13928571616993
- type: nauc_map_at_10_std
value: -41.05568100371164
- type: nauc_map_at_1_diff1
value: 76.4998637465399
- type: nauc_map_at_1_max
value: 21.06685662470698
- type: nauc_map_at_1_std
value: -38.00547244891296
- type: nauc_map_at_20_diff1
value: 73.84940921114446
- type: nauc_map_at_20_max
value: 26.087007693656954
- type: nauc_map_at_20_std
value: -40.71205453291879
- type: nauc_map_at_3_diff1
value: 73.81203219656123
- type: nauc_map_at_3_max
value: 25.150278168586997
- type: nauc_map_at_3_std
value: -42.602748531734086
- type: nauc_map_at_5_diff1
value: 73.67289623846195
- type: nauc_map_at_5_max
value: 26.196656627248256
- type: nauc_map_at_5_std
value: -41.62017040659859
- type: nauc_mrr_at_1000_diff1
value: 73.8925912463481
- type: nauc_mrr_at_1000_max
value: 26.00690333807939
- type: nauc_mrr_at_1000_std
value: -40.624829088300054
- type: nauc_mrr_at_100_diff1
value: 73.88530931992535
- type: nauc_mrr_at_100_max
value: 26.026860107313766
- type: nauc_mrr_at_100_std
value: -40.6028828700579
- type: nauc_mrr_at_10_diff1
value: 73.89360151928338
- type: nauc_mrr_at_10_max
value: 26.13928571616993
- type: nauc_mrr_at_10_std
value: -41.05568100371164
- type: nauc_mrr_at_1_diff1
value: 76.4998637465399
- type: nauc_mrr_at_1_max
value: 21.06685662470698
- type: nauc_mrr_at_1_std
value: -38.00547244891296
- type: nauc_mrr_at_20_diff1
value: 73.84940921114446
- type: nauc_mrr_at_20_max
value: 26.087007693656954
- type: nauc_mrr_at_20_std
value: -40.71205453291879
- type: nauc_mrr_at_3_diff1
value: 73.81203219656123
- type: nauc_mrr_at_3_max
value: 25.150278168586997
- type: nauc_mrr_at_3_std
value: -42.602748531734086
- type: nauc_mrr_at_5_diff1
value: 73.67289623846195
- type: nauc_mrr_at_5_max
value: 26.196656627248256
- type: nauc_mrr_at_5_std
value: -41.62017040659859
- type: nauc_ndcg_at_1000_diff1
value: 73.3437735456378
- type: nauc_ndcg_at_1000_max
value: 28.039245572901166
- type: nauc_ndcg_at_1000_std
value: -38.73993129535837
- type: nauc_ndcg_at_100_diff1
value: 73.14753266308807
- type: nauc_ndcg_at_100_max
value: 28.603678343054177
- type: nauc_ndcg_at_100_std
value: -38.073927474904096
- type: nauc_ndcg_at_10_diff1
value: 73.02013056288126
- type: nauc_ndcg_at_10_max
value: 29.170267818434247
- type: nauc_ndcg_at_10_std
value: -40.59407249225534
- type: nauc_ndcg_at_1_diff1
value: 76.4998637465399
- type: nauc_ndcg_at_1_max
value: 21.06685662470698
- type: nauc_ndcg_at_1_std
value: -38.00547244891296
- type: nauc_ndcg_at_20_diff1
value: 72.84448571022757
- type: nauc_ndcg_at_20_max
value: 29.0382168094389
- type: nauc_ndcg_at_20_std
value: -39.24432359606789
- type: nauc_ndcg_at_3_diff1
value: 72.82142120953982
- type: nauc_ndcg_at_3_max
value: 26.817489018075825
- type: nauc_ndcg_at_3_std
value: -44.19410460765078
- type: nauc_ndcg_at_5_diff1
value: 72.49464120259944
- type: nauc_ndcg_at_5_max
value: 29.020019052984235
- type: nauc_ndcg_at_5_std
value: -42.32190925689385
- type: nauc_precision_at_1000_diff1
value: 68.72704637410462
- type: nauc_precision_at_1000_max
value: 82.06037970743692
- type: nauc_precision_at_1000_std
value: 74.36041083099792
- type: nauc_precision_at_100_diff1
value: 62.34149573807953
- type: nauc_precision_at_100_max
value: 79.95843498689827
- type: nauc_precision_at_100_std
value: 46.918767507002784
- type: nauc_precision_at_10_diff1
value: 67.86899914992605
- type: nauc_precision_at_10_max
value: 49.596438638092216
- type: nauc_precision_at_10_std
value: -34.95861482707724
- type: nauc_precision_at_1_diff1
value: 76.4998637465399
- type: nauc_precision_at_1_max
value: 21.06685662470698
- type: nauc_precision_at_1_std
value: -38.00547244891296
- type: nauc_precision_at_20_diff1
value: 63.504421376393374
- type: nauc_precision_at_20_max
value: 58.752128302300854
- type: nauc_precision_at_20_std
value: -17.8907837644867
- type: nauc_precision_at_3_diff1
value: 69.11433596966778
- type: nauc_precision_at_3_max
value: 33.125013994382485
- type: nauc_precision_at_3_std
value: -50.10585972777495
- type: nauc_precision_at_5_diff1
value: 67.12901463705309
- type: nauc_precision_at_5_max
value: 42.36210257110566
- type: nauc_precision_at_5_std
value: -44.939105389266324
- type: nauc_recall_at_1000_diff1
value: 68.72704637410567
- type: nauc_recall_at_1000_max
value: 82.06037970743924
- type: nauc_recall_at_1000_std
value: 74.3604108309996
- type: nauc_recall_at_100_diff1
value: 62.34149573808014
- type: nauc_recall_at_100_max
value: 79.95843498689804
- type: nauc_recall_at_100_std
value: 46.91876750700249
- type: nauc_recall_at_10_diff1
value: 67.86899914992618
- type: nauc_recall_at_10_max
value: 49.59643863809242
- type: nauc_recall_at_10_std
value: -34.958614827076914
- type: nauc_recall_at_1_diff1
value: 76.4998637465399
- type: nauc_recall_at_1_max
value: 21.06685662470698
- type: nauc_recall_at_1_std
value: -38.00547244891296
- type: nauc_recall_at_20_diff1
value: 63.504421376393836
- type: nauc_recall_at_20_max
value: 58.75212830230132
- type: nauc_recall_at_20_std
value: -17.890783764486297
- type: nauc_recall_at_3_diff1
value: 69.11433596966754
- type: nauc_recall_at_3_max
value: 33.1250139943824
- type: nauc_recall_at_3_std
value: -50.10585972777511
- type: nauc_recall_at_5_diff1
value: 67.12901463705325
- type: nauc_recall_at_5_max
value: 42.36210257110583
- type: nauc_recall_at_5_std
value: -44.9391053892661
- type: ndcg_at_1
value: 63.9
- type: ndcg_at_10
value: 76.889
- type: ndcg_at_100
value: 78.521
- type: ndcg_at_1000
value: 78.729
- type: ndcg_at_20
value: 77.835
- type: ndcg_at_3
value: 73.406
- type: ndcg_at_5
value: 75.187
- type: precision_at_1
value: 63.9
- type: precision_at_10
value: 8.95
- type: precision_at_100
value: 0.9690000000000001
- type: precision_at_1000
value: 0.099
- type: precision_at_20
value: 4.66
- type: precision_at_3
value: 26.633000000000003
- type: precision_at_5
value: 16.84
- type: recall_at_1
value: 63.9
- type: recall_at_10
value: 89.5
- type: recall_at_100
value: 96.89999999999999
- type: recall_at_1000
value: 98.5
- type: recall_at_20
value: 93.2
- type: recall_at_3
value: 79.9
- type: recall_at_5
value: 84.2
task:
type: Retrieval
tags:
- mteb