jstephencorey commited on
Commit
e0d1737
·
verified ·
1 Parent(s): 01d618f

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +1999 -152
README.md CHANGED
@@ -64,6 +64,21 @@ model-index:
64
  value: 14.367489440317666
65
  - type: f1
66
  value: 50.48473578289779
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
  - task:
68
  type: Classification
69
  dataset:
@@ -211,6 +226,17 @@ model-index:
211
  value: 12.447
212
  - type: recall_at_5
213
  value: 16.145
 
 
 
 
 
 
 
 
 
 
 
214
  - task:
215
  type: Clustering
216
  dataset:
@@ -1119,259 +1145,1930 @@ model-index:
1119
  value: 6.784
1120
  - type: recall_at_5
1121
  value: 8.17
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1122
  - task:
1123
  type: Classification
1124
  dataset:
1125
- type: mteb/emotion
1126
- name: MTEB EmotionClassification
1127
- config: default
1128
  split: test
1129
- revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
1130
  metrics:
1131
  - type: accuracy
1132
- value: 23.279999999999998
1133
  - type: f1
1134
- value: 19.87865985032945
1135
  - task:
1136
- type: Retrieval
1137
  dataset:
1138
- type: fiqa
1139
- name: MTEB FiQA2018
1140
- config: default
1141
  split: test
1142
- revision: None
1143
  metrics:
1144
- - type: map_at_1
1145
- value: 1.166
1146
- - type: map_at_10
1147
- value: 2.283
1148
- - type: map_at_100
1149
- value: 2.564
1150
- - type: map_at_1000
1151
- value: 2.6519999999999997
1152
- - type: map_at_3
1153
- value: 1.867
1154
- - type: map_at_5
1155
- value: 2.0500000000000003
1156
- - type: mrr_at_1
1157
- value: 2.932
1158
- - type: mrr_at_10
1159
- value: 4.852
1160
- - type: mrr_at_100
1161
- value: 5.306
1162
- - type: mrr_at_1000
1163
- value: 5.4
1164
- - type: mrr_at_3
1165
- value: 4.141
1166
- - type: mrr_at_5
1167
- value: 4.457
1168
- - type: ndcg_at_1
1169
- value: 2.932
1170
- - type: ndcg_at_10
1171
- value: 3.5709999999999997
1172
- - type: ndcg_at_100
1173
- value: 5.489
1174
- - type: ndcg_at_1000
1175
- value: 8.309999999999999
1176
- - type: ndcg_at_3
1177
- value: 2.773
1178
- - type: ndcg_at_5
1179
- value: 2.979
1180
- - type: precision_at_1
1181
- value: 2.932
1182
- - type: precision_at_10
1183
- value: 1.049
1184
- - type: precision_at_100
1185
- value: 0.306
1186
- - type: precision_at_1000
1187
- value: 0.077
1188
- - type: precision_at_3
1189
- value: 1.8519999999999999
1190
- - type: precision_at_5
1191
- value: 1.389
1192
- - type: recall_at_1
1193
- value: 1.166
1194
- - type: recall_at_10
1195
- value: 5.178
1196
- - type: recall_at_100
1197
- value: 13.056999999999999
1198
- - type: recall_at_1000
1199
- value: 31.708
1200
- - type: recall_at_3
1201
- value: 2.714
1202
- - type: recall_at_5
1203
- value: 3.4909999999999997
1204
  - task:
1205
  type: Classification
1206
  dataset:
1207
- type: mteb/imdb
1208
- name: MTEB ImdbClassification
1209
- config: default
1210
  split: test
1211
- revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
1212
  metrics:
1213
  - type: accuracy
1214
- value: 56.96359999999999
1215
- - type: ap
1216
- value: 54.16760114570921
1217
  - type: f1
1218
- value: 56.193845361069116
1219
  - task:
1220
  type: Classification
1221
  dataset:
1222
- type: mteb/mtop_domain
1223
- name: MTEB MTOPDomainClassification (en)
1224
- config: en
1225
  split: test
1226
- revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1227
  metrics:
1228
  - type: accuracy
1229
- value: 64.39808481532147
1230
  - type: f1
1231
- value: 63.468270818712625
1232
  - task:
1233
  type: Classification
1234
  dataset:
1235
- type: mteb/mtop_domain
1236
- name: MTEB MTOPDomainClassification (de)
1237
- config: de
1238
  split: test
1239
- revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1240
  metrics:
1241
  - type: accuracy
1242
- value: 53.961679346294744
1243
  - type: f1
1244
- value: 51.6707117653683
1245
  - task:
1246
  type: Classification
1247
  dataset:
1248
- type: mteb/mtop_domain
1249
- name: MTEB MTOPDomainClassification (es)
1250
- config: es
1251
  split: test
1252
- revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1253
  metrics:
1254
  - type: accuracy
1255
- value: 57.018012008005336
1256
  - type: f1
1257
- value: 54.23413458037234
1258
  - task:
1259
  type: Classification
1260
  dataset:
1261
- type: mteb/mtop_domain
1262
- name: MTEB MTOPDomainClassification (fr)
1263
- config: fr
1264
  split: test
1265
- revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1266
  metrics:
1267
  - type: accuracy
1268
- value: 48.84434700908236
1269
  - type: f1
1270
- value: 46.48494180527987
1271
  - task:
1272
  type: Classification
1273
  dataset:
1274
- type: mteb/mtop_domain
1275
- name: MTEB MTOPDomainClassification (hi)
1276
- config: hi
1277
  split: test
1278
- revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1279
  metrics:
1280
  - type: accuracy
1281
- value: 39.7669415561133
1282
  - type: f1
1283
- value: 35.50974325529877
1284
  - task:
1285
  type: Classification
1286
  dataset:
1287
- type: mteb/mtop_domain
1288
- name: MTEB MTOPDomainClassification (th)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1289
  config: th
1290
  split: test
1291
- revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1292
  metrics:
1293
  - type: accuracy
1294
- value: 42.589511754068724
1295
  - type: f1
1296
- value: 40.47244422785889
1297
  - task:
1298
  type: Classification
1299
  dataset:
1300
- type: mteb/mtop_intent
1301
- name: MTEB MTOPIntentClassification (en)
1302
- config: en
1303
  split: test
1304
- revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1305
  metrics:
1306
  - type: accuracy
1307
- value: 34.01276789785682
1308
  - type: f1
1309
- value: 21.256775922291286
1310
  - task:
1311
  type: Classification
1312
  dataset:
1313
- type: mteb/mtop_intent
1314
- name: MTEB MTOPIntentClassification (de)
1315
- config: de
1316
  split: test
1317
- revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1318
  metrics:
1319
  - type: accuracy
1320
- value: 33.285432516201745
1321
  - type: f1
1322
- value: 19.841703666811565
1323
  - task:
1324
  type: Classification
1325
  dataset:
1326
- type: mteb/mtop_intent
1327
- name: MTEB MTOPIntentClassification (es)
1328
- config: es
1329
  split: test
1330
- revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1331
  metrics:
1332
  - type: accuracy
1333
- value: 32.121414276184126
1334
  - type: f1
1335
- value: 19.34706868150749
1336
  - task:
1337
  type: Classification
1338
  dataset:
1339
- type: mteb/mtop_intent
1340
- name: MTEB MTOPIntentClassification (fr)
1341
- config: fr
1342
  split: test
1343
- revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1344
  metrics:
1345
  - type: accuracy
1346
- value: 26.088318196053866
1347
  - type: f1
1348
- value: 17.22608011891254
1349
  - task:
1350
  type: Classification
1351
  dataset:
1352
- type: mteb/mtop_intent
1353
- name: MTEB MTOPIntentClassification (hi)
1354
- config: hi
1355
  split: test
1356
- revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1357
  metrics:
1358
  - type: accuracy
1359
- value: 15.320903549659375
1360
  - type: f1
1361
- value: 9.62002916015258
1362
  - task:
1363
  type: Classification
1364
  dataset:
1365
- type: mteb/mtop_intent
1366
- name: MTEB MTOPIntentClassification (th)
1367
- config: th
1368
  split: test
1369
- revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1370
  metrics:
1371
  - type: accuracy
1372
- value: 16.426763110307412
1373
  - type: f1
1374
- value: 11.023799171137183
1375
  - task:
1376
  type: Clustering
1377
  dataset:
@@ -1394,6 +3091,19 @@ model-index:
1394
  metrics:
1395
  - type: v_measure
1396
  value: 16.58582885790446
 
 
 
 
 
 
 
 
 
 
 
 
 
1397
  - task:
1398
  type: Retrieval
1399
  dataset:
@@ -1463,6 +3173,75 @@ model-index:
1463
  value: 1.702
1464
  - type: recall_at_5
1465
  value: 1.9879999999999998
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1466
  - task:
1467
  type: Retrieval
1468
  dataset:
@@ -2620,6 +4399,75 @@ model-index:
2620
  value: 0.231
2621
  - type: recall_at_5
2622
  value: 0.367
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2623
  - task:
2624
  type: Classification
2625
  dataset:
@@ -2768,5 +4616,4 @@ model-index:
2768
  - type: max_ap
2769
  value: 64.59241956109807
2770
  - type: max_f1
2771
- value: 57.83203629255339
2772
- ---
 
64
  value: 14.367489440317666
65
  - type: f1
66
  value: 50.48473578289779
67
+ - task:
68
+ type: Classification
69
+ dataset:
70
+ type: mteb/amazon_polarity
71
+ name: MTEB AmazonPolarityClassification
72
+ config: default
73
+ split: test
74
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
75
+ metrics:
76
+ - type: accuracy
77
+ value: 57.567425000000014
78
+ - type: ap
79
+ value: 54.53026421737829
80
+ - type: f1
81
+ value: 56.60093061259046
82
  - task:
83
  type: Classification
84
  dataset:
 
226
  value: 12.447
227
  - type: recall_at_5
228
  value: 16.145
229
+ - task:
230
+ type: Clustering
231
+ dataset:
232
+ type: mteb/arxiv-clustering-p2p
233
+ name: MTEB ArxivClusteringP2P
234
+ config: default
235
+ split: test
236
+ revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
237
+ metrics:
238
+ - type: v_measure
239
+ value: 25.92658946113241
240
  - task:
241
  type: Clustering
242
  dataset:
 
1145
  value: 6.784
1146
  - type: recall_at_5
1147
  value: 8.17
1148
+ - task:
1149
+ type: Retrieval
1150
+ dataset:
1151
+ type: climate-fever
1152
+ name: MTEB ClimateFEVER
1153
+ config: default
1154
+ split: test
1155
+ revision: None
1156
+ metrics:
1157
+ - type: map_at_1
1158
+ value: 2.09
1159
+ - type: map_at_10
1160
+ value: 3.469
1161
+ - type: map_at_100
1162
+ value: 3.93
1163
+ - type: map_at_1000
1164
+ value: 4.018
1165
+ - type: map_at_3
1166
+ value: 2.8209999999999997
1167
+ - type: map_at_5
1168
+ value: 3.144
1169
+ - type: mrr_at_1
1170
+ value: 4.756
1171
+ - type: mrr_at_10
1172
+ value: 7.853000000000001
1173
+ - type: mrr_at_100
1174
+ value: 8.547
1175
+ - type: mrr_at_1000
1176
+ value: 8.631
1177
+ - type: mrr_at_3
1178
+ value: 6.569
1179
+ - type: mrr_at_5
1180
+ value: 7.249999999999999
1181
+ - type: ndcg_at_1
1182
+ value: 4.756
1183
+ - type: ndcg_at_10
1184
+ value: 5.494000000000001
1185
+ - type: ndcg_at_100
1186
+ value: 8.275
1187
+ - type: ndcg_at_1000
1188
+ value: 10.892
1189
+ - type: ndcg_at_3
1190
+ value: 4.091
1191
+ - type: ndcg_at_5
1192
+ value: 4.588
1193
+ - type: precision_at_1
1194
+ value: 4.756
1195
+ - type: precision_at_10
1196
+ value: 1.8370000000000002
1197
+ - type: precision_at_100
1198
+ value: 0.475
1199
+ - type: precision_at_1000
1200
+ value: 0.094
1201
+ - type: precision_at_3
1202
+ value: 3.018
1203
+ - type: precision_at_5
1204
+ value: 2.528
1205
+ - type: recall_at_1
1206
+ value: 2.09
1207
+ - type: recall_at_10
1208
+ value: 7.127
1209
+ - type: recall_at_100
1210
+ value: 17.483999999999998
1211
+ - type: recall_at_1000
1212
+ value: 33.353
1213
+ - type: recall_at_3
1214
+ value: 3.742
1215
+ - type: recall_at_5
1216
+ value: 5.041
1217
+ - task:
1218
+ type: Retrieval
1219
+ dataset:
1220
+ type: dbpedia-entity
1221
+ name: MTEB DBPedia
1222
+ config: default
1223
+ split: test
1224
+ revision: None
1225
+ metrics:
1226
+ - type: map_at_1
1227
+ value: 0.573
1228
+ - type: map_at_10
1229
+ value: 1.282
1230
+ - type: map_at_100
1231
+ value: 1.625
1232
+ - type: map_at_1000
1233
+ value: 1.71
1234
+ - type: map_at_3
1235
+ value: 1.0
1236
+ - type: map_at_5
1237
+ value: 1.135
1238
+ - type: mrr_at_1
1239
+ value: 7.000000000000001
1240
+ - type: mrr_at_10
1241
+ value: 11.084
1242
+ - type: mrr_at_100
1243
+ value: 11.634
1244
+ - type: mrr_at_1000
1245
+ value: 11.715
1246
+ - type: mrr_at_3
1247
+ value: 9.792
1248
+ - type: mrr_at_5
1249
+ value: 10.404
1250
+ - type: ndcg_at_1
1251
+ value: 4.375
1252
+ - type: ndcg_at_10
1253
+ value: 3.7800000000000002
1254
+ - type: ndcg_at_100
1255
+ value: 4.353
1256
+ - type: ndcg_at_1000
1257
+ value: 6.087
1258
+ - type: ndcg_at_3
1259
+ value: 4.258
1260
+ - type: ndcg_at_5
1261
+ value: 3.988
1262
+ - type: precision_at_1
1263
+ value: 7.000000000000001
1264
+ - type: precision_at_10
1265
+ value: 3.35
1266
+ - type: precision_at_100
1267
+ value: 1.057
1268
+ - type: precision_at_1000
1269
+ value: 0.243
1270
+ - type: precision_at_3
1271
+ value: 5.75
1272
+ - type: precision_at_5
1273
+ value: 4.6
1274
+ - type: recall_at_1
1275
+ value: 0.573
1276
+ - type: recall_at_10
1277
+ value: 2.464
1278
+ - type: recall_at_100
1279
+ value: 5.6770000000000005
1280
+ - type: recall_at_1000
1281
+ value: 12.516
1282
+ - type: recall_at_3
1283
+ value: 1.405
1284
+ - type: recall_at_5
1285
+ value: 1.807
1286
+ - task:
1287
+ type: Classification
1288
+ dataset:
1289
+ type: mteb/emotion
1290
+ name: MTEB EmotionClassification
1291
+ config: default
1292
+ split: test
1293
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
1294
+ metrics:
1295
+ - type: accuracy
1296
+ value: 23.279999999999998
1297
+ - type: f1
1298
+ value: 19.87865985032945
1299
+ - task:
1300
+ type: Retrieval
1301
+ dataset:
1302
+ type: fever
1303
+ name: MTEB FEVER
1304
+ config: default
1305
+ split: test
1306
+ revision: None
1307
+ metrics:
1308
+ - type: map_at_1
1309
+ value: 3.145
1310
+ - type: map_at_10
1311
+ value: 4.721
1312
+ - type: map_at_100
1313
+ value: 5.086
1314
+ - type: map_at_1000
1315
+ value: 5.142
1316
+ - type: map_at_3
1317
+ value: 4.107
1318
+ - type: map_at_5
1319
+ value: 4.45
1320
+ - type: mrr_at_1
1321
+ value: 3.27
1322
+ - type: mrr_at_10
1323
+ value: 4.958
1324
+ - type: mrr_at_100
1325
+ value: 5.35
1326
+ - type: mrr_at_1000
1327
+ value: 5.409
1328
+ - type: mrr_at_3
1329
+ value: 4.303
1330
+ - type: mrr_at_5
1331
+ value: 4.6739999999999995
1332
+ - type: ndcg_at_1
1333
+ value: 3.27
1334
+ - type: ndcg_at_10
1335
+ value: 5.768
1336
+ - type: ndcg_at_100
1337
+ value: 7.854
1338
+ - type: ndcg_at_1000
1339
+ value: 9.729000000000001
1340
+ - type: ndcg_at_3
1341
+ value: 4.476
1342
+ - type: ndcg_at_5
1343
+ value: 5.102
1344
+ - type: precision_at_1
1345
+ value: 3.27
1346
+ - type: precision_at_10
1347
+ value: 0.942
1348
+ - type: precision_at_100
1349
+ value: 0.20600000000000002
1350
+ - type: precision_at_1000
1351
+ value: 0.038
1352
+ - type: precision_at_3
1353
+ value: 1.8849999999999998
1354
+ - type: precision_at_5
1355
+ value: 1.455
1356
+ - type: recall_at_1
1357
+ value: 3.145
1358
+ - type: recall_at_10
1359
+ value: 8.889
1360
+ - type: recall_at_100
1361
+ value: 19.092000000000002
1362
+ - type: recall_at_1000
1363
+ value: 34.35
1364
+ - type: recall_at_3
1365
+ value: 5.353
1366
+ - type: recall_at_5
1367
+ value: 6.836
1368
+ - task:
1369
+ type: Retrieval
1370
+ dataset:
1371
+ type: fiqa
1372
+ name: MTEB FiQA2018
1373
+ config: default
1374
+ split: test
1375
+ revision: None
1376
+ metrics:
1377
+ - type: map_at_1
1378
+ value: 1.166
1379
+ - type: map_at_10
1380
+ value: 2.283
1381
+ - type: map_at_100
1382
+ value: 2.564
1383
+ - type: map_at_1000
1384
+ value: 2.6519999999999997
1385
+ - type: map_at_3
1386
+ value: 1.867
1387
+ - type: map_at_5
1388
+ value: 2.0500000000000003
1389
+ - type: mrr_at_1
1390
+ value: 2.932
1391
+ - type: mrr_at_10
1392
+ value: 4.852
1393
+ - type: mrr_at_100
1394
+ value: 5.306
1395
+ - type: mrr_at_1000
1396
+ value: 5.4
1397
+ - type: mrr_at_3
1398
+ value: 4.141
1399
+ - type: mrr_at_5
1400
+ value: 4.457
1401
+ - type: ndcg_at_1
1402
+ value: 2.932
1403
+ - type: ndcg_at_10
1404
+ value: 3.5709999999999997
1405
+ - type: ndcg_at_100
1406
+ value: 5.489
1407
+ - type: ndcg_at_1000
1408
+ value: 8.309999999999999
1409
+ - type: ndcg_at_3
1410
+ value: 2.773
1411
+ - type: ndcg_at_5
1412
+ value: 2.979
1413
+ - type: precision_at_1
1414
+ value: 2.932
1415
+ - type: precision_at_10
1416
+ value: 1.049
1417
+ - type: precision_at_100
1418
+ value: 0.306
1419
+ - type: precision_at_1000
1420
+ value: 0.077
1421
+ - type: precision_at_3
1422
+ value: 1.8519999999999999
1423
+ - type: precision_at_5
1424
+ value: 1.389
1425
+ - type: recall_at_1
1426
+ value: 1.166
1427
+ - type: recall_at_10
1428
+ value: 5.178
1429
+ - type: recall_at_100
1430
+ value: 13.056999999999999
1431
+ - type: recall_at_1000
1432
+ value: 31.708
1433
+ - type: recall_at_3
1434
+ value: 2.714
1435
+ - type: recall_at_5
1436
+ value: 3.4909999999999997
1437
+ - task:
1438
+ type: Retrieval
1439
+ dataset:
1440
+ type: hotpotqa
1441
+ name: MTEB HotpotQA
1442
+ config: default
1443
+ split: test
1444
+ revision: None
1445
+ metrics:
1446
+ - type: map_at_1
1447
+ value: 6.138
1448
+ - type: map_at_10
1449
+ value: 8.212
1450
+ - type: map_at_100
1451
+ value: 8.548
1452
+ - type: map_at_1000
1453
+ value: 8.604000000000001
1454
+ - type: map_at_3
1455
+ value: 7.555000000000001
1456
+ - type: map_at_5
1457
+ value: 7.881
1458
+ - type: mrr_at_1
1459
+ value: 12.275
1460
+ - type: mrr_at_10
1461
+ value: 15.49
1462
+ - type: mrr_at_100
1463
+ value: 15.978
1464
+ - type: mrr_at_1000
1465
+ value: 16.043
1466
+ - type: mrr_at_3
1467
+ value: 14.488000000000001
1468
+ - type: mrr_at_5
1469
+ value: 14.975
1470
+ - type: ndcg_at_1
1471
+ value: 12.275
1472
+ - type: ndcg_at_10
1473
+ value: 11.078000000000001
1474
+ - type: ndcg_at_100
1475
+ value: 13.081999999999999
1476
+ - type: ndcg_at_1000
1477
+ value: 14.906
1478
+ - type: ndcg_at_3
1479
+ value: 9.574
1480
+ - type: ndcg_at_5
1481
+ value: 10.206999999999999
1482
+ - type: precision_at_1
1483
+ value: 12.275
1484
+ - type: precision_at_10
1485
+ value: 2.488
1486
+ - type: precision_at_100
1487
+ value: 0.41200000000000003
1488
+ - type: precision_at_1000
1489
+ value: 0.066
1490
+ - type: precision_at_3
1491
+ value: 5.991
1492
+ - type: precision_at_5
1493
+ value: 4.0969999999999995
1494
+ - type: recall_at_1
1495
+ value: 6.138
1496
+ - type: recall_at_10
1497
+ value: 12.438
1498
+ - type: recall_at_100
1499
+ value: 20.601
1500
+ - type: recall_at_1000
1501
+ value: 32.984
1502
+ - type: recall_at_3
1503
+ value: 8.987
1504
+ - type: recall_at_5
1505
+ value: 10.242999999999999
1506
+ - task:
1507
+ type: Classification
1508
+ dataset:
1509
+ type: mteb/imdb
1510
+ name: MTEB ImdbClassification
1511
+ config: default
1512
+ split: test
1513
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
1514
+ metrics:
1515
+ - type: accuracy
1516
+ value: 56.96359999999999
1517
+ - type: ap
1518
+ value: 54.16760114570921
1519
+ - type: f1
1520
+ value: 56.193845361069116
1521
+ - task:
1522
+ type: Retrieval
1523
+ dataset:
1524
+ type: msmarco
1525
+ name: MTEB MSMARCO
1526
+ config: default
1527
+ split: dev
1528
+ revision: None
1529
+ metrics:
1530
+ - type: map_at_1
1531
+ value: 1.34
1532
+ - type: map_at_10
1533
+ value: 2.2190000000000003
1534
+ - type: map_at_100
1535
+ value: 2.427
1536
+ - type: map_at_1000
1537
+ value: 2.461
1538
+ - type: map_at_3
1539
+ value: 1.8610000000000002
1540
+ - type: map_at_5
1541
+ value: 2.0340000000000003
1542
+ - type: mrr_at_1
1543
+ value: 1.375
1544
+ - type: mrr_at_10
1545
+ value: 2.284
1546
+ - type: mrr_at_100
1547
+ value: 2.5
1548
+ - type: mrr_at_1000
1549
+ value: 2.535
1550
+ - type: mrr_at_3
1551
+ value: 1.913
1552
+ - type: mrr_at_5
1553
+ value: 2.094
1554
+ - type: ndcg_at_1
1555
+ value: 1.375
1556
+ - type: ndcg_at_10
1557
+ value: 2.838
1558
+ - type: ndcg_at_100
1559
+ value: 4.043
1560
+ - type: ndcg_at_1000
1561
+ value: 5.205
1562
+ - type: ndcg_at_3
1563
+ value: 2.0629999999999997
1564
+ - type: ndcg_at_5
1565
+ value: 2.387
1566
+ - type: precision_at_1
1567
+ value: 1.375
1568
+ - type: precision_at_10
1569
+ value: 0.496
1570
+ - type: precision_at_100
1571
+ value: 0.11399999999999999
1572
+ - type: precision_at_1000
1573
+ value: 0.022000000000000002
1574
+ - type: precision_at_3
1575
+ value: 0.898
1576
+ - type: precision_at_5
1577
+ value: 0.705
1578
+ - type: recall_at_1
1579
+ value: 1.34
1580
+ - type: recall_at_10
1581
+ value: 4.787
1582
+ - type: recall_at_100
1583
+ value: 10.759
1584
+ - type: recall_at_1000
1585
+ value: 20.362
1586
+ - type: recall_at_3
1587
+ value: 2.603
1588
+ - type: recall_at_5
1589
+ value: 3.398
1590
+ - task:
1591
+ type: Classification
1592
+ dataset:
1593
+ type: mteb/mtop_domain
1594
+ name: MTEB MTOPDomainClassification (en)
1595
+ config: en
1596
+ split: test
1597
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1598
+ metrics:
1599
+ - type: accuracy
1600
+ value: 64.39808481532147
1601
+ - type: f1
1602
+ value: 63.468270818712625
1603
+ - task:
1604
+ type: Classification
1605
+ dataset:
1606
+ type: mteb/mtop_domain
1607
+ name: MTEB MTOPDomainClassification (de)
1608
+ config: de
1609
+ split: test
1610
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1611
+ metrics:
1612
+ - type: accuracy
1613
+ value: 53.961679346294744
1614
+ - type: f1
1615
+ value: 51.6707117653683
1616
+ - task:
1617
+ type: Classification
1618
+ dataset:
1619
+ type: mteb/mtop_domain
1620
+ name: MTEB MTOPDomainClassification (es)
1621
+ config: es
1622
+ split: test
1623
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1624
+ metrics:
1625
+ - type: accuracy
1626
+ value: 57.018012008005336
1627
+ - type: f1
1628
+ value: 54.23413458037234
1629
+ - task:
1630
+ type: Classification
1631
+ dataset:
1632
+ type: mteb/mtop_domain
1633
+ name: MTEB MTOPDomainClassification (fr)
1634
+ config: fr
1635
+ split: test
1636
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1637
+ metrics:
1638
+ - type: accuracy
1639
+ value: 48.84434700908236
1640
+ - type: f1
1641
+ value: 46.48494180527987
1642
+ - task:
1643
+ type: Classification
1644
+ dataset:
1645
+ type: mteb/mtop_domain
1646
+ name: MTEB MTOPDomainClassification (hi)
1647
+ config: hi
1648
+ split: test
1649
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1650
+ metrics:
1651
+ - type: accuracy
1652
+ value: 39.7669415561133
1653
+ - type: f1
1654
+ value: 35.50974325529877
1655
+ - task:
1656
+ type: Classification
1657
+ dataset:
1658
+ type: mteb/mtop_domain
1659
+ name: MTEB MTOPDomainClassification (th)
1660
+ config: th
1661
+ split: test
1662
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1663
+ metrics:
1664
+ - type: accuracy
1665
+ value: 42.589511754068724
1666
+ - type: f1
1667
+ value: 40.47244422785889
1668
+ - task:
1669
+ type: Classification
1670
+ dataset:
1671
+ type: mteb/mtop_intent
1672
+ name: MTEB MTOPIntentClassification (en)
1673
+ config: en
1674
+ split: test
1675
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1676
+ metrics:
1677
+ - type: accuracy
1678
+ value: 34.01276789785682
1679
+ - type: f1
1680
+ value: 21.256775922291286
1681
+ - task:
1682
+ type: Classification
1683
+ dataset:
1684
+ type: mteb/mtop_intent
1685
+ name: MTEB MTOPIntentClassification (de)
1686
+ config: de
1687
+ split: test
1688
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1689
+ metrics:
1690
+ - type: accuracy
1691
+ value: 33.285432516201745
1692
+ - type: f1
1693
+ value: 19.841703666811565
1694
+ - task:
1695
+ type: Classification
1696
+ dataset:
1697
+ type: mteb/mtop_intent
1698
+ name: MTEB MTOPIntentClassification (es)
1699
+ config: es
1700
+ split: test
1701
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1702
+ metrics:
1703
+ - type: accuracy
1704
+ value: 32.121414276184126
1705
+ - type: f1
1706
+ value: 19.34706868150749
1707
+ - task:
1708
+ type: Classification
1709
+ dataset:
1710
+ type: mteb/mtop_intent
1711
+ name: MTEB MTOPIntentClassification (fr)
1712
+ config: fr
1713
+ split: test
1714
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1715
+ metrics:
1716
+ - type: accuracy
1717
+ value: 26.088318196053866
1718
+ - type: f1
1719
+ value: 17.22608011891254
1720
+ - task:
1721
+ type: Classification
1722
+ dataset:
1723
+ type: mteb/mtop_intent
1724
+ name: MTEB MTOPIntentClassification (hi)
1725
+ config: hi
1726
+ split: test
1727
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1728
+ metrics:
1729
+ - type: accuracy
1730
+ value: 15.320903549659375
1731
+ - type: f1
1732
+ value: 9.62002916015258
1733
+ - task:
1734
+ type: Classification
1735
+ dataset:
1736
+ type: mteb/mtop_intent
1737
+ name: MTEB MTOPIntentClassification (th)
1738
+ config: th
1739
+ split: test
1740
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1741
+ metrics:
1742
+ - type: accuracy
1743
+ value: 16.426763110307412
1744
+ - type: f1
1745
+ value: 11.023799171137183
1746
+ - task:
1747
+ type: Classification
1748
+ dataset:
1749
+ type: mteb/amazon_massive_intent
1750
+ name: MTEB MassiveIntentClassification (af)
1751
+ config: af
1752
+ split: test
1753
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1754
+ metrics:
1755
+ - type: accuracy
1756
+ value: 27.347007397444518
1757
+ - type: f1
1758
+ value: 25.503551916252842
1759
+ - task:
1760
+ type: Classification
1761
+ dataset:
1762
+ type: mteb/amazon_massive_intent
1763
+ name: MTEB MassiveIntentClassification (am)
1764
+ config: am
1765
+ split: test
1766
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1767
+ metrics:
1768
+ - type: accuracy
1769
+ value: 10.655682582380631
1770
+ - type: f1
1771
+ value: 9.141696317946996
1772
+ - task:
1773
+ type: Classification
1774
+ dataset:
1775
+ type: mteb/amazon_massive_intent
1776
+ name: MTEB MassiveIntentClassification (ar)
1777
+ config: ar
1778
+ split: test
1779
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1780
+ metrics:
1781
+ - type: accuracy
1782
+ value: 17.347007397444518
1783
+ - type: f1
1784
+ value: 15.345346511499534
1785
+ - task:
1786
+ type: Classification
1787
+ dataset:
1788
+ type: mteb/amazon_massive_intent
1789
+ name: MTEB MassiveIntentClassification (az)
1790
+ config: az
1791
+ split: test
1792
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1793
+ metrics:
1794
+ - type: accuracy
1795
+ value: 20.39004707464694
1796
+ - type: f1
1797
+ value: 21.129515472610237
1798
+ - task:
1799
+ type: Classification
1800
+ dataset:
1801
+ type: mteb/amazon_massive_intent
1802
+ name: MTEB MassiveIntentClassification (bn)
1803
+ config: bn
1804
+ split: test
1805
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1806
+ metrics:
1807
+ - type: accuracy
1808
+ value: 14.082044384667114
1809
+ - type: f1
1810
+ value: 12.169922201279885
1811
+ - task:
1812
+ type: Classification
1813
+ dataset:
1814
+ type: mteb/amazon_massive_intent
1815
+ name: MTEB MassiveIntentClassification (cy)
1816
+ config: cy
1817
+ split: test
1818
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1819
+ metrics:
1820
+ - type: accuracy
1821
+ value: 27.108271687962336
1822
+ - type: f1
1823
+ value: 25.449222444030063
1824
+ - task:
1825
+ type: Classification
1826
+ dataset:
1827
+ type: mteb/amazon_massive_intent
1828
+ name: MTEB MassiveIntentClassification (da)
1829
+ config: da
1830
+ split: test
1831
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1832
+ metrics:
1833
+ - type: accuracy
1834
+ value: 27.780766644250164
1835
+ - type: f1
1836
+ value: 26.96237025531764
1837
+ - task:
1838
+ type: Classification
1839
+ dataset:
1840
+ type: mteb/amazon_massive_intent
1841
+ name: MTEB MassiveIntentClassification (de)
1842
+ config: de
1843
+ split: test
1844
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1845
+ metrics:
1846
+ - type: accuracy
1847
+ value: 21.768661735036986
1848
+ - type: f1
1849
+ value: 22.377462868662263
1850
+ - task:
1851
+ type: Classification
1852
+ dataset:
1853
+ type: mteb/amazon_massive_intent
1854
+ name: MTEB MassiveIntentClassification (el)
1855
+ config: el
1856
+ split: test
1857
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1858
+ metrics:
1859
+ - type: accuracy
1860
+ value: 21.967047747141898
1861
+ - type: f1
1862
+ value: 22.427583602797057
1863
+ - task:
1864
+ type: Classification
1865
+ dataset:
1866
+ type: mteb/amazon_massive_intent
1867
+ name: MTEB MassiveIntentClassification (en)
1868
+ config: en
1869
+ split: test
1870
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1871
+ metrics:
1872
+ - type: accuracy
1873
+ value: 33.221250840618694
1874
+ - type: f1
1875
+ value: 32.627621011904495
1876
+ - task:
1877
+ type: Classification
1878
+ dataset:
1879
+ type: mteb/amazon_massive_intent
1880
+ name: MTEB MassiveIntentClassification (es)
1881
+ config: es
1882
+ split: test
1883
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1884
+ metrics:
1885
+ - type: accuracy
1886
+ value: 27.047747141896426
1887
+ - type: f1
1888
+ value: 25.244455827652786
1889
+ - task:
1890
+ type: Classification
1891
+ dataset:
1892
+ type: mteb/amazon_massive_intent
1893
+ name: MTEB MassiveIntentClassification (fa)
1894
+ config: fa
1895
+ split: test
1896
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1897
+ metrics:
1898
+ - type: accuracy
1899
+ value: 18.850033624747812
1900
+ - type: f1
1901
+ value: 16.532690247057452
1902
+ - task:
1903
+ type: Classification
1904
+ dataset:
1905
+ type: mteb/amazon_massive_intent
1906
+ name: MTEB MassiveIntentClassification (fi)
1907
+ config: fi
1908
+ split: test
1909
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1910
+ metrics:
1911
+ - type: accuracy
1912
+ value: 25.934767989240083
1913
+ - type: f1
1914
+ value: 24.126974912341858
1915
+ - task:
1916
+ type: Classification
1917
+ dataset:
1918
+ type: mteb/amazon_massive_intent
1919
+ name: MTEB MassiveIntentClassification (fr)
1920
+ config: fr
1921
+ split: test
1922
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1923
+ metrics:
1924
+ - type: accuracy
1925
+ value: 25.59179556153329
1926
+ - type: f1
1927
+ value: 23.97686173045838
1928
+ - task:
1929
+ type: Classification
1930
+ dataset:
1931
+ type: mteb/amazon_massive_intent
1932
+ name: MTEB MassiveIntentClassification (he)
1933
+ config: he
1934
+ split: test
1935
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1936
+ metrics:
1937
+ - type: accuracy
1938
+ value: 17.683254875588432
1939
+ - type: f1
1940
+ value: 15.217082232778534
1941
+ - task:
1942
+ type: Classification
1943
+ dataset:
1944
+ type: mteb/amazon_massive_intent
1945
+ name: MTEB MassiveIntentClassification (hi)
1946
+ config: hi
1947
+ split: test
1948
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1949
+ metrics:
1950
+ - type: accuracy
1951
+ value: 14.277067921990588
1952
+ - type: f1
1953
+ value: 13.06156794974721
1954
+ - task:
1955
+ type: Classification
1956
+ dataset:
1957
+ type: mteb/amazon_massive_intent
1958
+ name: MTEB MassiveIntentClassification (hu)
1959
+ config: hu
1960
+ split: test
1961
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1962
+ metrics:
1963
+ - type: accuracy
1964
+ value: 25.817081371889717
1965
+ - type: f1
1966
+ value: 24.79443526877249
1967
+ - task:
1968
+ type: Classification
1969
+ dataset:
1970
+ type: mteb/amazon_massive_intent
1971
+ name: MTEB MassiveIntentClassification (hy)
1972
+ config: hy
1973
+ split: test
1974
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1975
+ metrics:
1976
+ - type: accuracy
1977
+ value: 12.326832548755885
1978
+ - type: f1
1979
+ value: 10.850963544530288
1980
+ - task:
1981
+ type: Classification
1982
+ dataset:
1983
+ type: mteb/amazon_massive_intent
1984
+ name: MTEB MassiveIntentClassification (id)
1985
+ config: id
1986
+ split: test
1987
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1988
+ metrics:
1989
+ - type: accuracy
1990
+ value: 28.244788164088764
1991
+ - type: f1
1992
+ value: 27.442212153664336
1993
+ - task:
1994
+ type: Classification
1995
+ dataset:
1996
+ type: mteb/amazon_massive_intent
1997
+ name: MTEB MassiveIntentClassification (is)
1998
+ config: is
1999
+ split: test
2000
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2001
+ metrics:
2002
+ - type: accuracy
2003
+ value: 26.0390047074647
2004
+ - type: f1
2005
+ value: 24.29180485465988
2006
+ - task:
2007
+ type: Classification
2008
+ dataset:
2009
+ type: mteb/amazon_massive_intent
2010
+ name: MTEB MassiveIntentClassification (it)
2011
+ config: it
2012
+ split: test
2013
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2014
+ metrics:
2015
+ - type: accuracy
2016
+ value: 26.099529253530594
2017
+ - type: f1
2018
+ value: 26.47963496597501
2019
+ - task:
2020
+ type: Classification
2021
+ dataset:
2022
+ type: mteb/amazon_massive_intent
2023
+ name: MTEB MassiveIntentClassification (ja)
2024
+ config: ja
2025
+ split: test
2026
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2027
+ metrics:
2028
+ - type: accuracy
2029
+ value: 25.383322125084057
2030
+ - type: f1
2031
+ value: 27.24527274982159
2032
+ - task:
2033
+ type: Classification
2034
+ dataset:
2035
+ type: mteb/amazon_massive_intent
2036
+ name: MTEB MassiveIntentClassification (jv)
2037
+ config: jv
2038
+ split: test
2039
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2040
+ metrics:
2041
+ - type: accuracy
2042
+ value: 26.970410221923334
2043
+ - type: f1
2044
+ value: 24.710215925904627
2045
+ - task:
2046
+ type: Classification
2047
+ dataset:
2048
+ type: mteb/amazon_massive_intent
2049
+ name: MTEB MassiveIntentClassification (ka)
2050
+ config: ka
2051
+ split: test
2052
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2053
+ metrics:
2054
+ - type: accuracy
2055
+ value: 14.159381304640215
2056
+ - type: f1
2057
+ value: 12.262797179154113
2058
+ - task:
2059
+ type: Classification
2060
+ dataset:
2061
+ type: mteb/amazon_massive_intent
2062
+ name: MTEB MassiveIntentClassification (km)
2063
+ config: km
2064
+ split: test
2065
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2066
+ metrics:
2067
+ - type: accuracy
2068
+ value: 12.078009414929387
2069
+ - type: f1
2070
+ value: 10.85388698579932
2071
+ - task:
2072
+ type: Classification
2073
+ dataset:
2074
+ type: mteb/amazon_massive_intent
2075
+ name: MTEB MassiveIntentClassification (kn)
2076
+ config: kn
2077
+ split: test
2078
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2079
+ metrics:
2080
+ - type: accuracy
2081
+ value: 10.144586415601884
2082
+ - type: f1
2083
+ value: 8.629316498328535
2084
+ - task:
2085
+ type: Classification
2086
+ dataset:
2087
+ type: mteb/amazon_massive_intent
2088
+ name: MTEB MassiveIntentClassification (ko)
2089
+ config: ko
2090
+ split: test
2091
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2092
+ metrics:
2093
+ - type: accuracy
2094
+ value: 23.799596503026226
2095
+ - type: f1
2096
+ value: 21.4839774342838
2097
+ - task:
2098
+ type: Classification
2099
+ dataset:
2100
+ type: mteb/amazon_massive_intent
2101
+ name: MTEB MassiveIntentClassification (lv)
2102
+ config: lv
2103
+ split: test
2104
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2105
+ metrics:
2106
+ - type: accuracy
2107
+ value: 27.047747141896433
2108
+ - type: f1
2109
+ value: 25.86288514660441
2110
+ - task:
2111
+ type: Classification
2112
+ dataset:
2113
+ type: mteb/amazon_massive_intent
2114
+ name: MTEB MassiveIntentClassification (ml)
2115
+ config: ml
2116
+ split: test
2117
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2118
+ metrics:
2119
+ - type: accuracy
2120
+ value: 10.82380632145259
2121
+ - type: f1
2122
+ value: 9.568319030257811
2123
+ - task:
2124
+ type: Classification
2125
+ dataset:
2126
+ type: mteb/amazon_massive_intent
2127
+ name: MTEB MassiveIntentClassification (mn)
2128
+ config: mn
2129
+ split: test
2130
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2131
+ metrics:
2132
+ - type: accuracy
2133
+ value: 14.435104236718225
2134
+ - type: f1
2135
+ value: 14.861584951252121
2136
+ - task:
2137
+ type: Classification
2138
+ dataset:
2139
+ type: mteb/amazon_massive_intent
2140
+ name: MTEB MassiveIntentClassification (ms)
2141
+ config: ms
2142
+ split: test
2143
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2144
+ metrics:
2145
+ - type: accuracy
2146
+ value: 26.83254875588433
2147
+ - type: f1
2148
+ value: 26.084749439191967
2149
+ - task:
2150
+ type: Classification
2151
+ dataset:
2152
+ type: mteb/amazon_massive_intent
2153
+ name: MTEB MassiveIntentClassification (my)
2154
+ config: my
2155
+ split: test
2156
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2157
+ metrics:
2158
+ - type: accuracy
2159
+ value: 13.792871553463351
2160
+ - type: f1
2161
+ value: 11.496310101617802
2162
+ - task:
2163
+ type: Classification
2164
+ dataset:
2165
+ type: mteb/amazon_massive_intent
2166
+ name: MTEB MassiveIntentClassification (nb)
2167
+ config: nb
2168
+ split: test
2169
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2170
+ metrics:
2171
+ - type: accuracy
2172
+ value: 25.292535305985208
2173
+ - type: f1
2174
+ value: 24.098118477282508
2175
+ - task:
2176
+ type: Classification
2177
+ dataset:
2178
+ type: mteb/amazon_massive_intent
2179
+ name: MTEB MassiveIntentClassification (nl)
2180
+ config: nl
2181
+ split: test
2182
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2183
+ metrics:
2184
+ - type: accuracy
2185
+ value: 26.24075319435104
2186
+ - type: f1
2187
+ value: 25.259998680849815
2188
+ - task:
2189
+ type: Classification
2190
+ dataset:
2191
+ type: mteb/amazon_massive_intent
2192
+ name: MTEB MassiveIntentClassification (pl)
2193
+ config: pl
2194
+ split: test
2195
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2196
+ metrics:
2197
+ - type: accuracy
2198
+ value: 24.909213180901148
2199
+ - type: f1
2200
+ value: 23.684726718537476
2201
+ - task:
2202
+ type: Classification
2203
+ dataset:
2204
+ type: mteb/amazon_massive_intent
2205
+ name: MTEB MassiveIntentClassification (pt)
2206
+ config: pt
2207
+ split: test
2208
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2209
+ metrics:
2210
+ - type: accuracy
2211
+ value: 27.310020174848688
2212
+ - type: f1
2213
+ value: 26.28728556240871
2214
+ - task:
2215
+ type: Classification
2216
+ dataset:
2217
+ type: mteb/amazon_massive_intent
2218
+ name: MTEB MassiveIntentClassification (ro)
2219
+ config: ro
2220
+ split: test
2221
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2222
+ metrics:
2223
+ - type: accuracy
2224
+ value: 21.930060524546064
2225
+ - type: f1
2226
+ value: 21.993435727293242
2227
+ - task:
2228
+ type: Classification
2229
+ dataset:
2230
+ type: mteb/amazon_massive_intent
2231
+ name: MTEB MassiveIntentClassification (ru)
2232
+ config: ru
2233
+ split: test
2234
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2235
+ metrics:
2236
+ - type: accuracy
2237
+ value: 25.773369199731
2238
+ - type: f1
2239
+ value: 23.857739120942632
2240
+ - task:
2241
+ type: Classification
2242
+ dataset:
2243
+ type: mteb/amazon_massive_intent
2244
+ name: MTEB MassiveIntentClassification (sl)
2245
+ config: sl
2246
+ split: test
2247
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2248
+ metrics:
2249
+ - type: accuracy
2250
+ value: 29.256893073301953
2251
+ - type: f1
2252
+ value: 26.6824726822139
2253
+ - task:
2254
+ type: Classification
2255
+ dataset:
2256
+ type: mteb/amazon_massive_intent
2257
+ name: MTEB MassiveIntentClassification (sq)
2258
+ config: sq
2259
+ split: test
2260
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2261
+ metrics:
2262
+ - type: accuracy
2263
+ value: 17.673167451244115
2264
+ - type: f1
2265
+ value: 18.74169530134767
2266
+ - task:
2267
+ type: Classification
2268
+ dataset:
2269
+ type: mteb/amazon_massive_intent
2270
+ name: MTEB MassiveIntentClassification (sv)
2271
+ config: sv
2272
+ split: test
2273
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2274
+ metrics:
2275
+ - type: accuracy
2276
+ value: 26.190316072629454
2277
+ - type: f1
2278
+ value: 26.000111409348474
2279
+ - task:
2280
+ type: Classification
2281
+ dataset:
2282
+ type: mteb/amazon_massive_intent
2283
+ name: MTEB MassiveIntentClassification (sw)
2284
+ config: sw
2285
+ split: test
2286
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2287
+ metrics:
2288
+ - type: accuracy
2289
+ value: 28.903833221250842
2290
+ - type: f1
2291
+ value: 27.166958835143152
2292
+ - task:
2293
+ type: Classification
2294
+ dataset:
2295
+ type: mteb/amazon_massive_intent
2296
+ name: MTEB MassiveIntentClassification (ta)
2297
+ config: ta
2298
+ split: test
2299
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2300
+ metrics:
2301
+ - type: accuracy
2302
+ value: 13.517148621385342
2303
+ - type: f1
2304
+ value: 10.124568781522633
2305
+ - task:
2306
+ type: Classification
2307
+ dataset:
2308
+ type: mteb/amazon_massive_intent
2309
+ name: MTEB MassiveIntentClassification (te)
2310
+ config: te
2311
+ split: test
2312
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2313
+ metrics:
2314
+ - type: accuracy
2315
+ value: 12.414256893073302
2316
+ - type: f1
2317
+ value: 10.72905194223371
2318
+ - task:
2319
+ type: Classification
2320
+ dataset:
2321
+ type: mteb/amazon_massive_intent
2322
+ name: MTEB MassiveIntentClassification (th)
2323
+ config: th
2324
+ split: test
2325
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2326
+ metrics:
2327
+ - type: accuracy
2328
+ value: 17.330195023537325
2329
+ - type: f1
2330
+ value: 17.262535319973026
2331
+ - task:
2332
+ type: Classification
2333
+ dataset:
2334
+ type: mteb/amazon_massive_intent
2335
+ name: MTEB MassiveIntentClassification (tl)
2336
+ config: tl
2337
+ split: test
2338
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2339
+ metrics:
2340
+ - type: accuracy
2341
+ value: 22.310020174848688
2342
+ - type: f1
2343
+ value: 22.770175966437712
2344
+ - task:
2345
+ type: Classification
2346
+ dataset:
2347
+ type: mteb/amazon_massive_intent
2348
+ name: MTEB MassiveIntentClassification (tr)
2349
+ config: tr
2350
+ split: test
2351
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2352
+ metrics:
2353
+ - type: accuracy
2354
+ value: 23.066577000672496
2355
+ - type: f1
2356
+ value: 22.50576183714979
2357
+ - task:
2358
+ type: Classification
2359
+ dataset:
2360
+ type: mteb/amazon_massive_intent
2361
+ name: MTEB MassiveIntentClassification (ur)
2362
+ config: ur
2363
+ split: test
2364
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2365
+ metrics:
2366
+ - type: accuracy
2367
+ value: 12.501681237390722
2368
+ - type: f1
2369
+ value: 11.803223778323549
2370
+ - task:
2371
+ type: Classification
2372
+ dataset:
2373
+ type: mteb/amazon_massive_intent
2374
+ name: MTEB MassiveIntentClassification (vi)
2375
+ config: vi
2376
+ split: test
2377
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2378
+ metrics:
2379
+ - type: accuracy
2380
+ value: 22.76059179556153
2381
+ - type: f1
2382
+ value: 21.55631039694901
2383
+ - task:
2384
+ type: Classification
2385
+ dataset:
2386
+ type: mteb/amazon_massive_intent
2387
+ name: MTEB MassiveIntentClassification (zh-CN)
2388
+ config: zh-CN
2389
+ split: test
2390
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2391
+ metrics:
2392
+ - type: accuracy
2393
+ value: 34.92602555480834
2394
+ - type: f1
2395
+ value: 33.727974171773425
2396
+ - task:
2397
+ type: Classification
2398
+ dataset:
2399
+ type: mteb/amazon_massive_intent
2400
+ name: MTEB MassiveIntentClassification (zh-TW)
2401
+ config: zh-TW
2402
+ split: test
2403
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
2404
+ metrics:
2405
+ - type: accuracy
2406
+ value: 28.365837256220576
2407
+ - type: f1
2408
+ value: 28.510022067424707
2409
+ - task:
2410
+ type: Classification
2411
+ dataset:
2412
+ type: mteb/amazon_massive_scenario
2413
+ name: MTEB MassiveScenarioClassification (af)
2414
+ config: af
2415
+ split: test
2416
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2417
+ metrics:
2418
+ - type: accuracy
2419
+ value: 36.02891728312038
2420
+ - type: f1
2421
+ value: 31.370502912090657
2422
+ - task:
2423
+ type: Classification
2424
+ dataset:
2425
+ type: mteb/amazon_massive_scenario
2426
+ name: MTEB MassiveScenarioClassification (am)
2427
+ config: am
2428
+ split: test
2429
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2430
+ metrics:
2431
+ - type: accuracy
2432
+ value: 19.330867518493612
2433
+ - type: f1
2434
+ value: 15.90055477686407
2435
+ - task:
2436
+ type: Classification
2437
+ dataset:
2438
+ type: mteb/amazon_massive_scenario
2439
+ name: MTEB MassiveScenarioClassification (ar)
2440
+ config: ar
2441
+ split: test
2442
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2443
+ metrics:
2444
+ - type: accuracy
2445
+ value: 23.880295897780766
2446
+ - type: f1
2447
+ value: 21.695586897546278
2448
+ - task:
2449
+ type: Classification
2450
+ dataset:
2451
+ type: mteb/amazon_massive_scenario
2452
+ name: MTEB MassiveScenarioClassification (az)
2453
+ config: az
2454
+ split: test
2455
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2456
+ metrics:
2457
+ - type: accuracy
2458
+ value: 28.853396099529256
2459
+ - type: f1
2460
+ value: 26.709933198206503
2461
+ - task:
2462
+ type: Classification
2463
+ dataset:
2464
+ type: mteb/amazon_massive_scenario
2465
+ name: MTEB MassiveScenarioClassification (bn)
2466
+ config: bn
2467
+ split: test
2468
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2469
+ metrics:
2470
+ - type: accuracy
2471
+ value: 18.382649630127773
2472
+ - type: f1
2473
+ value: 16.772353498005437
2474
+ - task:
2475
+ type: Classification
2476
+ dataset:
2477
+ type: mteb/amazon_massive_scenario
2478
+ name: MTEB MassiveScenarioClassification (cy)
2479
+ config: cy
2480
+ split: test
2481
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2482
+ metrics:
2483
+ - type: accuracy
2484
+ value: 35.40685944855414
2485
+ - type: f1
2486
+ value: 31.488014251203257
2487
+ - task:
2488
+ type: Classification
2489
+ dataset:
2490
+ type: mteb/amazon_massive_scenario
2491
+ name: MTEB MassiveScenarioClassification (da)
2492
+ config: da
2493
+ split: test
2494
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2495
+ metrics:
2496
+ - type: accuracy
2497
+ value: 34.4754539340955
2498
+ - type: f1
2499
+ value: 31.226537880607296
2500
+ - task:
2501
+ type: Classification
2502
+ dataset:
2503
+ type: mteb/amazon_massive_scenario
2504
+ name: MTEB MassiveScenarioClassification (de)
2505
+ config: de
2506
+ split: test
2507
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2508
+ metrics:
2509
+ - type: accuracy
2510
+ value: 30.0
2511
+ - type: f1
2512
+ value: 28.08153376359271
2513
+ - task:
2514
+ type: Classification
2515
+ dataset:
2516
+ type: mteb/amazon_massive_scenario
2517
+ name: MTEB MassiveScenarioClassification (el)
2518
+ config: el
2519
+ split: test
2520
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2521
+ metrics:
2522
+ - type: accuracy
2523
+ value: 26.879623402824482
2524
+ - type: f1
2525
+ value: 24.773267596830472
2526
+ - task:
2527
+ type: Classification
2528
+ dataset:
2529
+ type: mteb/amazon_massive_scenario
2530
+ name: MTEB MassiveScenarioClassification (en)
2531
+ config: en
2532
+ split: test
2533
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2534
+ metrics:
2535
+ - type: accuracy
2536
+ value: 43.77605917955615
2537
+ - type: f1
2538
+ value: 41.60519309254586
2539
+ - task:
2540
+ type: Classification
2541
+ dataset:
2542
+ type: mteb/amazon_massive_scenario
2543
+ name: MTEB MassiveScenarioClassification (es)
2544
+ config: es
2545
+ split: test
2546
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2547
+ metrics:
2548
+ - type: accuracy
2549
+ value: 32.807666442501684
2550
+ - type: f1
2551
+ value: 30.250350082554473
2552
+ - task:
2553
+ type: Classification
2554
+ dataset:
2555
+ type: mteb/amazon_massive_scenario
2556
+ name: MTEB MassiveScenarioClassification (fa)
2557
+ config: fa
2558
+ split: test
2559
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2560
+ metrics:
2561
+ - type: accuracy
2562
+ value: 24.515803631472764
2563
+ - type: f1
2564
+ value: 21.726717236406927
2565
+ - task:
2566
+ type: Classification
2567
+ dataset:
2568
+ type: mteb/amazon_massive_scenario
2569
+ name: MTEB MassiveScenarioClassification (fi)
2570
+ config: fi
2571
+ split: test
2572
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2573
+ metrics:
2574
+ - type: accuracy
2575
+ value: 33.315400134498994
2576
+ - type: f1
2577
+ value: 29.353644700383935
2578
+ - task:
2579
+ type: Classification
2580
+ dataset:
2581
+ type: mteb/amazon_massive_scenario
2582
+ name: MTEB MassiveScenarioClassification (fr)
2583
+ config: fr
2584
+ split: test
2585
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2586
+ metrics:
2587
+ - type: accuracy
2588
+ value: 32.42434431741762
2589
+ - type: f1
2590
+ value: 29.87668403274445
2591
+ - task:
2592
+ type: Classification
2593
+ dataset:
2594
+ type: mteb/amazon_massive_scenario
2595
+ name: MTEB MassiveScenarioClassification (he)
2596
+ config: he
2597
+ split: test
2598
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2599
+ metrics:
2600
+ - type: accuracy
2601
+ value: 24.40147948890384
2602
+ - type: f1
2603
+ value: 21.429656257015555
2604
+ - task:
2605
+ type: Classification
2606
+ dataset:
2607
+ type: mteb/amazon_massive_scenario
2608
+ name: MTEB MassiveScenarioClassification (hi)
2609
+ config: hi
2610
+ split: test
2611
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2612
+ metrics:
2613
+ - type: accuracy
2614
+ value: 21.435776731674512
2615
+ - type: f1
2616
+ value: 18.344740043506686
2617
+ - task:
2618
+ type: Classification
2619
+ dataset:
2620
+ type: mteb/amazon_massive_scenario
2621
+ name: MTEB MassiveScenarioClassification (hu)
2622
+ config: hu
2623
+ split: test
2624
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2625
+ metrics:
2626
+ - type: accuracy
2627
+ value: 31.533288500336248
2628
+ - type: f1
2629
+ value: 29.185785763469298
2630
+ - task:
2631
+ type: Classification
2632
+ dataset:
2633
+ type: mteb/amazon_massive_scenario
2634
+ name: MTEB MassiveScenarioClassification (hy)
2635
+ config: hy
2636
+ split: test
2637
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2638
+ metrics:
2639
+ - type: accuracy
2640
+ value: 19.983187626092803
2641
+ - type: f1
2642
+ value: 17.088318438817677
2643
+ - task:
2644
+ type: Classification
2645
+ dataset:
2646
+ type: mteb/amazon_massive_scenario
2647
+ name: MTEB MassiveScenarioClassification (id)
2648
+ config: id
2649
+ split: test
2650
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2651
+ metrics:
2652
+ - type: accuracy
2653
+ value: 32.73369199731002
2654
+ - type: f1
2655
+ value: 31.37876294000571
2656
+ - task:
2657
+ type: Classification
2658
+ dataset:
2659
+ type: mteb/amazon_massive_scenario
2660
+ name: MTEB MassiveScenarioClassification (is)
2661
+ config: is
2662
+ split: test
2663
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2664
+ metrics:
2665
+ - type: accuracy
2666
+ value: 31.503026227303295
2667
+ - type: f1
2668
+ value: 28.944584739759726
2669
+ - task:
2670
+ type: Classification
2671
+ dataset:
2672
+ type: mteb/amazon_massive_scenario
2673
+ name: MTEB MassiveScenarioClassification (it)
2674
+ config: it
2675
+ split: test
2676
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2677
+ metrics:
2678
+ - type: accuracy
2679
+ value: 31.116341627437794
2680
+ - type: f1
2681
+ value: 29.410419968584232
2682
+ - task:
2683
+ type: Classification
2684
+ dataset:
2685
+ type: mteb/amazon_massive_scenario
2686
+ name: MTEB MassiveScenarioClassification (ja)
2687
+ config: ja
2688
+ split: test
2689
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2690
+ metrics:
2691
+ - type: accuracy
2692
+ value: 31.136516476126424
2693
+ - type: f1
2694
+ value: 29.6719090248999
2695
+ - task:
2696
+ type: Classification
2697
+ dataset:
2698
+ type: mteb/amazon_massive_scenario
2699
+ name: MTEB MassiveScenarioClassification (jv)
2700
+ config: jv
2701
+ split: test
2702
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2703
+ metrics:
2704
+ - type: accuracy
2705
+ value: 34.3813046402152
2706
+ - type: f1
2707
+ value: 31.481374035431443
2708
+ - task:
2709
+ type: Classification
2710
+ dataset:
2711
+ type: mteb/amazon_massive_scenario
2712
+ name: MTEB MassiveScenarioClassification (ka)
2713
+ config: ka
2714
+ split: test
2715
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2716
+ metrics:
2717
+ - type: accuracy
2718
+ value: 18.3591123066577
2719
+ - type: f1
2720
+ value: 16.158113168004945
2721
+ - task:
2722
+ type: Classification
2723
+ dataset:
2724
+ type: mteb/amazon_massive_scenario
2725
+ name: MTEB MassiveScenarioClassification (km)
2726
+ config: km
2727
+ split: test
2728
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2729
+ metrics:
2730
+ - type: accuracy
2731
+ value: 19.250168123739073
2732
+ - type: f1
2733
+ value: 16.728590019294526
2734
+ - task:
2735
+ type: Classification
2736
+ dataset:
2737
+ type: mteb/amazon_massive_scenario
2738
+ name: MTEB MassiveScenarioClassification (kn)
2739
+ config: kn
2740
+ split: test
2741
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2742
+ metrics:
2743
+ - type: accuracy
2744
+ value: 16.04236718224613
2745
+ - type: f1
2746
+ value: 14.304518042661318
2747
+ - task:
2748
+ type: Classification
2749
+ dataset:
2750
+ type: mteb/amazon_massive_scenario
2751
+ name: MTEB MassiveScenarioClassification (ko)
2752
+ config: ko
2753
+ split: test
2754
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2755
+ metrics:
2756
+ - type: accuracy
2757
+ value: 28.95427034297243
2758
+ - type: f1
2759
+ value: 26.14194694048272
2760
+ - task:
2761
+ type: Classification
2762
+ dataset:
2763
+ type: mteb/amazon_massive_scenario
2764
+ name: MTEB MassiveScenarioClassification (lv)
2765
+ config: lv
2766
+ split: test
2767
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2768
+ metrics:
2769
+ - type: accuracy
2770
+ value: 32.286482851378615
2771
+ - type: f1
2772
+ value: 29.426003364232837
2773
+ - task:
2774
+ type: Classification
2775
+ dataset:
2776
+ type: mteb/amazon_massive_scenario
2777
+ name: MTEB MassiveScenarioClassification (ml)
2778
+ config: ml
2779
+ split: test
2780
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2781
+ metrics:
2782
+ - type: accuracy
2783
+ value: 18.762609280430397
2784
+ - type: f1
2785
+ value: 15.764539982254005
2786
+ - task:
2787
+ type: Classification
2788
+ dataset:
2789
+ type: mteb/amazon_massive_scenario
2790
+ name: MTEB MassiveScenarioClassification (mn)
2791
+ config: mn
2792
+ split: test
2793
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2794
+ metrics:
2795
+ - type: accuracy
2796
+ value: 21.570275722932077
2797
+ - type: f1
2798
+ value: 18.98517402487539
2799
+ - task:
2800
+ type: Classification
2801
+ dataset:
2802
+ type: mteb/amazon_massive_scenario
2803
+ name: MTEB MassiveScenarioClassification (ms)
2804
+ config: ms
2805
+ split: test
2806
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2807
+ metrics:
2808
+ - type: accuracy
2809
+ value: 35.95158036314728
2810
+ - type: f1
2811
+ value: 33.45853327725729
2812
+ - task:
2813
+ type: Classification
2814
+ dataset:
2815
+ type: mteb/amazon_massive_scenario
2816
+ name: MTEB MassiveScenarioClassification (my)
2817
+ config: my
2818
+ split: test
2819
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2820
+ metrics:
2821
+ - type: accuracy
2822
+ value: 20.03026227303295
2823
+ - type: f1
2824
+ value: 16.864192363299615
2825
+ - task:
2826
+ type: Classification
2827
+ dataset:
2828
+ type: mteb/amazon_massive_scenario
2829
+ name: MTEB MassiveScenarioClassification (nb)
2830
+ config: nb
2831
+ split: test
2832
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2833
+ metrics:
2834
+ - type: accuracy
2835
+ value: 32.89172831203766
2836
+ - type: f1
2837
+ value: 29.90554132334421
2838
+ - task:
2839
+ type: Classification
2840
+ dataset:
2841
+ type: mteb/amazon_massive_scenario
2842
+ name: MTEB MassiveScenarioClassification (nl)
2843
+ config: nl
2844
+ split: test
2845
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2846
+ metrics:
2847
+ - type: accuracy
2848
+ value: 32.36381977135171
2849
+ - type: f1
2850
+ value: 29.57501043505274
2851
  - task:
2852
  type: Classification
2853
  dataset:
2854
+ type: mteb/amazon_massive_scenario
2855
+ name: MTEB MassiveScenarioClassification (pl)
2856
+ config: pl
2857
  split: test
2858
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2859
  metrics:
2860
  - type: accuracy
2861
+ value: 31.856086079354405
2862
  - type: f1
2863
+ value: 30.242758819443548
2864
  - task:
2865
+ type: Classification
2866
  dataset:
2867
+ type: mteb/amazon_massive_scenario
2868
+ name: MTEB MassiveScenarioClassification (pt)
2869
+ config: pt
2870
  split: test
2871
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2872
  metrics:
2873
+ - type: accuracy
2874
+ value: 31.694687289845326
2875
+ - type: f1
2876
+ value: 29.495870419371684
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2877
  - task:
2878
  type: Classification
2879
  dataset:
2880
+ type: mteb/amazon_massive_scenario
2881
+ name: MTEB MassiveScenarioClassification (ro)
2882
+ config: ro
2883
  split: test
2884
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2885
  metrics:
2886
  - type: accuracy
2887
+ value: 27.01412239408204
 
 
2888
  - type: f1
2889
+ value: 26.44782204477556
2890
  - task:
2891
  type: Classification
2892
  dataset:
2893
+ type: mteb/amazon_massive_scenario
2894
+ name: MTEB MassiveScenarioClassification (ru)
2895
+ config: ru
2896
  split: test
2897
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2898
  metrics:
2899
  - type: accuracy
2900
+ value: 30.554808338937463
2901
  - type: f1
2902
+ value: 27.93696283146029
2903
  - task:
2904
  type: Classification
2905
  dataset:
2906
+ type: mteb/amazon_massive_scenario
2907
+ name: MTEB MassiveScenarioClassification (sl)
2908
+ config: sl
2909
  split: test
2910
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2911
  metrics:
2912
  - type: accuracy
2913
+ value: 35.813718897108274
2914
  - type: f1
2915
+ value: 33.021495683147286
2916
  - task:
2917
  type: Classification
2918
  dataset:
2919
+ type: mteb/amazon_massive_scenario
2920
+ name: MTEB MassiveScenarioClassification (sq)
2921
+ config: sq
2922
  split: test
2923
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2924
  metrics:
2925
  - type: accuracy
2926
+ value: 22.992602555480836
2927
  - type: f1
2928
+ value: 21.524928515996447
2929
  - task:
2930
  type: Classification
2931
  dataset:
2932
+ type: mteb/amazon_massive_scenario
2933
+ name: MTEB MassiveScenarioClassification (sv)
2934
+ config: sv
2935
  split: test
2936
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2937
  metrics:
2938
  - type: accuracy
2939
+ value: 34.7074646940148
2940
  - type: f1
2941
+ value: 31.54759971873104
2942
  - task:
2943
  type: Classification
2944
  dataset:
2945
+ type: mteb/amazon_massive_scenario
2946
+ name: MTEB MassiveScenarioClassification (sw)
2947
+ config: sw
2948
  split: test
2949
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2950
  metrics:
2951
  - type: accuracy
2952
+ value: 36.240753194351036
2953
  - type: f1
2954
+ value: 33.34397881816082
2955
  - task:
2956
  type: Classification
2957
  dataset:
2958
+ type: mteb/amazon_massive_scenario
2959
+ name: MTEB MassiveScenarioClassification (ta)
2960
+ config: ta
2961
+ split: test
2962
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2963
+ metrics:
2964
+ - type: accuracy
2965
+ value: 19.741089441829185
2966
+ - type: f1
2967
+ value: 16.129268723975766
2968
+ - task:
2969
+ type: Classification
2970
+ dataset:
2971
+ type: mteb/amazon_massive_scenario
2972
+ name: MTEB MassiveScenarioClassification (te)
2973
+ config: te
2974
+ split: test
2975
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2976
+ metrics:
2977
+ - type: accuracy
2978
+ value: 17.54203093476799
2979
+ - type: f1
2980
+ value: 15.537381383894061
2981
+ - task:
2982
+ type: Classification
2983
+ dataset:
2984
+ type: mteb/amazon_massive_scenario
2985
+ name: MTEB MassiveScenarioClassification (th)
2986
  config: th
2987
  split: test
2988
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
2989
  metrics:
2990
  - type: accuracy
2991
+ value: 22.817753866846
2992
  - type: f1
2993
+ value: 20.72245485990428
2994
  - task:
2995
  type: Classification
2996
  dataset:
2997
+ type: mteb/amazon_massive_scenario
2998
+ name: MTEB MassiveScenarioClassification (tl)
2999
+ config: tl
3000
  split: test
3001
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
3002
  metrics:
3003
  - type: accuracy
3004
+ value: 28.58439811701412
3005
  - type: f1
3006
+ value: 26.88190194852028
3007
  - task:
3008
  type: Classification
3009
  dataset:
3010
+ type: mteb/amazon_massive_scenario
3011
+ name: MTEB MassiveScenarioClassification (tr)
3012
+ config: tr
3013
  split: test
3014
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
3015
  metrics:
3016
  - type: accuracy
3017
+ value: 28.60457296570275
3018
  - type: f1
3019
+ value: 26.98989368733863
3020
  - task:
3021
  type: Classification
3022
  dataset:
3023
+ type: mteb/amazon_massive_scenario
3024
+ name: MTEB MassiveScenarioClassification (ur)
3025
+ config: ur
3026
  split: test
3027
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
3028
  metrics:
3029
  - type: accuracy
3030
+ value: 23.345662407531943
3031
  - type: f1
3032
+ value: 19.75032390408514
3033
  - task:
3034
  type: Classification
3035
  dataset:
3036
+ type: mteb/amazon_massive_scenario
3037
+ name: MTEB MassiveScenarioClassification (vi)
3038
+ config: vi
3039
  split: test
3040
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
3041
  metrics:
3042
  - type: accuracy
3043
+ value: 29.71082716879624
3044
  - type: f1
3045
+ value: 26.675920460240782
3046
  - task:
3047
  type: Classification
3048
  dataset:
3049
+ type: mteb/amazon_massive_scenario
3050
+ name: MTEB MassiveScenarioClassification (zh-CN)
3051
+ config: zh-CN
3052
  split: test
3053
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
3054
  metrics:
3055
  - type: accuracy
3056
+ value: 44.09549428379288
3057
  - type: f1
3058
+ value: 41.275350430825675
3059
  - task:
3060
  type: Classification
3061
  dataset:
3062
+ type: mteb/amazon_massive_scenario
3063
+ name: MTEB MassiveScenarioClassification (zh-TW)
3064
+ config: zh-TW
3065
  split: test
3066
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
3067
  metrics:
3068
  - type: accuracy
3069
+ value: 37.24277067921991
3070
  - type: f1
3071
+ value: 35.65629114113254
3072
  - task:
3073
  type: Clustering
3074
  dataset:
 
3091
  metrics:
3092
  - type: v_measure
3093
  value: 16.58582885790446
3094
+ - task:
3095
+ type: Reranking
3096
+ dataset:
3097
+ type: mteb/mind_small
3098
+ name: MTEB MindSmallReranking
3099
+ config: default
3100
+ split: test
3101
+ revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
3102
+ metrics:
3103
+ - type: map
3104
+ value: 26.730268595233923
3105
+ - type: mrr
3106
+ value: 27.065185919114704
3107
  - task:
3108
  type: Retrieval
3109
  dataset:
 
3173
  value: 1.702
3174
  - type: recall_at_5
3175
  value: 1.9879999999999998
3176
+ - task:
3177
+ type: Retrieval
3178
+ dataset:
3179
+ type: nq
3180
+ name: MTEB NQ
3181
+ config: default
3182
+ split: test
3183
+ revision: None
3184
+ metrics:
3185
+ - type: map_at_1
3186
+ value: 1.214
3187
+ - type: map_at_10
3188
+ value: 2.067
3189
+ - type: map_at_100
3190
+ value: 2.2399999999999998
3191
+ - type: map_at_1000
3192
+ value: 2.2689999999999997
3193
+ - type: map_at_3
3194
+ value: 1.691
3195
+ - type: map_at_5
3196
+ value: 1.916
3197
+ - type: mrr_at_1
3198
+ value: 1.506
3199
+ - type: mrr_at_10
3200
+ value: 2.413
3201
+ - type: mrr_at_100
3202
+ value: 2.587
3203
+ - type: mrr_at_1000
3204
+ value: 2.616
3205
+ - type: mrr_at_3
3206
+ value: 2.023
3207
+ - type: mrr_at_5
3208
+ value: 2.246
3209
+ - type: ndcg_at_1
3210
+ value: 1.506
3211
+ - type: ndcg_at_10
3212
+ value: 2.703
3213
+ - type: ndcg_at_100
3214
+ value: 3.66
3215
+ - type: ndcg_at_1000
3216
+ value: 4.6
3217
+ - type: ndcg_at_3
3218
+ value: 1.9300000000000002
3219
+ - type: ndcg_at_5
3220
+ value: 2.33
3221
+ - type: precision_at_1
3222
+ value: 1.506
3223
+ - type: precision_at_10
3224
+ value: 0.539
3225
+ - type: precision_at_100
3226
+ value: 0.11
3227
+ - type: precision_at_1000
3228
+ value: 0.02
3229
+ - type: precision_at_3
3230
+ value: 0.9369999999999999
3231
+ - type: precision_at_5
3232
+ value: 0.7939999999999999
3233
+ - type: recall_at_1
3234
+ value: 1.214
3235
+ - type: recall_at_10
3236
+ value: 4.34
3237
+ - type: recall_at_100
3238
+ value: 8.905000000000001
3239
+ - type: recall_at_1000
3240
+ value: 16.416
3241
+ - type: recall_at_3
3242
+ value: 2.3009999999999997
3243
+ - type: recall_at_5
3244
+ value: 3.2489999999999997
3245
  - task:
3246
  type: Retrieval
3247
  dataset:
 
4399
  value: 0.231
4400
  - type: recall_at_5
4401
  value: 0.367
4402
+ - task:
4403
+ type: Retrieval
4404
+ dataset:
4405
+ type: webis-touche2020
4406
+ name: MTEB Touche2020
4407
+ config: default
4408
+ split: test
4409
+ revision: None
4410
+ metrics:
4411
+ - type: map_at_1
4412
+ value: 0.6799999999999999
4413
+ - type: map_at_10
4414
+ value: 2.1420000000000003
4415
+ - type: map_at_100
4416
+ value: 2.888
4417
+ - type: map_at_1000
4418
+ value: 3.3779999999999997
4419
+ - type: map_at_3
4420
+ value: 1.486
4421
+ - type: map_at_5
4422
+ value: 1.7579999999999998
4423
+ - type: mrr_at_1
4424
+ value: 12.245000000000001
4425
+ - type: mrr_at_10
4426
+ value: 22.12
4427
+ - type: mrr_at_100
4428
+ value: 23.407
4429
+ - type: mrr_at_1000
4430
+ value: 23.483999999999998
4431
+ - type: mrr_at_3
4432
+ value: 19.048000000000002
4433
+ - type: mrr_at_5
4434
+ value: 20.986
4435
+ - type: ndcg_at_1
4436
+ value: 10.204
4437
+ - type: ndcg_at_10
4438
+ value: 7.374
4439
+ - type: ndcg_at_100
4440
+ value: 10.524000000000001
4441
+ - type: ndcg_at_1000
4442
+ value: 18.4
4443
+ - type: ndcg_at_3
4444
+ value: 9.913
4445
+ - type: ndcg_at_5
4446
+ value: 8.938
4447
+ - type: precision_at_1
4448
+ value: 12.245000000000001
4449
+ - type: precision_at_10
4450
+ value: 7.142999999999999
4451
+ - type: precision_at_100
4452
+ value: 2.4490000000000003
4453
+ - type: precision_at_1000
4454
+ value: 0.731
4455
+ - type: precision_at_3
4456
+ value: 11.565
4457
+ - type: precision_at_5
4458
+ value: 9.796000000000001
4459
+ - type: recall_at_1
4460
+ value: 0.6799999999999999
4461
+ - type: recall_at_10
4462
+ value: 4.038
4463
+ - type: recall_at_100
4464
+ value: 14.151
4465
+ - type: recall_at_1000
4466
+ value: 40.111999999999995
4467
+ - type: recall_at_3
4468
+ value: 1.921
4469
+ - type: recall_at_5
4470
+ value: 2.604
4471
  - task:
4472
  type: Classification
4473
  dataset:
 
4616
  - type: max_ap
4617
  value: 64.59241956109807
4618
  - type: max_f1
4619
+ value: 57.83203629255339