Text Generation
Transformers.js
ONNX
bloom
Eval Results (legacy)
aboly47 commited on
Commit
d4f52d5
·
verified ·
1 Parent(s): eddfa8d

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ onnx/model.onnx_data filter=lfs diff=lfs merge=lfs -text
37
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,914 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ datasets:
3
+ - bigscience/xP3
4
+ license: bigscience-bloom-rail-1.0
5
+ language:
6
+ - ak
7
+ - ar
8
+ - as
9
+ - bm
10
+ - bn
11
+ - ca
12
+ - code
13
+ - en
14
+ - es
15
+ - eu
16
+ - fon
17
+ - fr
18
+ - gu
19
+ - hi
20
+ - id
21
+ - ig
22
+ - ki
23
+ - kn
24
+ - lg
25
+ - ln
26
+ - ml
27
+ - mr
28
+ - ne
29
+ - nso
30
+ - ny
31
+ - or
32
+ - pa
33
+ - pt
34
+ - rn
35
+ - rw
36
+ - sn
37
+ - st
38
+ - sw
39
+ - ta
40
+ - te
41
+ - tn
42
+ - ts
43
+ - tum
44
+ - tw
45
+ - ur
46
+ - vi
47
+ - wo
48
+ - xh
49
+ - yo
50
+ - zh
51
+ - zu
52
+ programming_language:
53
+ - C
54
+ - C++
55
+ - C#
56
+ - Go
57
+ - Java
58
+ - JavaScript
59
+ - Lua
60
+ - PHP
61
+ - Python
62
+ - Ruby
63
+ - Rust
64
+ - Scala
65
+ - TypeScript
66
+ pipeline_tag: text-generation
67
+ widget:
68
+ - text: "\u4E00\u4E2A\u4F20\u5947\u7684\u5F00\u7AEF\uFF0C\u4E00\u4E2A\u4E0D\u706D\u7684\
69
+ \u795E\u8BDD\uFF0C\u8FD9\u4E0D\u4EC5\u4EC5\u662F\u4E00\u90E8\u7535\u5F71\uFF0C\
70
+ \u800C\u662F\u4F5C\u4E3A\u4E00\u4E2A\u8D70\u8FDB\u65B0\u65F6\u4EE3\u7684\u6807\
71
+ \u7B7E\uFF0C\u6C38\u8FDC\u5F6A\u70B3\u53F2\u518C\u3002Would you rate the previous\
72
+ \ review as positive, neutral or negative?"
73
+ example_title: zh-en sentiment
74
+ - text: "\u4E00\u4E2A\u4F20\u5947\u7684\u5F00\u7AEF\uFF0C\u4E00\u4E2A\u4E0D\u706D\u7684\
75
+ \u795E\u8BDD\uFF0C\u8FD9\u4E0D\u4EC5\u4EC5\u662F\u4E00\u90E8\u7535\u5F71\uFF0C\
76
+ \u800C\u662F\u4F5C\u4E3A\u4E00\u4E2A\u8D70\u8FDB\u65B0\u65F6\u4EE3\u7684\u6807\
77
+ \u7B7E\uFF0C\u6C38\u8FDC\u5F6A\u70B3\u53F2\u518C\u3002\u4F60\u8BA4\u4E3A\u8FD9\
78
+ \u53E5\u8BDD\u7684\u7ACB\u573A\u662F\u8D5E\u626C\u3001\u4E2D\u7ACB\u8FD8\u662F\
79
+ \u6279\u8BC4\uFF1F"
80
+ example_title: zh-zh sentiment
81
+ - text: "Suggest at least five related search terms to \"M\u1EA1ng neural nh\xE2n\
82
+ \ t\u1EA1o\"."
83
+ example_title: vi-en query
84
+ - text: "Proposez au moins cinq mots cl\xE9s concernant \xABR\xE9seau de neurones\
85
+ \ artificiels\xBB."
86
+ example_title: fr-fr query
87
+ - text: Explain in a sentence in Telugu what is backpropagation in neural networks.
88
+ example_title: te-en qa
89
+ - text: Why is the sky blue?
90
+ example_title: en-en qa
91
+ - text: 'Write a fairy tale about a troll saving a princess from a dangerous dragon.
92
+ The fairy tale is a masterpiece that has achieved praise worldwide and its moral
93
+ is "Heroes Come in All Shapes and Sizes". Story (in Spanish):'
94
+ example_title: es-en fable
95
+ - text: 'Write a fable about wood elves living in a forest that is suddenly invaded
96
+ by ogres. The fable is a masterpiece that has achieved praise worldwide and its
97
+ moral is "Violence is the last refuge of the incompetent". Fable (in Hindi):'
98
+ example_title: hi-en fable
99
+ model-index:
100
+ - name: bloomz-560m
101
+ results:
102
+ - task:
103
+ type: Coreference resolution
104
+ dataset:
105
+ type: winogrande
106
+ name: Winogrande XL (xl)
107
+ config: xl
108
+ split: validation
109
+ revision: a80f460359d1e9a67c006011c94de42a8759430c
110
+ metrics:
111
+ - type: Accuracy
112
+ value: 52.41
113
+ - task:
114
+ type: Coreference resolution
115
+ dataset:
116
+ type: Muennighoff/xwinograd
117
+ name: XWinograd (en)
118
+ config: en
119
+ split: test
120
+ revision: 9dd5ea5505fad86b7bedad667955577815300cee
121
+ metrics:
122
+ - type: Accuracy
123
+ value: 51.01
124
+ - task:
125
+ type: Coreference resolution
126
+ dataset:
127
+ type: Muennighoff/xwinograd
128
+ name: XWinograd (fr)
129
+ config: fr
130
+ split: test
131
+ revision: 9dd5ea5505fad86b7bedad667955577815300cee
132
+ metrics:
133
+ - type: Accuracy
134
+ value: 51.81
135
+ - task:
136
+ type: Coreference resolution
137
+ dataset:
138
+ type: Muennighoff/xwinograd
139
+ name: XWinograd (jp)
140
+ config: jp
141
+ split: test
142
+ revision: 9dd5ea5505fad86b7bedad667955577815300cee
143
+ metrics:
144
+ - type: Accuracy
145
+ value: 52.03
146
+ - task:
147
+ type: Coreference resolution
148
+ dataset:
149
+ type: Muennighoff/xwinograd
150
+ name: XWinograd (pt)
151
+ config: pt
152
+ split: test
153
+ revision: 9dd5ea5505fad86b7bedad667955577815300cee
154
+ metrics:
155
+ - type: Accuracy
156
+ value: 53.99
157
+ - task:
158
+ type: Coreference resolution
159
+ dataset:
160
+ type: Muennighoff/xwinograd
161
+ name: XWinograd (ru)
162
+ config: ru
163
+ split: test
164
+ revision: 9dd5ea5505fad86b7bedad667955577815300cee
165
+ metrics:
166
+ - type: Accuracy
167
+ value: 53.97
168
+ - task:
169
+ type: Coreference resolution
170
+ dataset:
171
+ type: Muennighoff/xwinograd
172
+ name: XWinograd (zh)
173
+ config: zh
174
+ split: test
175
+ revision: 9dd5ea5505fad86b7bedad667955577815300cee
176
+ metrics:
177
+ - type: Accuracy
178
+ value: 54.76
179
+ - task:
180
+ type: Natural language inference
181
+ dataset:
182
+ type: anli
183
+ name: ANLI (r1)
184
+ config: r1
185
+ split: validation
186
+ revision: 9dbd830a06fea8b1c49d6e5ef2004a08d9f45094
187
+ metrics:
188
+ - type: Accuracy
189
+ value: 33.4
190
+ - task:
191
+ type: Natural language inference
192
+ dataset:
193
+ type: anli
194
+ name: ANLI (r2)
195
+ config: r2
196
+ split: validation
197
+ revision: 9dbd830a06fea8b1c49d6e5ef2004a08d9f45094
198
+ metrics:
199
+ - type: Accuracy
200
+ value: 33.4
201
+ - task:
202
+ type: Natural language inference
203
+ dataset:
204
+ type: anli
205
+ name: ANLI (r3)
206
+ config: r3
207
+ split: validation
208
+ revision: 9dbd830a06fea8b1c49d6e5ef2004a08d9f45094
209
+ metrics:
210
+ - type: Accuracy
211
+ value: 33.5
212
+ - task:
213
+ type: Natural language inference
214
+ dataset:
215
+ type: super_glue
216
+ name: SuperGLUE (cb)
217
+ config: cb
218
+ split: validation
219
+ revision: 9e12063561e7e6c79099feb6d5a493142584e9e2
220
+ metrics:
221
+ - type: Accuracy
222
+ value: 53.57
223
+ - task:
224
+ type: Natural language inference
225
+ dataset:
226
+ type: super_glue
227
+ name: SuperGLUE (rte)
228
+ config: rte
229
+ split: validation
230
+ revision: 9e12063561e7e6c79099feb6d5a493142584e9e2
231
+ metrics:
232
+ - type: Accuracy
233
+ value: 67.15
234
+ - task:
235
+ type: Natural language inference
236
+ dataset:
237
+ type: xnli
238
+ name: XNLI (ar)
239
+ config: ar
240
+ split: validation
241
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
242
+ metrics:
243
+ - type: Accuracy
244
+ value: 44.46
245
+ - task:
246
+ type: Natural language inference
247
+ dataset:
248
+ type: xnli
249
+ name: XNLI (bg)
250
+ config: bg
251
+ split: validation
252
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
253
+ metrics:
254
+ - type: Accuracy
255
+ value: 39.76
256
+ - task:
257
+ type: Natural language inference
258
+ dataset:
259
+ type: xnli
260
+ name: XNLI (de)
261
+ config: de
262
+ split: validation
263
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
264
+ metrics:
265
+ - type: Accuracy
266
+ value: 39.36
267
+ - task:
268
+ type: Natural language inference
269
+ dataset:
270
+ type: xnli
271
+ name: XNLI (el)
272
+ config: el
273
+ split: validation
274
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
275
+ metrics:
276
+ - type: Accuracy
277
+ value: 40.96
278
+ - task:
279
+ type: Natural language inference
280
+ dataset:
281
+ type: xnli
282
+ name: XNLI (en)
283
+ config: en
284
+ split: validation
285
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
286
+ metrics:
287
+ - type: Accuracy
288
+ value: 46.43
289
+ - task:
290
+ type: Natural language inference
291
+ dataset:
292
+ type: xnli
293
+ name: XNLI (es)
294
+ config: es
295
+ split: validation
296
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
297
+ metrics:
298
+ - type: Accuracy
299
+ value: 44.98
300
+ - task:
301
+ type: Natural language inference
302
+ dataset:
303
+ type: xnli
304
+ name: XNLI (fr)
305
+ config: fr
306
+ split: validation
307
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
308
+ metrics:
309
+ - type: Accuracy
310
+ value: 45.54
311
+ - task:
312
+ type: Natural language inference
313
+ dataset:
314
+ type: xnli
315
+ name: XNLI (hi)
316
+ config: hi
317
+ split: validation
318
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
319
+ metrics:
320
+ - type: Accuracy
321
+ value: 41.81
322
+ - task:
323
+ type: Natural language inference
324
+ dataset:
325
+ type: xnli
326
+ name: XNLI (ru)
327
+ config: ru
328
+ split: validation
329
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
330
+ metrics:
331
+ - type: Accuracy
332
+ value: 39.64
333
+ - task:
334
+ type: Natural language inference
335
+ dataset:
336
+ type: xnli
337
+ name: XNLI (sw)
338
+ config: sw
339
+ split: validation
340
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
341
+ metrics:
342
+ - type: Accuracy
343
+ value: 38.35
344
+ - task:
345
+ type: Natural language inference
346
+ dataset:
347
+ type: xnli
348
+ name: XNLI (th)
349
+ config: th
350
+ split: validation
351
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
352
+ metrics:
353
+ - type: Accuracy
354
+ value: 35.5
355
+ - task:
356
+ type: Natural language inference
357
+ dataset:
358
+ type: xnli
359
+ name: XNLI (tr)
360
+ config: tr
361
+ split: validation
362
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
363
+ metrics:
364
+ - type: Accuracy
365
+ value: 37.31
366
+ - task:
367
+ type: Natural language inference
368
+ dataset:
369
+ type: xnli
370
+ name: XNLI (ur)
371
+ config: ur
372
+ split: validation
373
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
374
+ metrics:
375
+ - type: Accuracy
376
+ value: 38.96
377
+ - task:
378
+ type: Natural language inference
379
+ dataset:
380
+ type: xnli
381
+ name: XNLI (vi)
382
+ config: vi
383
+ split: validation
384
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
385
+ metrics:
386
+ - type: Accuracy
387
+ value: 44.74
388
+ - task:
389
+ type: Natural language inference
390
+ dataset:
391
+ type: xnli
392
+ name: XNLI (zh)
393
+ config: zh
394
+ split: validation
395
+ revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
396
+ metrics:
397
+ - type: Accuracy
398
+ value: 44.66
399
+ - task:
400
+ type: Program synthesis
401
+ dataset:
402
+ type: openai_humaneval
403
+ name: HumanEval
404
+ config: None
405
+ split: test
406
+ revision: e8dc562f5de170c54b5481011dd9f4fa04845771
407
+ metrics:
408
+ - type: Pass@1
409
+ value: 2.18
410
+ - type: Pass@10
411
+ value: 4.11
412
+ - type: Pass@100
413
+ value: 9.0
414
+ - task:
415
+ type: Sentence completion
416
+ dataset:
417
+ type: story_cloze
418
+ name: StoryCloze (2016)
419
+ config: '2016'
420
+ split: validation
421
+ revision: e724c6f8cdf7c7a2fb229d862226e15b023ee4db
422
+ metrics:
423
+ - type: Accuracy
424
+ value: 60.29
425
+ - task:
426
+ type: Sentence completion
427
+ dataset:
428
+ type: super_glue
429
+ name: SuperGLUE (copa)
430
+ config: copa
431
+ split: validation
432
+ revision: 9e12063561e7e6c79099feb6d5a493142584e9e2
433
+ metrics:
434
+ - type: Accuracy
435
+ value: 52.0
436
+ - task:
437
+ type: Sentence completion
438
+ dataset:
439
+ type: xcopa
440
+ name: XCOPA (et)
441
+ config: et
442
+ split: validation
443
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
444
+ metrics:
445
+ - type: Accuracy
446
+ value: 53.0
447
+ - task:
448
+ type: Sentence completion
449
+ dataset:
450
+ type: xcopa
451
+ name: XCOPA (ht)
452
+ config: ht
453
+ split: validation
454
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
455
+ metrics:
456
+ - type: Accuracy
457
+ value: 49.0
458
+ - task:
459
+ type: Sentence completion
460
+ dataset:
461
+ type: xcopa
462
+ name: XCOPA (id)
463
+ config: id
464
+ split: validation
465
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
466
+ metrics:
467
+ - type: Accuracy
468
+ value: 57.0
469
+ - task:
470
+ type: Sentence completion
471
+ dataset:
472
+ type: xcopa
473
+ name: XCOPA (it)
474
+ config: it
475
+ split: validation
476
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
477
+ metrics:
478
+ - type: Accuracy
479
+ value: 52.0
480
+ - task:
481
+ type: Sentence completion
482
+ dataset:
483
+ type: xcopa
484
+ name: XCOPA (qu)
485
+ config: qu
486
+ split: validation
487
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
488
+ metrics:
489
+ - type: Accuracy
490
+ value: 55.0
491
+ - task:
492
+ type: Sentence completion
493
+ dataset:
494
+ type: xcopa
495
+ name: XCOPA (sw)
496
+ config: sw
497
+ split: validation
498
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
499
+ metrics:
500
+ - type: Accuracy
501
+ value: 56.0
502
+ - task:
503
+ type: Sentence completion
504
+ dataset:
505
+ type: xcopa
506
+ name: XCOPA (ta)
507
+ config: ta
508
+ split: validation
509
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
510
+ metrics:
511
+ - type: Accuracy
512
+ value: 58.0
513
+ - task:
514
+ type: Sentence completion
515
+ dataset:
516
+ type: xcopa
517
+ name: XCOPA (th)
518
+ config: th
519
+ split: validation
520
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
521
+ metrics:
522
+ - type: Accuracy
523
+ value: 58.0
524
+ - task:
525
+ type: Sentence completion
526
+ dataset:
527
+ type: xcopa
528
+ name: XCOPA (tr)
529
+ config: tr
530
+ split: validation
531
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
532
+ metrics:
533
+ - type: Accuracy
534
+ value: 61.0
535
+ - task:
536
+ type: Sentence completion
537
+ dataset:
538
+ type: xcopa
539
+ name: XCOPA (vi)
540
+ config: vi
541
+ split: validation
542
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
543
+ metrics:
544
+ - type: Accuracy
545
+ value: 61.0
546
+ - task:
547
+ type: Sentence completion
548
+ dataset:
549
+ type: xcopa
550
+ name: XCOPA (zh)
551
+ config: zh
552
+ split: validation
553
+ revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
554
+ metrics:
555
+ - type: Accuracy
556
+ value: 61.0
557
+ - task:
558
+ type: Sentence completion
559
+ dataset:
560
+ type: Muennighoff/xstory_cloze
561
+ name: XStoryCloze (ar)
562
+ config: ar
563
+ split: validation
564
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
565
+ metrics:
566
+ - type: Accuracy
567
+ value: 54.4
568
+ - task:
569
+ type: Sentence completion
570
+ dataset:
571
+ type: Muennighoff/xstory_cloze
572
+ name: XStoryCloze (es)
573
+ config: es
574
+ split: validation
575
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
576
+ metrics:
577
+ - type: Accuracy
578
+ value: 56.45
579
+ - task:
580
+ type: Sentence completion
581
+ dataset:
582
+ type: Muennighoff/xstory_cloze
583
+ name: XStoryCloze (eu)
584
+ config: eu
585
+ split: validation
586
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
587
+ metrics:
588
+ - type: Accuracy
589
+ value: 50.56
590
+ - task:
591
+ type: Sentence completion
592
+ dataset:
593
+ type: Muennighoff/xstory_cloze
594
+ name: XStoryCloze (hi)
595
+ config: hi
596
+ split: validation
597
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
598
+ metrics:
599
+ - type: Accuracy
600
+ value: 55.79
601
+ - task:
602
+ type: Sentence completion
603
+ dataset:
604
+ type: Muennighoff/xstory_cloze
605
+ name: XStoryCloze (id)
606
+ config: id
607
+ split: validation
608
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
609
+ metrics:
610
+ - type: Accuracy
611
+ value: 57.84
612
+ - task:
613
+ type: Sentence completion
614
+ dataset:
615
+ type: Muennighoff/xstory_cloze
616
+ name: XStoryCloze (my)
617
+ config: my
618
+ split: validation
619
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
620
+ metrics:
621
+ - type: Accuracy
622
+ value: 47.05
623
+ - task:
624
+ type: Sentence completion
625
+ dataset:
626
+ type: Muennighoff/xstory_cloze
627
+ name: XStoryCloze (ru)
628
+ config: ru
629
+ split: validation
630
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
631
+ metrics:
632
+ - type: Accuracy
633
+ value: 53.14
634
+ - task:
635
+ type: Sentence completion
636
+ dataset:
637
+ type: Muennighoff/xstory_cloze
638
+ name: XStoryCloze (sw)
639
+ config: sw
640
+ split: validation
641
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
642
+ metrics:
643
+ - type: Accuracy
644
+ value: 51.36
645
+ - task:
646
+ type: Sentence completion
647
+ dataset:
648
+ type: Muennighoff/xstory_cloze
649
+ name: XStoryCloze (te)
650
+ config: te
651
+ split: validation
652
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
653
+ metrics:
654
+ - type: Accuracy
655
+ value: 54.86
656
+ - task:
657
+ type: Sentence completion
658
+ dataset:
659
+ type: Muennighoff/xstory_cloze
660
+ name: XStoryCloze (zh)
661
+ config: zh
662
+ split: validation
663
+ revision: 8bb76e594b68147f1a430e86829d07189622b90d
664
+ metrics:
665
+ - type: Accuracy
666
+ value: 56.52
667
+ library_name: transformers.js
668
+ base_model:
669
+ - bigscience/bloomz-560m
670
+ ---
671
+
672
+
673
+
674
+ # bloomz-560m (ONNX)
675
+
676
+
677
+ This is an ONNX version of [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m). It was automatically converted and uploaded using [this Hugging Face Space](https://huggingface.co/spaces/onnx-community/convert-to-onnx).
678
+
679
+
680
+ ## Usage with Transformers.js
681
+
682
+
683
+ See the pipeline documentation for `text-generation`: https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.TextGenerationPipeline
684
+
685
+
686
+ ---
687
+
688
+
689
+ ![xmtf](https://github.com/bigscience-workshop/xmtf/blob/master/xmtf_banner.png?raw=true)
690
+
691
+ # Table of Contents
692
+
693
+ 1. [Model Summary](#model-summary)
694
+ 2. [Use](#use)
695
+ 3. [Limitations](#limitations)
696
+ 4. [Training](#training)
697
+ 5. [Evaluation](#evaluation)
698
+ 7. [Citation](#citation)
699
+
700
+ # Model Summary
701
+
702
+ > We present BLOOMZ & mT0, a family of models capable of following human instructions in dozens of languages zero-shot. We finetune BLOOM & mT5 pretrained multilingual language models on our crosslingual task mixture (xP3) and find the resulting models capable of crosslingual generalization to unseen tasks & languages.
703
+
704
+ - **Repository:** [bigscience-workshop/xmtf](https://github.com/bigscience-workshop/xmtf)
705
+ - **Paper:** [Crosslingual Generalization through Multitask Finetuning](https://arxiv.org/abs/2211.01786)
706
+ - **Point of Contact:** [Niklas Muennighoff](mailto:niklas@hf.co)
707
+ - **Languages:** Refer to [bloom](https://huggingface.co/bigscience/bloom) for pretraining & [xP3](https://huggingface.co/datasets/bigscience/xP3) for finetuning language proportions. It understands both pretraining & finetuning languages.
708
+ - **BLOOMZ & mT0 Model Family:**
709
+
710
+ <div class="max-w-full overflow-auto">
711
+ <table>
712
+ <tr>
713
+ <th colspan="12">Multitask finetuned on <a style="font-weight:bold" href=https://huggingface.co/datasets/bigscience/xP3>xP3</a>. Recommended for prompting in English.
714
+ </tr>
715
+ <tr>
716
+ <td>Parameters</td>
717
+ <td>300M</td>
718
+ <td>580M</td>
719
+ <td>1.2B</td>
720
+ <td>3.7B</td>
721
+ <td>13B</td>
722
+ <td>560M</td>
723
+ <td>1.1B</td>
724
+ <td>1.7B</td>
725
+ <td>3B</td>
726
+ <td>7.1B</td>
727
+ <td>176B</td>
728
+ </tr>
729
+ <tr>
730
+ <td>Finetuned Model</td>
731
+ <td><a href=https://huggingface.co/bigscience/mt0-small>mt0-small</a></td>
732
+ <td><a href=https://huggingface.co/bigscience/mt0-base>mt0-base</a></td>
733
+ <td><a href=https://huggingface.co/bigscience/mt0-large>mt0-large</a></td>
734
+ <td><a href=https://huggingface.co/bigscience/mt0-xl>mt0-xl</a></td>
735
+ <td><a href=https://huggingface.co/bigscience/mt0-xxl>mt0-xxl</a></td>
736
+ <td><a href=https://huggingface.co/bigscience/bloomz-560m>bloomz-560m</a></td>
737
+ <td><a href=https://huggingface.co/bigscience/bloomz-1b1>bloomz-1b1</a></td>
738
+ <td><a href=https://huggingface.co/bigscience/bloomz-1b7>bloomz-1b7</a></td>
739
+ <td><a href=https://huggingface.co/bigscience/bloomz-3b>bloomz-3b</a></td>
740
+ <td><a href=https://huggingface.co/bigscience/bloomz-7b1>bloomz-7b1</a></td>
741
+ <td><a href=https://huggingface.co/bigscience/bloomz>bloomz</a></td>
742
+ </tr>
743
+ </tr>
744
+ <tr>
745
+ <th colspan="12">Multitask finetuned on <a style="font-weight:bold" href=https://huggingface.co/datasets/bigscience/xP3mt>xP3mt</a>. Recommended for prompting in non-English.</th>
746
+ </tr>
747
+ <tr>
748
+ <td>Finetuned Model</td>
749
+ <td></td>
750
+ <td></td>
751
+ <td></td>
752
+ <td></td>
753
+ <td><a href=https://huggingface.co/bigscience/mt0-xxl-mt>mt0-xxl-mt</a></td>
754
+ <td></td>
755
+ <td></td>
756
+ <td></td>
757
+ <td></td>
758
+ <td><a href=https://huggingface.co/bigscience/bloomz-7b1-mt>bloomz-7b1-mt</a></td>
759
+ <td><a href=https://huggingface.co/bigscience/bloomz-mt>bloomz-mt</a></td>
760
+ </tr>
761
+ <th colspan="12">Multitask finetuned on <a style="font-weight:bold" href=https://huggingface.co/datasets/Muennighoff/P3>P3</a>. Released for research purposes only. Strictly inferior to above models!</th>
762
+ </tr>
763
+ <tr>
764
+ <td>Finetuned Model</td>
765
+ <td></td>
766
+ <td></td>
767
+ <td></td>
768
+ <td></td>
769
+ <td><a href=https://huggingface.co/bigscience/mt0-xxl-p3>mt0-xxl-p3</a></td>
770
+ <td></td>
771
+ <td></td>
772
+ <td></td>
773
+ <td></td>
774
+ <td><a href=https://huggingface.co/bigscience/bloomz-7b1-p3>bloomz-7b1-p3</a></td>
775
+ <td><a href=https://huggingface.co/bigscience/bloomz-p3>bloomz-p3</a></td>
776
+ </tr>
777
+ <th colspan="12">Original pretrained checkpoints. Not recommended.</th>
778
+ <tr>
779
+ <td>Pretrained Model</td>
780
+ <td><a href=https://huggingface.co/google/mt5-small>mt5-small</a></td>
781
+ <td><a href=https://huggingface.co/google/mt5-base>mt5-base</a></td>
782
+ <td><a href=https://huggingface.co/google/mt5-large>mt5-large</a></td>
783
+ <td><a href=https://huggingface.co/google/mt5-xl>mt5-xl</a></td>
784
+ <td><a href=https://huggingface.co/google/mt5-xxl>mt5-xxl</a></td>
785
+ <td><a href=https://huggingface.co/bigscience/bloom-560m>bloom-560m</a></td>
786
+ <td><a href=https://huggingface.co/bigscience/bloom-1b1>bloom-1b1</a></td>
787
+ <td><a href=https://huggingface.co/bigscience/bloom-1b7>bloom-1b7</a></td>
788
+ <td><a href=https://huggingface.co/bigscience/bloom-3b>bloom-3b</a></td>
789
+ <td><a href=https://huggingface.co/bigscience/bloom-7b1>bloom-7b1</a></td>
790
+ <td><a href=https://huggingface.co/bigscience/bloom>bloom</a></td>
791
+ </tr>
792
+ </table>
793
+ </div>
794
+
795
+ # Use
796
+
797
+ ## Intended use
798
+
799
+ We recommend using the model to perform tasks expressed in natural language. For example, given the prompt "*Translate to English: Je t’aime.*", the model will most likely answer "*I love you.*". Some prompt ideas from our paper:
800
+ - 一个传奇的开端,一个不灭的神话,这不仅仅是一部电影,而是作为一个走进新时代的标签,永远彪炳史册。你认为这句话的立场是赞扬、中立还是批评?
801
+ - Suggest at least five related search terms to "Mạng neural nhân tạo".
802
+ - Write a fairy tale about a troll saving a princess from a dangerous dragon. The fairy tale is a masterpiece that has achieved praise worldwide and its moral is "Heroes Come in All Shapes and Sizes". Story (in Spanish):
803
+ - Explain in a sentence in Telugu what is backpropagation in neural networks.
804
+
805
+ **Feel free to share your generations in the Community tab!**
806
+
807
+ ## How to use
808
+
809
+ ### CPU
810
+
811
+ <details>
812
+ <summary> Click to expand </summary>
813
+
814
+ ```python
815
+ # pip install -q transformers
816
+ from transformers import AutoModelForCausalLM, AutoTokenizer
817
+
818
+ checkpoint = "bigscience/bloomz-560m"
819
+
820
+ tokenizer = AutoTokenizer.from_pretrained(checkpoint)
821
+ model = AutoModelForCausalLM.from_pretrained(checkpoint)
822
+
823
+ inputs = tokenizer.encode("Translate to English: Je t’aime.", return_tensors="pt")
824
+ outputs = model.generate(inputs)
825
+ print(tokenizer.decode(outputs[0]))
826
+ ```
827
+
828
+ </details>
829
+
830
+ ### GPU
831
+
832
+ <details>
833
+ <summary> Click to expand </summary>
834
+
835
+ ```python
836
+ # pip install -q transformers accelerate
837
+ from transformers import AutoModelForCausalLM, AutoTokenizer
838
+
839
+ checkpoint = "bigscience/bloomz-560m"
840
+
841
+ tokenizer = AutoTokenizer.from_pretrained(checkpoint)
842
+ model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype="auto", device_map="auto")
843
+
844
+ inputs = tokenizer.encode("Translate to English: Je t’aime.", return_tensors="pt").to("cuda")
845
+ outputs = model.generate(inputs)
846
+ print(tokenizer.decode(outputs[0]))
847
+ ```
848
+
849
+ </details>
850
+
851
+ ### GPU in 8bit
852
+
853
+ <details>
854
+ <summary> Click to expand </summary>
855
+
856
+ ```python
857
+ # pip install -q transformers accelerate bitsandbytes
858
+ from transformers import AutoModelForCausalLM, AutoTokenizer
859
+
860
+ checkpoint = "bigscience/bloomz-560m"
861
+
862
+ tokenizer = AutoTokenizer.from_pretrained(checkpoint)
863
+ model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto", load_in_8bit=True)
864
+
865
+ inputs = tokenizer.encode("Translate to English: Je t’aime.", return_tensors="pt").to("cuda")
866
+ outputs = model.generate(inputs)
867
+ print(tokenizer.decode(outputs[0]))
868
+ ```
869
+
870
+ </details>
871
+
872
+ <!-- Necessary for whitespace -->
873
+ ###
874
+
875
+ # Limitations
876
+
877
+ **Prompt Engineering:** The performance may vary depending on the prompt. For BLOOMZ models, we recommend making it very clear when the input stops to avoid the model trying to continue it. For example, the prompt "*Translate to English: Je t'aime*" without the full stop (.) at the end, may result in the model trying to continue the French sentence. Better prompts are e.g. "*Translate to English: Je t'aime.*", "*Translate to English: Je t'aime. Translation:*" "*What is "Je t'aime." in English?*", where it is clear for the model when it should answer. Further, we recommend providing the model as much context as possible. For example, if you want it to answer in Telugu, then tell the model, e.g. "*Explain in a sentence in Telugu what is backpropagation in neural networks.*".
878
+
879
+ # Training
880
+
881
+ ## Model
882
+
883
+ - **Architecture:** Same as [bloom-560m](https://huggingface.co/bigscience/bloom-560m), also refer to the `config.json` file
884
+ - **Finetuning steps:** 1750
885
+ - **Finetuning tokens:** 3.67 billion
886
+ - **Finetuning layout:** 1x pipeline parallel, 1x tensor parallel, 1x data parallel
887
+ - **Precision:** float16
888
+
889
+ ## Hardware
890
+
891
+ - **CPUs:** AMD CPUs with 512GB memory per node
892
+ - **GPUs:** 64 A100 80GB GPUs with 8 GPUs per node (8 nodes) using NVLink 4 inter-gpu connects, 4 OmniPath links
893
+ - **Communication:** NCCL-communications network with a fully dedicated subnet
894
+
895
+ ## Software
896
+
897
+ - **Orchestration:** [Megatron-DeepSpeed](https://github.com/bigscience-workshop/Megatron-DeepSpeed)
898
+ - **Optimizer & parallelism:** [DeepSpeed](https://github.com/microsoft/DeepSpeed)
899
+ - **Neural networks:** [PyTorch](https://github.com/pytorch/pytorch) (pytorch-1.11 w/ CUDA-11.5)
900
+ - **FP16 if applicable:** [apex](https://github.com/NVIDIA/apex)
901
+
902
+ # Evaluation
903
+
904
+ We refer to Table 7 from our [paper](https://arxiv.org/abs/2211.01786) & [bigscience/evaluation-results](https://huggingface.co/datasets/bigscience/evaluation-results) for zero-shot results on unseen tasks. The sidebar reports zero-shot performance of the best prompt per dataset config.
905
+
906
+ # Citation
907
+ ```bibtex
908
+ @article{muennighoff2022crosslingual,
909
+ title={Crosslingual generalization through multitask finetuning},
910
+ author={Muennighoff, Niklas and Wang, Thomas and Sutawika, Lintang and Roberts, Adam and Biderman, Stella and Scao, Teven Le and Bari, M Saiful and Shen, Sheng and Yong, Zheng-Xin and Schoelkopf, Hailey and others},
911
+ journal={arXiv preprint arXiv:2211.01786},
912
+ year={2022}
913
+ }
914
+ ```
config.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "apply_residual_connection_post_layernorm": false,
3
+ "architectures": [
4
+ "BloomForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "attention_softmax_in_fp32": true,
8
+ "bias_dropout_fusion": true,
9
+ "bos_token_id": 1,
10
+ "dtype": "float32",
11
+ "eos_token_id": 2,
12
+ "hidden_dropout": 0.0,
13
+ "hidden_size": 1024,
14
+ "initializer_range": 0.02,
15
+ "layer_norm_epsilon": 1e-05,
16
+ "masked_softmax_fusion": true,
17
+ "model_type": "bloom",
18
+ "n_head": 16,
19
+ "n_inner": null,
20
+ "n_layer": 24,
21
+ "offset_alibi": 100,
22
+ "pad_token_id": 3,
23
+ "pretraining_tp": 1,
24
+ "seq_length": 2048,
25
+ "skip_bias_add": true,
26
+ "skip_bias_add_qkv": false,
27
+ "slow_but_exact": false,
28
+ "transformers_version": "4.57.6",
29
+ "unk_token_id": 0,
30
+ "use_cache": true,
31
+ "vocab_size": 250880
32
+ }
generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 1,
4
+ "eos_token_id": 2,
5
+ "pad_token_id": 3,
6
+ "transformers_version": "4.57.6"
7
+ }
onnx/model.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d934ee4baa1ce99845c8fcb17d6f9ac7f1ddd0c57ff3d01404d3fde0bdb681f4
3
+ size 706681
onnx/model.onnx_data ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f556f3d76c92a1dcad11e5bdc702d8337314b1515f7940c192d0690e51c578af
3
+ size 3264462848
onnx/model_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54dc92ed36108e7c95c6153a3ecfb2342f128b94a71c11fc0f4b108da61ec4b7
3
+ size 1378908293
onnx/model_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7648eb81c4e9fef7b51879c6fcd6fb4b8863eea0cb2a978c508711dc52c80041
3
+ size 2
onnx/model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c23716e68ab81e706a2aa59fd48784487cd774d5f4251c01a7388b51fa26f68c
3
+ size 818157926
onnx/model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54dc92ed36108e7c95c6153a3ecfb2342f128b94a71c11fc0f4b108da61ec4b7
3
+ size 1378908293
onnx/model_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90049959c28bcc5c7da634153e5169d19f7111ae7e1d9a1fbacb32f7051a6e53
3
+ size 829793311
onnx/model_quantized.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c23716e68ab81e706a2aa59fd48784487cd774d5f4251c01a7388b51fa26f68c
3
+ size 818157926
onnx/model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b70b932daf114fcf13a7eece82f2bf2366d23307c215a825b6b408d92baad7c
3
+ size 818157969
special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<pad>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d963066d6adae5034a1dc114c3ac444512de09928cf14ed4562ba94d9a440e66
3
+ size 21763085
tokenizer_config.json ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<unk>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<s>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "2": {
21
+ "content": "</s>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "3": {
29
+ "content": "<pad>",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ }
36
+ },
37
+ "bos_token": "<s>",
38
+ "clean_up_tokenization_spaces": false,
39
+ "eos_token": "</s>",
40
+ "extra_special_tokens": {},
41
+ "merges_file": null,
42
+ "model_max_length": 1000000000000000019884624838656,
43
+ "pad_token": "<pad>",
44
+ "padding_side": "left",
45
+ "tokenizer_class": "BloomTokenizer",
46
+ "unk_token": "<unk>",
47
+ "vocab_file": null
48
+ }