Spaces:

Duplicated from Xenova/the-tokenizer-playground

Jhayase
/

superbpe-tokenizer-playground

Running

App Files Files Community

superbpe-tokenizer-playground / assets /index-DEbmRw68.js

Commit History

Update tokenizers

cafb8f5

Jhayase commited on Mar 21

Updates for SuperBPE website embed

98c69b7

Jhayase commited on Mar 19

Prettify

65ddd4a

Jhayase commited on Mar 19

Update title

124de93

Jhayase commited on Mar 19

Add reference tokenizers

b7055d5

Jhayase commited on Mar 19

Set default to SuperBPE t180k

ee8ca32

Jhayase commited on Mar 19

Add SuperBPE tokenizers

a8729e9

Jhayase commited on Mar 19

Upload 4 files

2cb5f4f
verified

Xenova HF Staff commited on Oct 31, 2024