leafspark
/

claude-3-tokenizer

Model card Files Files and versions Community

claude-3-tokenizer

Ctrl+K

Ctrl+K

1 contributor

History: 5 commits

leafspark's picture

readme: add model card

6c0c4b1 verified 12 months ago

.gitattributes

1.6 kB

model: add test files and support command line arguments 12 months ago
README.md

3.33 kB

readme: add model card 12 months ago
golden_ratio_1_million_digits.txt

1 MB

model: add test files and support command line arguments 12 months ago
golden_ratio_1_million_digits.txt.tokens

16.8 MB
LFS

model: add test files and support command line arguments 12 months ago
groups_merged-enhancedV3.txt

280 kB

model: add test files and support command line arguments 12 months ago
groups_merged-enhancedV3.txt.tokens

3.71 MB

model: add test files and support command line arguments 12 months ago
prompt_test.txt

648 Bytes

model: add test files and support command line arguments 12 months ago
prompt_test.txt.tokens

7.87 kB

model: add test files and support command line arguments 12 months ago
tokenize.py

5.88 kB

model: add test files and support command line arguments 12 months ago
tokenizer_config.json

1.05 MB

add tokenizer and script to run 12 months ago