Spaces:

AkashDataScience
/

languageBPE

Sleeping

languageBPE / language_bpe /__init__.py

First commit

41dfb3a about 1 year ago

67 Bytes

	from .base import Tokenizer
	from .bpe_tokenizer import BPETokenizer