devngho
/

phi4-jamo-tokenizer

Model card Files Files and versions

devngho commited on Mar 22

Commit

ecbdfb8

·

verified ·

1 Parent(s): 0985611

Fix bos token

Files changed (1) hide show

tokenizer.json +2 -2

tokenizer.json CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6b24ca761478140c55718fb1021dfb002a1448dd10ac401caa40b8148c8697f4
-size 11318242

 version https://git-lfs.github.com/spec/v1
+oid sha256:b535a26b9b09cb661b5642a9ddffbb577d45cb1bf872e3039b975d6b67a16ed6
+size 11319120