Instructions to use kaivoss/AliceAI-T5-35B-A0.6B-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use kaivoss/AliceAI-T5-35B-A0.6B-chat with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("kaivoss/AliceAI-T5-35B-A0.6B-chat", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
AliceAI-T5-35B-A0.6B-chat (merged, experimental v1)
Status: experimental v1, known issues (updated 2026-10-01). Not recommended as an agent. Tested in an agent harness, this adapter over-uses tools (a bare "hi" produced 17 to 22 parallel tool calls), ignores "do not use tools" instructions, sometimes answers with nothing, can run on without stopping, and ends turns with
[COT_END]instead of[REPLY_END].Measured cause. The delimiter tokens this chat format is built on (
[INST_START],[REPLY_END],[SYSTEM_*],[TOOL_RESULT_*], ...) are untrained placeholders in the base model: their embeddings are identical to the never-used<SPAN#5000>([REPLY_END],[TOOL_RESULT_*]bit-identical, the rest within 0.004). They are therefore indistinguishable to the model, and LoRA on attention cannot teach them, so the model cannot tell roles apart or emit[REPLY_END]. Secondary cause: outside the Tool-Use subsplit about 98% of training turns are tool calls. A v2 without placeholder tokens is in progress. The evaluation below predates this finding and only measures format.
kaivoss/AliceAI-T5-35B-A0.6B-chat-lora merged into yandex/AliceAI-T5-35B-A0.6B (bf16). Usage, the chat template, the
training recipe and the evaluation are documented in kaivoss/AliceAI-T5-35B-A0.6B-chat-lora.
- Downloads last month
- 29
Model tree for kaivoss/AliceAI-T5-35B-A0.6B-chat
Base model
yandex/AliceAI-T5-35B-A0.6B