Merge pull request #92 from OpenAccess-AI-Collective/flash-optimum

chore: Refactor inf_kwargs out

Merge branch 'main' into flash-optimum

Merge pull request #177 from NanoCode012/fix/landmark-patch

Merge pull request #159 from AngainorDev/patch-1

Fix strict and Lint

Fix set mem_id for inference and refactor

Set mem cache args on inference

fix formatting

pass a prompt in from stdin for inference

Update scripts/finetune.py

address PR feedback

add streaming dataset support for pretraining datasets

more tweaks to do pre-training with bettertransformers

experimental expansion of ctx len

add flash attn context for efficient training and attempt setting model to train mode:

add support for opimum bettertransformers

Merge branch 'main' into patch-1

Remove explicit definition of cfg.inference

formatting for linter

Add streaming inference & fix stopping at EOS

Feed cfg.inference

Set matmul tf32

new prompters, misc fixes for output dir missing using fsdp, and changing max seq len

fix device map

fix batch size calculation

Merge pull request #119 from NanoCode012/feat/update-inference

Increase max_new_tokens

Merge pull request #120 from OpenAccess-AI-Collective/model-from-path

black formatting

add support for gradient accumulation steps

fix up tokenizer config, isort fix

Feat: Swap to GenerationConfig

Merge pull request #108 from OpenAccess-AI-Collective/docker-gptq

Fix security issue or ignore false positives

Apply isort then black

Delete extract_lora.py

Fix mypy typing

Lint finetune.py

Lint and format

default to qlora support, make gptq specific image

bnb fixes

refactor: fix previous refactors

Refactor to use DictDefault instead

Fix load error

Convert attrdict to addict

move list not in list logic to fn

load the tokenizer seperately from the model

add logging and make sure model unloads to float16

remove un-needed code, add validation