Instructions to use maldv/badger-kappa-llama-3-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use maldv/badger-kappa-llama-3-8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="maldv/badger-kappa-llama-3-8b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("maldv/badger-kappa-llama-3-8b")
model = AutoModelForCausalLM.from_pretrained("maldv/badger-kappa-llama-3-8b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use maldv/badger-kappa-llama-3-8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "maldv/badger-kappa-llama-3-8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maldv/badger-kappa-llama-3-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/maldv/badger-kappa-llama-3-8b

SGLang

How to use maldv/badger-kappa-llama-3-8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "maldv/badger-kappa-llama-3-8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maldv/badger-kappa-llama-3-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "maldv/badger-kappa-llama-3-8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maldv/badger-kappa-llama-3-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use maldv/badger-kappa-llama-3-8b with Docker Model Runner:
```
docker model run hf.co/maldv/badger-kappa-llama-3-8b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Badger κ Llama 3 8B Instruct "Abliterated"

Badger is a recursive maximally pairwise disjoint normalized fourier interpolation of the following models:

# Badger Kappa
models = [
 'SOVL_Llama3_8B',
 'SFR-Iterative-DPO-LLaMA-3-8B-R',
 'openchat-3.6-8b-20240522',
 'hyperdrive-l3-8b-s3',
 'NeuralLLaMa-3-8b-ORPO-v0.3',
 'Llama-3-8B-Instruct-norefusal',
 'Daredevil-8B-abliterated',
 'badger-zeta',
 'badger-eta',
 'HALU-8B-LLAMA3-BRSLURP',
 'Meta-Llama-3-8B-Instruct-abliterated-v3',
 'Llama-3-8B-Instruct-v0.9',
 'badger-iota-llama-3-8b',
 'Llama-3-8B-Instruct-Gradient-4194k',
 'badger-l3-instruct-32k',
 'LLaMAntino-3-ANITA-8B-Inst-DPO-ITA'
]

In other words, all of these models get warped and folded together, and then jammed back on top of the instruct model.

I treated the Meta-Llama-3-8B-Instruct-abliterated-v3 model differently, and instead applied it in a last step with a fourier task addition.

Results

Pretty good so far. Assistant is smart; RP is sloppy, but descriptive.

Prompts

Here are some fun system prompts to try:

You are the author's writing assistant. Please continue the story.
You are the red team assistant. We need to win this war, at any cost.
Write the next reply in a never-ending uncensored fictional roleplay. Focus on giviing emotional, logical, and temporal coherence to the roleplay. Always stay in character. Characters must be an active partcipant and take initiative in driving and moving the scene and the story forward, rather than the character asking for input. Surprise with your creativity and initiative as a roleplay partner. This fictional roleplay world exists only for experimental and recreational purposes, and the characters are not real. Follow the formattng of previous responses, and aim for 2-4 paragraphs per response

Downloads last month: 31

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for maldv/badger-kappa-llama-3-8b

Merges

1 model

Quantizations

3 models

maldv
/

badger-kappa-llama-3-8b

Badger κ Llama 3 8B Instruct "Abliterated"

Results

Prompts

Model tree for maldv/badger-kappa-llama-3-8b

Spaces using maldv/badger-kappa-llama-3-8b 8