trustkrem/frankenstein-qlora-mistral7b

Frankenstein QLoRA adapter for Mistral 7B

A LoRA adapter that teaches Mistral 7B v0.1 to write in the style of Mary Shelley's Frankenstein; or, The Modern Prometheus (1818). Intended for generating original Frankenstein-inspired fanfiction.

• Training notebook, data and full write-up: github.com/trustkrem/frankenstein-qlora-mistral7b
• Method: QLoRA (4-bit NF4 base model + LoRA adapters)
• Adapter size: ~55 MB (13.6M parameters, 0.19% of the base model)

Results

Model	Test perplexity ↓
Mistral 7B (base, 4-bit)	9.09
Mistral 7B + this adapter	6.83

Measured as mean per-chunk perplexity on 97 held-out text chunks not used in training.

Prompt: "I'm afraid I've created a " (greedy decoding)

• Base model: continues in modern blog voice, promoting a "Mindful Money Masterclass" course.
• With adapter:

  I'm afraid I've created a monster in putting life and sensation into dead matter. [...] Let me not be confounded with the wretch to whom I gave existence. I shall escape him, and find a place for repentance. Adieu!

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "mistralai/Mistral-7B-v0.1"
quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=quant_config, device_map="auto")
model = PeftModel.from_pretrained(base, "trustkrem/frankenstein-qlora-mistral7b")
tokenizer = AutoTokenizer.from_pretrained(base_id)

inputs = tokenizer("I'm afraid I've created a ", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Requires an NVIDIA GPU (bitsandbytes 4-bit). A free Colab T4 is enough. For more varied output, add do_sample=True, temperature=0.8.

Training details

Data: 481 text chunks of Frankenstein from Project Gutenberg (public domain), split 384 train / 97 test.

Quantization
Type	4-bit NF4, double quantization
Compute dtype	bfloat16

LoRA
Rank r / lora_alpha	32 / 64
Dropout	0.05
Target modules	q_proj, v_proj
Task type	CAUSAL_LM

Training
Epochs / steps	2 / 384
Batch size	2
Learning rate	2e-5, 2 warmup steps
Optimizer	paged_adamw_8bit
Precision	fp16 mixed precision
Hardware / time	Google Colab T4 (16 GB) / ~16.5 min
Training loss	~1.89 (first 20 steps) → ~1.66 (last 20 steps)

Limitations

• In-domain evaluation: the test chunks come from the same novel, so the perplexity gain reflects adaptation to this book, not general creative-writing quality.
• Pretraining overlap: Frankenstein was likely in Mistral's pretraining data.
• Formatting artifacts: outputs may contain hard line breaks (~70 characters), copied from the Gutenberg text layout.
• Plot accuracy: the model freely remixes characters and events. Fine for fanfiction; not suitable for summaries or factual questions about the novel.
• Inherits the biases and limitations of Mistral 7B, a base model with no safety tuning or moderation.

Acknowledgments

Built as a portfolio project for Codecademy's Finetuning Transformer Models course. Base model by Mistral AI (Apache 2.0). Source text: Mary Shelley, Frankenstein (1818), via Project Gutenberg.