Model Card for `ai-ml-t-tes2-dftopcat-data-dsr1-1.5b`

This is a fine-tuned version of the facebook/opt-1.3b model using the LoRA (Low-Rank Adaptation) technique. The model has been trained on a dataset focused on Ayurveda and the concept of doshas (Vata, Pitta, Kapha). Compared to the previous model (ai-ml-t-tes1-dftopcat-data-dsr1-1.5b), this version uses a larger base model and improved training parameters to generate more coherent and informative responses about Ayurvedic principles and their role in promoting health.

Model Details

Model Description

This model is a fine-tuned adaptation of the facebook/opt-1.3b base model, optimized for generating explanations related to Ayurveda and doshas. It uses the LoRA technique to reduce computational costs while maintaining performance. The training data consists of instructional prompts and corresponding outputs that explain Ayurvedic concepts like doshic constitution, balance, and their influence on health.

Compared to the previous model (facebook/opt-350m), this version demonstrates significant improvements in coherence, reduced repetition, and fewer inaccuracies. However, it still struggles with depth and specificity, particularly in explaining Vata, Pitta, and Kapha doshas in detail.

Developed by: kas1
Model type: Causal Language Model (Fine-Tuned)
Language(s): English
License: MIT License
Finetuned from model: facebook/opt-1.3b

Model Sources

Repository: kas1/ai-ml-t-tes2-dftopcat-data-dsr1-1.5b
Dataset: Abhaykoul/Ancient-Indian-Wisdom

Uses

Direct Use

The model can be used to generate responses to questions about Ayurveda, particularly focusing on doshas and their role in health. It is suitable for educational purposes, answering FAQs, or providing introductory insights into Ayurvedic principles.

Downstream Use

The model can be integrated into applications like chatbots, virtual assistants, or educational platforms that focus on alternative medicine and wellness.

Out-of-Scope Use

The model is not designed for medical diagnosis, treatment recommendations, or generating content outside the scope of Ayurveda. Misuse or reliance on the model for critical health decisions is strongly discouraged.

Bias, Risks, and Limitations

Known Limitations

While the model shows improvements over the previous version, it still occasionally generates repetitive or nonsensical phrases.
Responses lack depth and specificity about Vata, Pitta, and Kapha doshas compared to expert-level explanations.
The model sometimes introduces inaccuracies (e.g., misinterpreting doshas as "disease-causing elements") due to limitations in training data or fine-tuning.

Improvements Over Previous Model

Reduced Repetition: Adjustments to generation parameters (e.g., repetition_penalty) have significantly reduced redundant phrases.
Improved Coherence: The use of a larger base model (facebook/opt-1.3b) has led to more structured and logical responses.
Fewer Inaccuracies: The model avoids major errors (e.g., "doshas as hallucinations") seen in the previous version.

Recommendations

Use post-processing techniques to filter out irrelevant or inaccurate statements.
Fine-tune the model further with more diverse and high-quality training data.
Experiment with even larger base models (e.g., facebook/opt-6.7b) for improved performance.

How to Get Started with the Model

To use this model, follow these steps:

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig
import torch

# Load the base model
base_model = AutoModelForCausalLM.from_pretrained(
    "facebook/opt-1.3b",  # Original base model
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load the LoRA configuration and adapter
peft_config = PeftConfig.from_pretrained("kas1/ai-ml-t-tes2-dftopcat-data-dsr1-1.5b")
model = PeftModel.from_pretrained(base_model, "kas1/ai-ml-t-tes2-dftopcat-data-dsr1-1.5b")

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained("kas1/ai-ml-t-tes2-dftopcat-data-dsr1-1.5b")
tokenizer.pad_token = tokenizer.eos_token

# Generate text
def generate_text(prompt, max_new_tokens=500):
    inputs = tokenizer(prompt, return_tensors="pt").to('cuda')
    with torch.no_grad():
        output = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=0.4,
            top_k=25,
            top_p=0.87,
            repetition_penalty=1.3
        )
    return tokenizer.decode(output[0], skip_special_tokens=True)

# Test the model
prompt = "Ayurveda emphasizes the balance between doshas. How can understanding our doshic constitution promote better health?"
output = generate_text(prompt)
print(output)

Model Card for ai-ml-t-tes2-dftopcat-data-dsr1-1.5b