Instructions to use AGBonnet/medinote-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AGBonnet/medinote-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AGBonnet/medinote-7b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AGBonnet/medinote-7b")
model = AutoModelForCausalLM.from_pretrained("AGBonnet/medinote-7b")

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use AGBonnet/medinote-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AGBonnet/medinote-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AGBonnet/medinote-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/AGBonnet/medinote-7b

SGLang

How to use AGBonnet/medinote-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AGBonnet/medinote-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AGBonnet/medinote-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AGBonnet/medinote-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AGBonnet/medinote-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use AGBonnet/medinote-7b with Docker Model Runner:
```
docker model run hf.co/AGBonnet/medinote-7b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Card for MediNote-7B-v1.0

MediNote is a suite of open-source medical Large Language Models (LLMs) fine-tuned for clinical note generation from the Meditron foundation model. MediNote-7B is a 7 billion parameters model trained to generate clinical notes from doctor-patient conversations.

Model Details

Developed by: Antoine Bonnet and Paul Boulenger
Model type: Causal decoder-only transformer language model
Language(s): English only
Model License: LLAMA 2 COMMUNITY LICENSE AGREEMENT
Code License: MIT
Fine-tuned from model: Meditron-7B.v1.0
Context length: 2K tokens
Input: Patient-doctor conversation transcripts (text)
Output: Clinical notes (text)
Repository: EPFL-IC-Make-Team/ClinicalNotes
Trainer: epflLLM/Megatron-LLM
Report: MediNote: Automatic Clinical Notes

Model pipeline

Uses

Direct Use

It is possible to use this model to generate clinical notes, which is useful for experimentation and understanding its capabilities. It should not be used directly for production or work that may impact people.

Out-of-Scope Use

This model is not yet robust enough for use in a real clinical setting. We do not recommend using this model for natural language generation in a production environment.