Instructions to use zjunlp/OceanGPT-basic-7B-v0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zjunlp/OceanGPT-basic-7B-v0.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zjunlp/OceanGPT-basic-7B-v0.1")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zjunlp/OceanGPT-basic-7B-v0.1")
model = AutoModelForCausalLM.from_pretrained("zjunlp/OceanGPT-basic-7B-v0.1")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zjunlp/OceanGPT-basic-7B-v0.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zjunlp/OceanGPT-basic-7B-v0.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zjunlp/OceanGPT-basic-7B-v0.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/zjunlp/OceanGPT-basic-7B-v0.1

SGLang

How to use zjunlp/OceanGPT-basic-7B-v0.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zjunlp/OceanGPT-basic-7B-v0.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zjunlp/OceanGPT-basic-7B-v0.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zjunlp/OceanGPT-basic-7B-v0.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zjunlp/OceanGPT-basic-7B-v0.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use zjunlp/OceanGPT-basic-7B-v0.1 with Docker Model Runner:
```
docker model run hf.co/zjunlp/OceanGPT-basic-7B-v0.1
```

Ningyu commited on Aug 12, 2024

Commit

d196a84

verified ·

1 Parent(s): 196830f

Update README.md

Browse files

Files changed (1) hide show

README.md +3 -0

README.md CHANGED Viewed

@@ -30,6 +30,8 @@ datasets:
 OceanGPT-7b-v0.1 is based on LLaMA2 and has been trained on an English dataset in the ocean domain.
 ## ⏩Quickstart
 ### Download the model
@@ -101,6 +103,7 @@ OceanGPT(沧渊) is trained based on the open-sourced large language models incl
 - The model's output is influenced by prompt tokens, which may result in inconsistent results across multiple attempts.
 ### 🚩Citation

 OceanGPT-7b-v0.1 is based on LLaMA2 and has been trained on an English dataset in the ocean domain.
+- ❗**Disclaimer: This project is purely an academic exploration rather than a product. Please be aware that due to the inherent limitations of large language models, there may be issues such as hallucinations.**
 ## ⏩Quickstart
 ### Download the model
 - The model's output is influenced by prompt tokens, which may result in inconsistent results across multiple attempts.
+- The model requires the inclusion of specific simulator code instructions for training in order to possess simulated embodied intelligence capabilities (the simulator is subject to copyright restrictions and cannot be made available for now), and its current capabilities are quite limited.
 ### 🚩Citation