Guidance needed: creating a custom Modelfile for Ollama testing
#1
by
MatteoBenvenuti
- opened
Hi everyone,
I’m trying to use the FastwebMIIA‑7B model with Ollama for local testing. I performed the GGUF quantization myself, but I’m not sure how to structure a proper Modelfile and template for optimal results.
From the Hugging Face documentation, it seems possible to customize the prompt and parameters using:
FROM hf.co/{username}/{repo}:<quant>
TEMPLATE """
{{ if .System }}<|system|>
{{ .System }}<|end|>{{ end }}
{{ if .Prompt }}<|user|>
{{ .Prompt }}<|end|>{{ end }}
<|assistant|>
{{ .Response }}<|end|>
"""
SYSTEM """<your system prompt>"""
PARAMETER temperature 0.1
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
But I’m missing the precise details:
- What’s the recommended TEMPLATE syntax for conversational use‑cases with this model?
- How should I include stop sequences (e.g.
</s>
) correctly? - Are there additional PARAMETERS (like
num_ctx
,max_tokens
) I should define? - Is there an example Modelfile template available for this specific model?
Any guidance, examples or best practices would be really appreciated.
Thanks in advance