Guidance needed: creating a custom Modelfile for Ollama testing

#1
by MatteoBenvenuti - opened

Hi everyone,

I’m trying to use the FastwebMIIA‑7B model with Ollama for local testing. I performed the GGUF quantization myself, but I’m not sure how to structure a proper Modelfile and template for optimal results.

From the Hugging Face documentation, it seems possible to customize the prompt and parameters using:

FROM hf.co/{username}/{repo}:<quant>
TEMPLATE """
  {{ if .System }}<|system|>
  {{ .System }}<|end|>{{ end }}
  {{ if .Prompt }}<|user|>
  {{ .Prompt }}<|end|>{{ end }}
  <|assistant|>
  {{ .Response }}<|end|>
"""
SYSTEM """<your system prompt>"""
PARAMETER temperature 0.1
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1

But I’m missing the precise details:

  1. What’s the recommended TEMPLATE syntax for conversational use‑cases with this model?
  2. How should I include stop sequences (e.g. </s>) correctly?
  3. Are there additional PARAMETERS (like num_ctx, max_tokens) I should define?
  4. Is there an example Modelfile template available for this specific model?

Any guidance, examples or best practices would be really appreciated.
Thanks in advance

Sign up or log in to comment