Hugging Face's logo

NetoAISolutions
/
TSLAM-Mini-2B

Safetensors
phi3
custom_code
Eval Results
4-bit precision
bitsandbytes
Model card Files Community
1
TSLAM-Mini-2B / README.md
vigneshe's picture
vigneshe
Update README.md
0d31e28 verified
preview code
|
raw
history blame contribute delete
13 kB
metadata
model-index:
  - name: TSLAM-Mini-2B
    results:
      - task:
          type: domain-modeling
        dataset:
          type: telecom-eval-suite
          name: Telecom Internal Benchmark
        metrics:
          - type: accuracy
            value: 88.2
      - task:
          type: open-domain-qa
        dataset:
          type: bigbench-hard
          name: BIG-Bench Hard
        metrics:
          - type: accuracy
            value: 70.4
      - task:
          type: question-answering
        dataset:
          type: mmlu
          name: MMLU
        metrics:
          - type: accuracy
            value: 67.3
      - task:
          type: commonsense-reasoning
        dataset:
          type: arc
          name: ARC Challenge
        metrics:
          - type: accuracy
            value: 83.7
      - task:
          type: boolean-classification
        dataset:
          type: boolq
          name: BoolQ
        metrics:
          - type: accuracy
            value: 81.2
      - task:
          type: question-answering
        dataset:
          type: gpqa
          name: GPQA
        metrics:
          - type: accuracy
            value: 25.2
      - task:
          type: commonsense-reasoning
        dataset:
          type: hellaswag
          name: HellaSwag
        metrics:
          - type: accuracy
            value: 69.1
      - task:
          type: open-book-qa
        dataset:
          type: openbookqa
          name: OpenBookQA
        metrics:
          - type: accuracy
            value: 79.2
      - task:
          type: physical-reasoning
        dataset:
          type: piqa
          name: PIQA
        metrics:
          - type: accuracy
            value: 77.6
      - task:
          type: social-intelligence
        dataset:
          type: socialiqa
          name: Social IQa
        metrics:
          - type: accuracy
            value: 72.5
      - task:
          type: truthfulness
        dataset:
          type: truthfulqa
          name: TruthfulQA
        metrics:
          - type: accuracy
            value: 66.4
      - task:
          type: winograd-schema
        dataset:
          type: winogrande
          name: WinoGrande
        metrics:
          - type: accuracy
            value: 67
      - task:
          type: question-answering
        dataset:
          type: mmlu
          name: MMLU (Multilingual)
        metrics:
          - type: accuracy
            value: 49.3
      - task:
          type: mathematical-reasoning
        dataset:
          type: mgsm
          name: MGSM
        metrics:
          - type: accuracy
            value: 63.9
      - task:
          type: mathematical-reasoning
        dataset:
          type: gsm8k
          name: GSM8K
        metrics:
          - type: accuracy
            value: 88.6
      - task:
          type: mathematical-reasoning
        dataset:
          type: math
          name: MATH
        metrics:
          - type: accuracy
            value: 64
extra_gated_prompt: Please provide answers to the below questions to gain access to the model
extra_gated_fields:
  Company: text
  Full Name: text
  Email: text
  I want to use this model for:
    type: select
    options:
      - Research
      - Education
      - Commercial
      - label: Other
        value: other

TSLAM-Mini-2B

Base Model: microsoft/Phi-4-mini-instruct
License: MIT

Overview

TSLAM-Mini-2B is a domain-adapted language model fine-tuned on 100,000 telecom-specific examples, designed to emulate the intelligence and conversational expertise of a Telecom Subject Matter Expert (SME). Built on top of the Phi-4-mini-instruct foundation, TSLAM-Mini-2B is optimized for real-time, industry-grade interactions across key telecom scenarios, including:

  • SME-style responses in customer support and internal queries
  • Network configuration, diagnostics, and troubleshooting workflows
  • Device provisioning and service activation dialogues
  • Operational support for field and NOC teams
  • Intelligent retrieval and summarization of telecom-specific documentation

This fine-tuning strategy enables TSLAM-Mini-2B to reason like an SME, offering accurate, context-aware responses that align with real-world telecom operations. Though this model offers superior performance on Telecom specific usecases, For enterprises requiring specialized capabilities please contact us [email protected] for our enterprise grade commercial models which offers greater capabilities required for production.

Key Features

  • Telecom-Tuned: Finetuned on domain-specific conversations, logs, and structured dialogues.
  • Instruction-Following: Retains Phi-4’s compact instruction-tuned behavior while adapting to industry-specific patterns.
  • Real-Time Scenarios: Performs well in use cases that require contextual understanding of real-world telecom operations.

Intended Use

The areas TSLAM-Mini-2B excels in are:

  • Customer Support Agents (AI copilots or chatbots)
  • Network Operations Tools that process commands or log queries
  • Internal Assistants for engineers and field technicians
  • Telecom Knowledge Graphs & RAG Pipelines

Model Details

Property Value
Base Model microsoft/Phi-4-mini-instruct
Fine-tuning Data 100k telecom domain examples
Training Method Supervised fine-tuning (SFT)
License MIT

Benchmarks Results

Benchmark TSLAM-Mini-2B Phi-3.5-mini-Ins Llama-3.2-3B-Ins Mistral-3B Qwen2.5-3B-Ins Qwen2.5-7B-Ins Mistral-8B-2410 Llama-3.1-8B-Ins Llama-3.1-Tulu-3-8B Gemma2-9B-Ins GPT-4o-mini-2024-07-18
Popular aggregated benchmark
Arena Hard 32.8 34.4 17.0 26.9 32.0 55.5 37.3 25.7 42.7 43.7 53.7
BigBench Hard (0-shot, CoT) 70.4 63.1 55.4 51.2 56.2 72.4 53.3 63.4 55.5 65.7 80.4
MMLU (5-shot) 67.3 65.5 61.8 60.8 65.0 72.6 63.0 68.1 65.0 71.3 77.2
MMLU-Pro (0-shot, CoT) 52.8 47.4 39.2 35.3 44.7 56.2 36.6 44.0 40.9 50.1 62.8
Reasoning
ARC Challenge (10-shot) 83.7 84.6 76.1 80.3 82.6 90.1 82.7 83.1 79.4 89.8 93.5
BoolQ (2-shot) 81.2 77.7 71.4 79.4 65.4 80.0 80.5 82.8 79.3 85.7 88.7
GPQA (0-shot, CoT) 25.2 26.6 24.3 24.4 23.4 30.6 26.3 26.3 29.9 39.1 41.1
HellaSwag (5-shot) 69.1 72.2 77.2 74.6 74.6 80.0 73.5 72.8 80.9 87.1 88.7
OpenBookQA (10-shot) 79.2 81.2 72.6 79.8 79.3 82.6 80.2 84.8 79.8 90.0 90.0
PIQA (5-shot) 77.6 78.2 68.2 73.2 72.6 76.2 81.2 83.2 78.3 83.7 88.7
Social IQA (5-shot) 72.5 75.1 68.3 73.9 75.3 75.3 77.6 71.8 73.4 74.7 82.9
TruthfulQA (MC2) (10-shot) 66.4 65.2 59.2 62.9 64.3 69.4 63.0 69.2 64.1 76.6 78.2
Winogrande (5-shot) 67.0 72.2 53.2 59.8 63.3 71.1 63.1 64.7 65.4 74.0 76.9
Multilingual
Multilingual MMLU (5-shot) 49.3 51.8 48.1 46.4 55.9 64.4 53.7 56.2 54.5 63.8 72.9
MGSM (0-shot, CoT) 63.9 49.6 44.6 44.6 53.5 64.5 56.7 56.7 58.6 75.1 81.7
Math
GSM8K (8-shot, CoT) 88.6 76.9 75.6 80.1 80.6 88.7 81.9 82.4 84.3 84.9 91.3
MATH (0-shot, CoT) 64.0 49.8 46.7 41.8 61.7 60.4 41.6 47.6 46.1 51.3 70.2
Telecom (domain-specific) 88.2 52.1 47.6 49.3 58.0 61.5 54.9 57.3 59.0 64.1 70.3
Overall 63.5 60.5 56.2 56.9 60.1 67.9 60.2 62.3 60.9 65.0 75.5

Example 

**User**: How do I reconfigure a 5G core node remotely?

**Model**: To reconfigure a 5G core node remotely, ensure you have SSH access enabled and the necessary configuration scripts preloaded. From your NOC terminal, run the secure update command with the node's IP and authentication key...

How to Use 

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load tokenizer and model
model_name = "NetoAISolutions/TSLAM-Mini-2B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Define the input using the Phi-style chat template
def build_prompt(user_input):
    system = "<|system|>\nYou are a helpful assistant.<|end|>\n"
    user = f"<|user|>\n{user_input}<|end|>\n"
    assistant = "<|assistant|>\n"  # Start assistant response
    return system + user + assistant

# Example input
user_query = "How do I activate VoLTE on a user's device?"
prompt = build_prompt(user_query)

# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    eos_token_id=tokenizer.convert_tokens_to_ids("<|end|>")
)

# Decode output
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Acknowledgements