library_name: transformers
base_model: google/pegasus-xsum
tags:
- summarization
- transformers
- fine-tuned
- google-pegasus-xsum
- ccdv/govreport-summarization
model-index:
- name: a-text-summarizer
results: []
language:
- en
a-text-summarizer
This model is a fine-tuned version of the google/pegasus-xsum model (https://huggingface.co/google/pegasus-xsum). It has been trained to generate summaries for governmental reports based on the GovReport summarization dataset (https://huggingface.co/datasets/ccdv/govreport-summarization). It achieves the following results on the evaluation set:
- Loss: 2.3989
Model description
This is a summarization model fine-tuned on the ccdv/govreport-summarization dataset.
Intended uses & limitations
This model is intended for generating concise summaries of governmental reports or similar long-form documents in an official or formal American English register.
The model's performance is limited by the data it was trained on (GovReport summarization dataset). It may not generalize well to other domains or types of text. Summarization models can sometimes hallucinate information or produce summaries that are not entirely accurate. Potential biases present in the training data may be reflected in the generated summaries. Further analysis is needed to identify and mitigate potential biases.
Training and evaluation data
The model was fine-tuned on a subset of the ccdv/govreport-summarization dataset. Specifically, a subset of 5000 training examples and 500 validation examples were used for fine-tuning.
The GovReport dataset contains governmental reports and their corresponding summaries.
Training procedure
The model was fine-tuned using the Hugging Face transformers library and Trainer API.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.7786 | 1.0 | 1250 | 2.4630 |
| 2.6139 | 2.0 | 2500 | 2.4117 |
| 2.5811 | 3.0 | 3750 | 2.3989 |
Framework versions
- Transformers 4.55.0
- Pytorch 2.6.0+cu124
- Datasets 4.0.0
- Tokenizers 0.21.4