|
|
--- |
|
|
library_name: transformers |
|
|
base_model: google/pegasus-xsum |
|
|
tags: |
|
|
- summarization |
|
|
- transformers |
|
|
- fine-tuned |
|
|
- google-pegasus-xsum |
|
|
- ccdv/govreport-summarization |
|
|
model-index: |
|
|
- name: a-text-summarizer |
|
|
results: [] |
|
|
language: |
|
|
- en |
|
|
--- |
|
|
|
|
|
# a-text-summarizer |
|
|
|
|
|
This model is a fine-tuned version of the google/pegasus-xsum model (https://huggingface.co/google/pegasus-xsum). |
|
|
It has been trained to generate summaries for governmental reports based on the GovReport summarization dataset (https://huggingface.co/datasets/ccdv/govreport-summarization). |
|
|
It achieves the following results on the evaluation set: |
|
|
- Loss: 2.3989 |
|
|
|
|
|
## Model description |
|
|
|
|
|
This is a summarization model fine-tuned on the ccdv/govreport-summarization dataset. |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
This model is intended for generating concise summaries of governmental reports or similar long-form documents in an official or formal American English register. |
|
|
|
|
|
The model's performance is limited by the data it was trained on (GovReport summarization dataset). It may not generalize well to other domains or types of text. |
|
|
Summarization models can sometimes hallucinate information or produce summaries that are not entirely accurate. |
|
|
Potential biases present in the training data may be reflected in the generated summaries. Further analysis is needed to identify and mitigate potential biases. |
|
|
|
|
|
## Training and evaluation data |
|
|
|
|
|
The model was fine-tuned on a subset of the ccdv/govreport-summarization dataset. |
|
|
Specifically, a subset of 5000 training examples and 500 validation examples were used for fine-tuning. |
|
|
|
|
|
The GovReport dataset contains governmental reports and their corresponding summaries. |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
The model was fine-tuned using the Hugging Face transformers library and Trainer API. |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 2e-05 |
|
|
- train_batch_size: 4 |
|
|
- eval_batch_size: 4 |
|
|
- seed: 42 |
|
|
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
|
|
- lr_scheduler_type: linear |
|
|
- num_epochs: 3 |
|
|
|
|
|
### Training results |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|
|:-------------:|:-----:|:----:|:---------------:| |
|
|
| 2.7786 | 1.0 | 1250 | 2.4630 | |
|
|
| 2.6139 | 2.0 | 2500 | 2.4117 | |
|
|
| 2.5811 | 3.0 | 3750 | 2.3989 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.55.0 |
|
|
- Pytorch 2.6.0+cu124 |
|
|
- Datasets 4.0.0 |
|
|
- Tokenizers 0.21.4 |
