a-text-summarizer

This model is a fine-tuned version of the google/pegasus-xsum model (https://huggingface.co/google/pegasus-xsum). It has been trained to generate summaries for governmental reports based on the GovReport summarization dataset (https://huggingface.co/datasets/ccdv/govreport-summarization). It achieves the following results on the evaluation set:

• Loss: 2.3989

Model description

This is a summarization model fine-tuned on the ccdv/govreport-summarization dataset.

Intended uses & limitations

This model is intended for generating concise summaries of governmental reports or similar long-form documents in an official or formal American English register.

The model's performance is limited by the data it was trained on (GovReport summarization dataset). It may not generalize well to other domains or types of text. Summarization models can sometimes hallucinate information or produce summaries that are not entirely accurate. Potential biases present in the training data may be reflected in the generated summaries. Further analysis is needed to identify and mitigate potential biases.

Training and evaluation data

The model was fine-tuned on a subset of the ccdv/govreport-summarization dataset. Specifically, a subset of 5000 training examples and 500 validation examples were used for fine-tuning.

The GovReport dataset contains governmental reports and their corresponding summaries.

Training procedure

The model was fine-tuned using the Hugging Face transformers library and Trainer API.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
2.7786	1.0	1250	2.4630
2.6139	2.0	2500	2.4117
2.5811	3.0	3750	2.3989

Framework versions

• Transformers 4.55.0
• Pytorch 2.6.0+cu124
• Datasets 4.0.0
• Tokenizers 0.21.4