Hugging Face's logo

softhell
/
code_docstring_model

Transformers
Safetensors
t5
text2text-generation
Generated from Trainer
Eval Results
text-generation-inference
Model card Files
xet
Community
code_docstring_model / README.md
softhell's picture
softhell
codet5_train_2
e70b1bd verified
preview code
|
raw
history blame
3.11 kB
metadata
library_name: transformers
license: apache-2.0
base_model: Salesforce/codet5-small
tags:
  - generated_from_trainer
datasets:
  - code_search_net
metrics:
  - bleu
model-index:
  - name: code_docstring_model
    results:
      - task:
          name: Sequence-to-sequence Language Modeling
          type: text2text-generation
        dataset:
          name: code_search_net
          type: code_search_net
          config: python
          split: validation
          args: python
        metrics:
          - name: Bleu
            type: bleu
            value: 0.08377962983846503

code_docstring_model

This model is a fine-tuned version of Salesforce/codet5-small on the code_search_net dataset. It achieves the following results on the evaluation set:

  • Loss: 0.9435
  • Bleu: 0.0838

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 32
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 20
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Bleu
1.2873 0.9993 1076 1.1220 0.2248
1.1474 1.9993 2152 1.0673 0.2589
1.112 2.9993 3228 1.0406 0.2379
1.0849 3.9993 4304 1.0252 0.2432
1.0602 4.9993 5380 1.0126 0.1499
1.0565 5.9993 6456 1.0030 0.1413
1.0576 6.9993 7532 0.9964 0.0616
1.0174 7.9993 8608 0.9883 0.0935
1.0301 8.9993 9684 0.9828 0.0843
0.9884 9.9993 10760 0.9768 0.1363
0.9998 10.9993 11836 0.9702 0.1104
0.995 11.9993 12912 0.9647 0.1169
0.9852 12.9993 13988 0.9593 0.0962
0.9594 13.9993 15064 0.9549 0.0973
0.9553 14.9993 16140 0.9510 0.0923
0.9717 15.9993 17216 0.9486 0.0821
0.9667 16.9993 18292 0.9468 0.0888
0.957 17.9993 19368 0.9448 0.0858
0.9649 18.9993 20444 0.9438 0.0875
0.9454 19.9993 21520 0.9435 0.0838

Framework versions

  • Transformers 4.47.0
  • Pytorch 2.5.1+cu121
  • Datasets 3.3.1
  • Tokenizers 0.21.0