gptr-noomo-0.3B

• EPOCHS: 1
• BATCH_SIZE: 7
• LR: 8e-5
• BLOCK_SZ: 512
• GPU: NVIDIA A100-SXM4-80GB
• Time: 40(hours)

How to use:

    import torch
    from transformers import GPT2TokenizerFast
    from modeling_gptr import GPTRForCausalLM

    model = GPTRForCausalLM.from_pretrained("aitetic/gptr-noomo-0.3b", local_files_only=False)
    tokenizer = GPT2TokenizerFast.from_pretrained(f"aitetic/gptr-noomo-0.3b", local_files_only=False)

    prompt = "The future of AI is"
    input_ids = tokenizer(prompt, return_tensors="pt")

    generated_ids = model.generate(
        input_ids,
        max_new_tokens=120,
        do_sample=False,
        eos_token_id=tokenizer.eos_token_id
    )

    output_ids = generated_ids[0].detach().cpu().tolist()
    print("Generated:", tokenizer.decode(output_ids, skip_special_tokens=True))