Update README.md

README.md

@@ -3,3 +3,62 @@ First release of ProkBERT
 ---
 license: cc-by-nc-4.0
 ---
+
+## ProkBERT-mini Model
+
+ProkBERT-mini-k6s1 is part of the ProkBERT family of genomic language models, specifically designed for microbiome applications. This model, optimized for DNA sequence analysis, employs a unique tokenization strategy to effectively capture and interpret complex genomic data.
+
+### Model Details
+
+**Developed by:** Neural Bioinformatics Research Group
+
+**Architecture:** ProkBERT-mini-k6s1 is based on the MegatronBert architecture, a variant of the BERT model optimized for large-scale training. The model employs a learnable relative key-value positional embedding, mapping input vectors into a 384-dimensional space.
+
+**Tokenizer:** The model uses a 6-mer tokenizer with a shift of 1 (k6s1), specifically designed to handle DNA sequences efficiently.
+
+**Parameters:**
+
+| Parameter            | Description                          |
+|----------------------|--------------------------------------|
+| Model Size           | 20.6 million parameters              |
+| Max. Context Size    | 1024 bp                              |
+| Training Data        | 206.65 billion nucleotides           |
+| Layers               | 6                                    |
+| Attention Heads      | 6                                    |
+
+### Intended Use
+
+**Intended Use Cases:** ProkBERT-mini-k6-s1 is intended for bioinformatics researchers and practitioners focusing on genomic sequence analysis, including:
+- sequence classification tasks
+- Exploration of genomic patterns and features
+
+**Out-of-scope Uses:** Not intended for use in non-genomic contexts or applications outside the realm of bioinformatics.
+
+### Training Data and Process
+
+**Overview:** The model was pretrained on a comprehensive dataset of genomic sequences to ensure broad coverage and robust learning.
+
+**Training Process:**
+- **Masked Language Modeling (MLM):** The MLM objective was modified for genomic sequences, involving an intricate strategy for masking overlapping k-mers.
+- **Training Phases:** The model underwent initial training with complete sequence restoration and selective masking, followed by a succeeding phase with variable-length datasets for increased complexity.
+
+### Evaluation Results
+
+| Metric                  | Result       | Notes |
+|-------------------------|--------------|-------|
+| Metric 1 (e.g., Accuracy) | To be filled |       |
+| Metric 2 (e.g., Precision) | To be filled |       |
+| Metric 3 (e.g., Recall)   | To be filled |       |
+
+*Additional details and metrics can be included as they become available.*
+
+### Ethical Considerations and Limitations
+
+As with all models in the bioinformatics domain, ProkBERT-mini-k6-s1 should be used responsibly. Testing and evaluation have been conducted within specific genomic contexts, and the model's outputs in other scenarios are not guaranteed. Users should exercise caution and perform additional testing as necessary for their specific use cases.
+
+### Reporting Issues
+
+Please report any issues with the model or its outputs to the Neural Bioinformatics Research Group through the following means:
+
+- **Model issues:** [GitHub repository link](https://github.com/nbrg-ppcu/prokbert)
+- **Feedback and inquiries:** [[email protected]](mailto:[email protected])