Update README.md

README.md

@@ -1,195 +1,107 @@
-# Azure Sky Optimizer
-
-Azure Sky Optimizer is a hybrid optimizer for PyTorch, integrating Simulated Annealing (SA) with Adam to provide robust exploration and precise exploitation in non-convex optimization tasks. Designed for complex machine learning challenges, Azure Sky excels in domains requiring deep exploration of rugged loss landscapes, such as scientific machine learning, symbolic reasoning, and protein folding.
-
-Developed as part of an R&D initiative, Azure Sky combines structured stochastic exploration with gradient-based refinement, achieving stable convergence and strong generalization in multi-modal search spaces.
-
----
-
-## Overview
-
-Conventional optimizers like Adam and AdamW often converge prematurely to sharp local minima, compromising generalization. Azure Sky leverages SA’s global search in early stages and Adam’s local convergence later, ensuring both deep exploration and precise convergence.
-
-### Core Innovations
-
-- **Dynamic Temperature Scaling:** Adjusts SA temperature based on training progress for controlled exploration.
-- **Exploration-Exploitation Fusion:** Seamlessly transitions between SA and Adam using a sigmoid-based blending mechanism.
-- **Stability Enhancements:** Built-in gradient clipping and loss spike monitoring for robust training.
-
----
-
-## Key Features
-
-- **Hybrid Optimization:** Combines SA’s global search with Adam’s local refinement.
-- **Optimized Hyperparameters:** Tuned via Optuna (the best trial: 0.0893 on Two Moons dataset).
-- **Flexible Parameter Handling:** Supports parameter lists, named parameters, and parameter groups with group-specific learning rates.
-- **Production-Ready Stability:** Includes gradient clipping and loss spike detection.
-- **PyTorch Compatibility:** Fully integrated with PyTorch’s `optim` module.
-
----
-
-## Installation
-
-Clone the repository and install using [uv](https://github.com/astral-sh/uv):
-
-```bash
-git clone https://github.com/yourusername/azure-sky-optimizer.git
-cd azure-sky-optimizer
-uv pip install -e .
-```
-
-**Requirements:**
-- Python >= 3.8
-- PyTorch >= 1.10.0
-- NumPy >= 1.20.0
-
-> **Note:** Ensure `uv` is installed. See [uv documentation](https://github.com/astral-sh/uv) for instructions.
-
----
-
-## Usage Examples
-
-Azure Sky integrates seamlessly into PyTorch workflows. Below are usage examples for various parameter configurations.
-
-### Basic Usage
-
-```python
-import torch
-import torch.nn as nn
-from azure_optimizer import Azure
-
-model = nn.Linear(10, 2)
-criterion = nn.CrossEntropyLoss()
-optimizer = Azure(model.parameters())
-
-inputs = torch.randn(32, 10)
-targets = torch.randint(0, 2, (32,))
-optimizer.zero_grad()
-outputs = model(inputs)
-loss = criterion(outputs, targets)
-loss.backward()
-optimizer.step()
-```
-
-### Parameter Lists
-
-```python
-var1 = torch.nn.Parameter(torch.randn(2, 2))
-var2 = torch.nn.Parameter(torch.randn(2, 2))
-optimizer = Azure([var1, var2])
-```
-
-### Parameter Groups with Custom Learning Rates
-
-```python
-class SimpleModel(nn.Module):
-    def __init__(self):
-        super().__init__()
-        self.base = nn.Linear(10, 5)
-        self.classifier = nn.Linear(5, 2)
-
-    def forward(self, x):
-        x = torch.relu(self.base(x))
-        return self.classifier(x)
-
-model = SimpleModel()
-optimizer = Azure([
-    {'params': model.base.parameters(), 'lr': 1e-2},
-    {'params': model.classifier.parameters()}
-])
-```
-
-For additional examples, see `azure_optimizer/usage_example.py`.
-
----
-
-## Hyperparameters
-
-Default hyperparameters (from Optuna Trial 99, the best validation loss: 0.0893 on Two Moons):
-
-| Parameter   | Value                 | Description                  |
-|-------------|-----------------------|------------------------------|
-| lr          | 0.0007518383921113902 | Learning rate for Adam phase |
-| T0          | 2.2723218904585964    | Initial temperature for SA   |
-| sigma       | 0.17181058166567398   | Perturbation strength for SA |
-| SA_steps    | 5                     | Steps for SA phase           |
-| sa_momentum | 0.6612913488540948    | Momentum for SA updates      |
-
----
-
-## Performance
-
-Evaluated on the Two Moons dataset (5000 samples, 20% noise):
-
-- **Best Validation Loss:** 0.0919
-- **Final Validation Accuracy:** 96.7%
-- **Epochs to Convergence:** 50
-
-Compared to:
-- **Adam:** loss 0.0927, acc 96.8%
-- **AdamW:** loss 0.0917, acc 97.1%
-
-Azure Sky prioritizes robust generalization over rapid convergence, making it ideal for pre-training and tasks requiring deep exploration.
-
----
-
-## Contributing
-
-Contributions are welcome!
-
-1. Fork the repository.
-2. Create a feature branch: `git checkout -b feature/your-feature`
-3. Commit your changes.
-4. Push to your branch.
-5. Open a pull request.
-
-Please follow PEP 8 standards. Tests are not yet implemented; contributions to add testing infrastructure are highly encouraged.
-
----
-
-## License
-
-This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
-
----
-
-## Citation
-
-If you use Azure Sky Optimizer in your research or engineering projects, please cite:
-
-```
-[Allan]. (2025). Azure Sky Optimizer: A Hybrid Approach for Exploration and Exploitation. GitHub Repository.
-```
-
----
-
-## Project Status
-
-As of May 27, 2025, Azure Sky Optimizer is stable and production-ready.
-
-**Planned improvements:**
-- Testing on larger datasets (e.g., CIFAR-10, MNIST)
-- Ablation studies for hyperparameter impact
-- Integration with PyTorch Lightning
-- Adding a comprehensive test suite
-
-For questions or collaboration, please open an issue on GitHub.
-
-Kaggle Notebook: https://www.kaggle.com/code/allanwandia/non-convex-research
-
-Writeup It has old metrics so watch out: https://github.com/DarkStarStrix/CSE-Repo-of-Advanced-Computation-ML-and-Systems-Engineering/blob/main/Papers/Computer_Science/Optimization/Optimization_Algothrims_The_HimmelBlau_Function_Case_Study.pdf
-
----
-
-## Repository Structure
-
-```
-azure-sky-optimizer/
-├── azure_optimizer/
-│   ├── __init__.py
-│   ├── azure.py        # Updated Azure class
-│   ├── hooks.py
-│   └── usage_example.py  # Usage demonstrations
-├── README.md
-└── LICENSE
-```
+---
+title: Nexa R&D
+emoji: 🔬
+colourFrom: blue
+colorTo: green
+sdk: gradio
+sdk_version: 4.44.0
+app_file: App.py
+pinned: false
+license: apache-2.0
+tags:
+- optimization
+- machine-learning
+- research-tool
+- gradio
+- azure-sky
+---
+
+# Nexa R&D
+
+Nexa R&D is a visual research platform designed for researchers and industry professionals to compare and evaluate optimisers (e.g., AzureSky, Adam, SGD, AdamW, RMSprop) on analytical benchmark functions (e.g., Himmelblau, Ackley) and machine learning tasks (e.g., MNIST, CIFAR-10). It supports ablation studies, hyperparameter tuning, and side-by-side evaluations through an intuitive Gradio-based interface, optimised for deployment on Hugging Face Spaces.
+
+## Features
+
+- **Modes**:
+  - **Benchmark Optimisation**: Visualise optimiser trajectories on loss surfaces of mathematical functions.
+  - **ML Task Training**: Train and compare optimisers on datasets like MNIST and CIFAR-10.
+- **Optimisers**: AzureSky (hybrid SA + Adam), Adam, AdamW, SGD, RMSprop.
+- **Ablation Suite**: Configure AzureSky’s Simulated Annealing (SA) with options to enable/disable SA, set initial temperature, and adjust cooling rate.
+- **Interactive UI**: Gradio interface with plots, metrics tables, and JSON export for results.
+- **Metrics**:
+  - **Benchmark Mode**: Distance to global minimum, final loss, convergence rate.
+  - **ML Mode**: Train/validation accuracy, generalisation gap, final loss, best epoch.
+- **Deployment**: Optimised for Hugging Face Spaces with optional GPU acceleration.
+
+
+# Usage
+
+Configure a Study:
+
+Select Mode: Choose "Benchmark Optimisation" or "ML Task Training" from the Study Configuration tab.
+Select Optimisers: Pick one or more optimisers (e.g., AzureSky, Adam).
+Configure Parameters:
+Benchmark Mode: Select a function (e.g., Himmelblau) and dimensionality (default: 2).
+ML Task Mode: Select a dataset (e.g., MNIST), epochs (default: 10), batch size (default: 32), and learning rate (default: 0.001).
+
+
+Ablation Settings (if AzureSky is selected):
+Enable/disable simulated annealing (default: enabled).
+Set initial SA temperature (default: 1.0).
+Set SA cooling rate (default: 0.95).
+
+
+Run Study:
+
+Click "Run Study" to execute the experiment.
+View results in the "Results" tab, including:
+Plots (loss surfaces for benchmarks or accuracy/loss curves for ML tasks).
+Metrics table summarising performance.
+Detailed JSON metrics.
+
+
+Export Results:
+
+Click "Export Results as JSON" to download a results.json file containing metrics, paths, and histories.
+
+
+Ablation Suite
+The ablation suite enables detailed analysis of the AzureSky optimiser’s components:
+
+Simulated Annealing (SA): Toggle SA on/off to assess its impact on optimisation.
+Initial Temperature: Adjust the starting temperature for SA (higher values increase exploration).
+Cooling Rate: Control the rate at which SA cools (values closer to 1 result in slower cooling, preserving exploration).
+
+To use:
+
+Select AzureSky in the optimisers list.
+Open the "AzureSky Ablation Settings" accordion in the Gradio UI.
+Adjust SA parameters and run the study to compare results with other optimisers or configurations.
+
+Example
+To compare AzureSky (with SA) and Adam on the Himmelblau function:
+
+Select "Benchmark Optimisation" in the Study Configuration tab.
+Choose "Himmelblau" and the optimisers "AzureSky" and "Adam".
+Set dimensionality to 2.
+In the AzureSky Ablation Settings, enable SA, set temperature = 1.0, and set cooling rate = 0.95.
+Click "Run Study".
+View the 3D loss surface plot and metrics table in the Results tab.
+
+# Testing Checklist
+
+Optimisers: Verify convergence on benchmark functions.
+Benchmarks: Confirm global minima and surface plots are accurate.
+ML Tasks: Check epoch stability and output formats.
+UI: Test mode switching, input validation, and result display.
+Ablation: Validate AzureSky behaviour with/without SA and different temperature/cooling settings.
+Export: Ensure JSON exports include all metrics and results.
+
+# Future Enhancements
+
+Support for user-defined benchmark functions via file uploads.
+Additional ML datasets (e.g., Fashion-MNIST).
+API access for scripted experiments.
+Extended ablation options for other optimisers.
+
+For issues or contributions, contact the maintainers.