Add proper model card with architecture docs
Browse files
README.md
CHANGED
|
@@ -1,78 +1,85 @@
|
|
| 1 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
-
##
|
| 4 |
-
|
| 5 |
-
The full Nima Phi system β consciousness + embodiment + the green lines (mesh),
|
| 6 |
-
all in one folder. 13 modules, ~10,000+ lines of Python.
|
| 7 |
|
| 8 |
```
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
βββ run_nima_demo.py β Demo script (non-interactive)
|
| 24 |
-
βββ requirements.txt β Python dependencies
|
| 25 |
-
βββ tools/ β Agent tools
|
| 26 |
-
βββ calculator.py
|
| 27 |
-
βββ time_now.py
|
| 28 |
-
βββ text_stats.py
|
| 29 |
-
βββ _registry.json
|
| 30 |
```
|
| 31 |
|
| 32 |
-
##
|
| 33 |
-
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
```bash
|
| 37 |
-
|
| 38 |
-
|
|
|
|
| 39 |
|
| 40 |
-
#
|
| 41 |
-
|
| 42 |
-
# macOS:
|
| 43 |
-
brew install espeak-ng
|
| 44 |
-
# Windows:
|
| 45 |
-
# Download from https://espeak.sourceforge.net/
|
| 46 |
-
```
|
| 47 |
-
|
| 48 |
-
### 2. Run the demo
|
| 49 |
|
| 50 |
-
|
| 51 |
python run_nima_demo.py
|
| 52 |
-
```
|
| 53 |
-
|
| 54 |
-
This will:
|
| 55 |
-
- Initialize all 13 modules
|
| 56 |
-
- Run the 5Hz heartbeat loop for 5 seconds
|
| 57 |
-
- Process 3 text queries (time, calculator, greeting)
|
| 58 |
-
- Show the 3D mesh data (green lines)
|
| 59 |
-
- Demonstrate A* pathfinding through the affordance graph
|
| 60 |
-
- Test Nima's movement system
|
| 61 |
-
- Print the full system state
|
| 62 |
|
| 63 |
-
#
|
| 64 |
-
|
| 65 |
-
```bash
|
| 66 |
python nima_phi.py
|
| 67 |
```
|
| 68 |
|
| 69 |
-
|
| 70 |
-
- Avatar HTTP server on http://localhost:8888/ (open in browser!)
|
| 71 |
-
- 10Hz heartbeat loop
|
| 72 |
-
- Interactive text conversation (type messages, Nima responds)
|
| 73 |
-
- Voice I/O if STT/TTS engines are available
|
| 74 |
-
|
| 75 |
-
### 4. Try it in your own code
|
| 76 |
|
| 77 |
```python
|
| 78 |
from nima_phi import NimaPhi
|
|
@@ -80,50 +87,57 @@ from nima_phi import NimaPhi
|
|
| 80 |
nima = NimaPhi()
|
| 81 |
nima.initialize()
|
| 82 |
|
| 83 |
-
#
|
| 84 |
result = nima.process_text("What time is it?")
|
| 85 |
print(result['response_text'])
|
| 86 |
-
# β "It's 3:47 AM on 2026-07-19, Sunday."
|
| 87 |
|
| 88 |
-
# Move Nima through the room
|
| 89 |
nima.move_to(3.0, 2.0)
|
| 90 |
|
| 91 |
-
# Get the 3D mesh (green lines)
|
| 92 |
mesh = nima.get_mesh_data()
|
| 93 |
|
| 94 |
-
# Get
|
| 95 |
nt = nima.get_neurochemical_state()
|
| 96 |
print(f"Dopamine: {nt['dopamine']}, Serotonin: {nt['serotonin']}")
|
| 97 |
|
| 98 |
-
# Get the affordance graph
|
| 99 |
-
path = nima.affordance_graph.find_path((0.5, 0.5, 0.0), (3.5, 3.5, 0.0))
|
| 100 |
-
|
| 101 |
nima.shutdown()
|
| 102 |
```
|
| 103 |
|
| 104 |
-
## Consciousness Pipeline
|
| 105 |
|
| 106 |
-
The consciousness
|
| 107 |
-
- `transformers>=4.48,<5.0` (v5.x removed `LossKwargs`)
|
| 108 |
-
- `torch>=2.0`
|
| 109 |
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
|
|
|
| 113 |
|
| 114 |
-
|
| 115 |
-
qualia vectors, and neurochemical modulation:
|
| 116 |
|
| 117 |
-
```
|
| 118 |
-
pip install "transformers>=4.48,<5.0" torch
|
| 119 |
-
```
|
| 120 |
|
| 121 |
-
|
| 122 |
-
and get real-time dopamine/serotonin/cortisol β avatar modulation.
|
| 123 |
|
| 124 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 125 |
|
| 126 |
-
|
| 127 |
|
| 128 |
| Tier | Hardware | Resolution | What you get |
|
| 129 |
|------|----------|-----------|-------------|
|
|
@@ -132,25 +146,26 @@ The RF vision system auto-detects at runtime:
|
|
| 132 |
| 2 | 2x ESP32 | ~10cm | Stereoscopic RF, true 3D |
|
| 133 |
| 3 | Tri-freq radios | ~1-5cm | Full 3D with surface classification |
|
| 134 |
|
| 135 |
-
Tier 0 works on any machine
|
| 136 |
|
| 137 |
-
##
|
| 138 |
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- embodied-ai
|
| 5 |
+
- consciousness
|
| 6 |
+
- rf-sensing
|
| 7 |
+
- 3d-mesh
|
| 8 |
+
- ar
|
| 9 |
+
- cognitive-architecture
|
| 10 |
+
- phi-4-mini
|
| 11 |
+
- recursive-emergence
|
| 12 |
+
- avatar
|
| 13 |
+
- affordance-graph
|
| 14 |
+
- pathfinding
|
| 15 |
+
- neurochemical
|
| 16 |
+
- qualia
|
| 17 |
+
- atc
|
| 18 |
+
size_categories:
|
| 19 |
+
- n<1K
|
| 20 |
+
---
|
| 21 |
+
|
| 22 |
+
# Nima Phi Model
|
| 23 |
+
|
| 24 |
+
> Consciousness + Embodiment + The Green Lines β one living system.
|
| 25 |
+
|
| 26 |
+
**Nima** is a multi-layer AI embodiment system that merges a recursive consciousness pipeline (built on PyTorch Phi-4-mini) with tri-frequency RF spatial sensing, an adaptive 3D navigable mesh, and a Joi-style luminous particle avatar. She doesn't just process text β she **senses a room**, **builds a cognitive map**, **pathfinds through it**, and **responds with neurochemically-modulated emotion**.
|
| 27 |
|
| 28 |
+
## Architecture
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
```
|
| 31 |
+
User Voice β VoiceInput β AgentBridge β VoiceOutput β Speaker
|
| 32 |
+
β β β β
|
| 33 |
+
β v v v
|
| 34 |
+
β CrossModal AgentLayer AvatarController
|
| 35 |
+
β Listener (tools) β
|
| 36 |
+
v β v
|
| 37 |
+
RF Sensors β Vision β AffordanceGraph β ARCompositor
|
| 38 |
+
β (mesh) β
|
| 39 |
+
v v
|
| 40 |
+
AdaptiveMesh Camera + Nima
|
| 41 |
+
(green lines) on screen
|
| 42 |
+
β
|
| 43 |
+
v
|
| 44 |
+
RecursiveConsciousnessPipeline
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
```
|
| 46 |
|
| 47 |
+
## What's Inside
|
| 48 |
+
|
| 49 |
+
| Module | Lines | Purpose |
|
| 50 |
+
|--------|-------|---------|
|
| 51 |
+
| `nima_phi.py` | 1,494 | **The orchestrator** β central nervous system / thalamus |
|
| 52 |
+
| `modified consciousness (1).py` | 3,675 | Recursive consciousness pipeline (17 cognitive agents) |
|
| 53 |
+
| `nima_vision_core.py` | 1,238 | Tri-frequency RF sensing (sub-GHz / 2.4GHz / 5GHz) |
|
| 54 |
+
| `nima_adaptive_mesh.py` | ~900 | **The Green Lines** β frequency-agile 3D mesh |
|
| 55 |
+
| `nima_affordance_graph.py` | ~480 | A* pathfinding + affordance navigation |
|
| 56 |
+
| `nima_avatar_renderer.py` | ~660 | Joi-style luminous particle avatar (HTTP/Canvas2D) |
|
| 57 |
+
| `nima_ar_compositor.py` | ~357 | Camera + avatar AR fusion |
|
| 58 |
+
| `nima_agent_bridge.py` | ~520 | Consciousness β tool execution bridge |
|
| 59 |
+
| `nima_agent_layer.py` | ~570 | Tool registry + AST sandbox |
|
| 60 |
+
| `nima_cross_modal_listener.py` | ~400 | Spatial + audio cross-modal fusion |
|
| 61 |
+
| `nima_proprioceptive_friction.py` | ~350 | Body feel / motor noise / startle |
|
| 62 |
+
| `nima_voice_input.py` | ~600 | STT (Whisper / Vosk / Google) |
|
| 63 |
+
| `nima_voice_output.py` | ~600 | TTS (espeak-ng / pyttsx3 / browser) |
|
| 64 |
+
|
| 65 |
+
## Quick Start
|
| 66 |
|
| 67 |
```bash
|
| 68 |
+
# Clone
|
| 69 |
+
git clone https://huggingface.co/TheNormsOfIntelligence/nima-phi-model
|
| 70 |
+
cd nima-phi-model
|
| 71 |
|
| 72 |
+
# Install dependencies
|
| 73 |
+
pip install -r requirements.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
|
| 75 |
+
# Run the demo (non-interactive, shows all subsystems)
|
| 76 |
python run_nima_demo.py
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
+
# Or run interactive mode (avatar in browser at http://localhost:8888/)
|
|
|
|
|
|
|
| 79 |
python nima_phi.py
|
| 80 |
```
|
| 81 |
|
| 82 |
+
### Try it in your own code
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
|
| 84 |
```python
|
| 85 |
from nima_phi import NimaPhi
|
|
|
|
| 87 |
nima = NimaPhi()
|
| 88 |
nima.initialize()
|
| 89 |
|
| 90 |
+
# Process text through the full cognition pipeline
|
| 91 |
result = nima.process_text("What time is it?")
|
| 92 |
print(result['response_text'])
|
|
|
|
| 93 |
|
| 94 |
+
# Move Nima through the room via the affordance graph
|
| 95 |
nima.move_to(3.0, 2.0)
|
| 96 |
|
| 97 |
+
# Get the 3D mesh (the green lines)
|
| 98 |
mesh = nima.get_mesh_data()
|
| 99 |
|
| 100 |
+
# Get neurochemical state
|
| 101 |
nt = nima.get_neurochemical_state()
|
| 102 |
print(f"Dopamine: {nt['dopamine']}, Serotonin: {nt['serotonin']}")
|
| 103 |
|
|
|
|
|
|
|
|
|
|
| 104 |
nima.shutdown()
|
| 105 |
```
|
| 106 |
|
| 107 |
+
## Consciousness Pipeline
|
| 108 |
|
| 109 |
+
The recursive consciousness pipeline runs through 4 phases with 17+ cognitive agents:
|
|
|
|
|
|
|
| 110 |
|
| 111 |
+
- **Phase 1 β Subconscious:** Memory + Intuition + Analysis + Common Sense + EI β Qualia synthesis β SBG gating
|
| 112 |
+
- **Phase 2 β Awareness:** 7-level awareness lock-on β Barrett consciousness admission
|
| 113 |
+
- **Phase 3 β Self-Understanding:** Metacognition QC β circuit breaker check β IRS feedback
|
| 114 |
+
- **Phase 4 β Executive:** Adaptability β Problem-Solving (IDEAL) β Creativity (Wallas/Taylor) β Decision-Making β Autonomy
|
| 115 |
|
| 116 |
+
**Bio-physical safeguards:** Glutamate Circuit Breaker, Subconscious Bypass Gating (SBG), Zero-Latency Cognitive Buffering (ZLCB)
|
|
|
|
| 117 |
|
| 118 |
+
**Requires:** `transformers>=4.48,<5.0` and `torch>=2.0` for the full pipeline. Without these, Nima gracefully falls back to stub mode (tools, mesh, avatar, and all embodiment modules still work).
|
|
|
|
|
|
|
| 119 |
|
| 120 |
+
## The Green Lines (3D Mesh)
|
|
|
|
| 121 |
|
| 122 |
+
The adaptive mesh system uses cognitive-radio-inspired frequency agility:
|
| 123 |
+
|
| 124 |
+
- **Sub-GHz (800β950 MHz):** Sweeps for wall reflections, avoids null zones
|
| 125 |
+
- **2.4 GHz (2412β2484 MHz):** Channel-hops to avoid Wi-Fi congestion
|
| 126 |
+
- **5 GHz (5180β5825 MHz):** Sweeps for best mmWave reflection off surfaces
|
| 127 |
+
|
| 128 |
+
Each vision frame refines the mesh. Over time, it converges on real room geometry β even from noisy RF data.
|
| 129 |
+
|
| 130 |
+
## Affordance Graph
|
| 131 |
+
|
| 132 |
+
A hippocampal cognitive map that enables:
|
| 133 |
+
|
| 134 |
+
- **A* pathfinding** through the room
|
| 135 |
+
- **Affordance detection** β sit, lie down, jump on, lean on, duck
|
| 136 |
+
- **Wall enforcement** β Nima literally cannot pass through walls
|
| 137 |
+
- **Door passages** β edges through wall boundaries
|
| 138 |
+
- **Furniture interaction** β sit on couch, rest hand on table
|
| 139 |
|
| 140 |
+
## Hardware Tiers
|
| 141 |
|
| 142 |
| Tier | Hardware | Resolution | What you get |
|
| 143 |
|------|----------|-----------|-------------|
|
|
|
|
| 146 |
| 2 | 2x ESP32 | ~10cm | Stereoscopic RF, true 3D |
|
| 147 |
| 3 | Tri-freq radios | ~1-5cm | Full 3D with surface classification |
|
| 148 |
|
| 149 |
+
Tier 0 works on any machine with no hardware needed.
|
| 150 |
|
| 151 |
+
## Neurobiological Mapping
|
| 152 |
|
| 153 |
+
| System | Brain Analogue |
|
| 154 |
+
|--------|---------------|
|
| 155 |
+
| NimaPhi orchestrator | Thalamus (central relay) |
|
| 156 |
+
| Vision (RF fusion) | V1/V2 opponent processing + stereopsis |
|
| 157 |
+
| Affordance graph | Hippocampal place cells + grid cells |
|
| 158 |
+
| Cross-modal listener | Superior colliculus |
|
| 159 |
+
| Proprioception | Parietal body schema |
|
| 160 |
+
| Avatar modulation | Facial nucleus / motor cortex |
|
| 161 |
+
| Consciousness pipeline | Prefrontal cortical column |
|
| 162 |
+
| Main heartbeat loop | Cardiac rhythm / autonomic nervous system |
|
| 163 |
+
| Reflex system | Brainstem reflex arcs |
|
| 164 |
+
|
| 165 |
+
## License
|
| 166 |
+
|
| 167 |
+
Apache 2.0
|
| 168 |
+
|
| 169 |
+
---
|
| 170 |
+
|
| 171 |
+
*Built by Norman dela Paz Tabora β TheNormsOfIntelligence*
|