Spaces:
Running
Running
File size: 9,071 Bytes
540cfa6 c8b4583 ec936d5 c8b4583 540cfa6 c8b4583 540cfa6 ec936d5 540cfa6 ec936d5 540cfa6 ec936d5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 |
# LeRobot.js Conventions
## Project Overview
**lerobot.js** is a TypeScript/JavaScript implementation of Hugging Face's [lerobot](https://github.com/huggingface/lerobot) robotics library. Our goal is to bring state-of-the-art AI for real-world robotics directly to the JavaScript ecosystem, enabling robot control without any Python dependencies.
### Vision Statement
> Lower the barrier to entry for robotics by making cutting-edge robotic AI accessible through JavaScript, the world's most widely used programming language.
## Core Rules
- **Never Start/Stop Dev Server**: The development server is already managed by the user - never run commands to start, stop, or restart the server
- **Python lerobot Faithfulness**: Maintain exact UX/API compatibility with Python lerobot - commands, terminology, and workflows must match identically
- **Serial API Separation**: Always use `serialport` package for Node.js and Web Serial API for browsers - never mix or bridge these incompatible APIs
- **Minimal Console Output**: Only show essential information - reduce cognitive load for users
- **Hardware-First Testing**: Always validate with real hardware, not just simulation
## Project Goals
### Primary Objectives
1. **Native JavaScript/TypeScript Implementation**: Complete robotics stack running purely in JS/TS
2. **Feature Parity**: Implement core functionality from the original Python lerobot
3. **Web-First Design**: Enable robotics applications to run in browsers, Edge devices, and Node.js
4. **Real-World Robot Control**: Direct hardware interface without Python bridge
5. **Hugging Face Integration**: Seamless model and dataset loading from HF Hub
### Core Features to Implement
- **Pretrained Models**: Load and run robotics policies (ACT, Diffusion, TDMPC, VQ-BeT)
- **Dataset Management**: LeRobotDataset format with HF Hub integration
- **Simulation Environments**: Browser-based robotics simulations
- **Real Robot Support**: Hardware interfaces for motors, cameras, sensors
- **Training Infrastructure**: Policy training and evaluation tools
- **Visualization Tools**: Dataset and robot state visualization
## Technical Foundation
### Core Stack
- **Runtime**: Node.js 18+ / Modern Browsers
- **Language**: TypeScript with strict type checking
- **Build Tool**: Vite (development and production builds)
- **Package Manager**: pnpm
- **Module System**: ES Modules
- **Target**: ES2020
## Architecture Principles
### 1. Python lerobot Faithfulness (Primary Principle)
**lerobot.js must maintain UX/API compatibility with Python lerobot**
- **Identical Commands**: `npx lerobot find-port` matches `python -m lerobot.find_port`
- **Same Terminology**: Use "MotorsBus", not "robot arms" - keep Python's exact wording
- **Matching Output**: Error messages, prompts, and flow identical to Python version
- **Familiar Workflows**: Python lerobot users should feel immediately at home
- **No "Improvements"**: Resist urge to add features/UX that Python version doesn't have
> **Why?** Users are already trained on Python lerobot. Our goal is seamless migration to TypeScript, not learning a new tool.
### 2. Modular Design
```
lerobot/
├── common/
│ ├── datasets/ # Dataset loading and management
│ ├── envs/ # Simulation environments
│ ├── policies/ # AI policies and models
│ ├── devices/ # Hardware device interfaces
│ └── utils/ # Shared utilities
├── core/ # Core robotics primitives
├── node/ # Node.js-specific implementations
└── web/ # Browser-specific implementations
```
### 3. Platform Abstraction
- **Universal Core**: Platform-agnostic robotics logic
- **Web Adapters**: Browser-specific implementations (WebGL, WebAssembly, **Web Serial API**)
- **Node Adapters**: Node.js implementations (native modules, **serialport package**)
### 4. Serial Communication Standards (Critical)
**Serial communication must use platform-appropriate APIs - never mix or bridge:**
- **Node.js Platform**: ALWAYS use `serialport` package
- Event-based: `port.on('data', callback)`
- Programmatic port listing: `SerialPort.list()`
- Direct system access: `new SerialPort({ path: 'COM4' })`
- **Web Platform**: ALWAYS use Web Serial API
- Promise/Stream-based: `await reader.read()`
- User permission required: `navigator.serial.requestPort()`
- Browser security model: User must select port via dialog
**Why this matters:** The APIs are completely incompatible - different patterns, different capabilities, different security models. Mixing them leads to broken implementations.
### 5. Progressive Enhancement
- **Core Functionality**: Works everywhere (basic policy inference)
- **Enhanced Features**: Leverage platform capabilities (GPU acceleration, hardware access)
- **Premium Features**: Advanced capabilities (real-time training, complex simulations)
## Development Standards
### Code Style
- **Formatting**: Prettier with default settings
- **Linting**: ESLint with TypeScript recommended rules
- **Naming**:
- camelCase for variables/functions
- PascalCase for classes/types
- snake_case for file names (following lerobot convention)
- **File Structure**: Feature-based organization with index.ts barrels
### TypeScript Standards
- **Strict Mode**: All strict compiler options enabled
- **Type Safety**: Prefer types over interfaces for data structures
- **Generics**: Use generics for reusable components
- **Error Handling**: Use Result<T, E> pattern for recoverable errors
### Implementation Philosophy
- **Python First**: When in doubt, check how Python lerobot does it
- **Port, Don't Innovate**: Direct ports are better than clever improvements
- **User Expectations**: Maintain the exact experience Python users expect
- **Terminology Consistency**: Use Python lerobot's exact naming and messaging
### Development Process
- **Python Reference**: Always check Python lerobot implementation first
- **UX Matching**: Test that commands, outputs, and workflows match exactly
- **User Story Validation**: Validate against real Python lerobot users
### Testing Strategy
- **Unit Tests**: Vitest for individual functions and classes
- **Integration Tests**: Test component interactions
- **E2E Tests**: Playwright for full workflow testing
- **Hardware Tests**: Mock/stub hardware interfaces for CI
- **UX Compatibility Tests**: Verify outputs match Python version
## Package Structure
### NPM Package Name
- **Public Package**: `lerobot` (on npm)
- **Development Name**: `lerobot.js` (GitHub repository)
## Dependencies Strategy
### Core Dependencies
- **ML Inference**: ONNX.js for model execution (browser + Node.js)
- **Tensor Operations**: Custom lightweight tensor lib for data manipulation
- **Math**: Custom math utilities for robotics
- **Networking**: Fetch API (universal)
- **File I/O**: Platform-appropriate abstractions
### Optional Enhanced Dependencies
- **3D Graphics**: Three.js for simulation and visualization
- **Hardware**: Platform-specific libraries for device access
- **Development**: Vitest, ESLint, Prettier
## Hardware Implementation Lessons
### Critical Hardware Compatibility
#### Baudrate Configuration
- **Feetech Motors (SO-100)**: MUST use 1,000,000 baud to match Python lerobot
- **Python Reference**: `DEFAULT_BAUDRATE = 1_000_000` in Python lerobot codebase
- **Common Mistake**: Using 9600 baud causes "Read timeout" errors despite device connection
- **Verification**: Always test with real hardware - simulation won't catch baudrate issues
#### Console Output Philosophy
- **Minimal Cognitive Load**: Reduce console noise to absolute minimum
- **Silent Operations**: Connection, initialization, cleanup should be silent unless error occurs
- **Error-Only Logging**: Only show output when user needs to take action or when errors occur
- **Professional UX**: Robotics tools should have clean, distraction-free interfaces
#### Calibration Flow Matching
- **Python Behavior**: When user hits Enter during range recording, reading stops IMMEDIATELY
- **No Final Reads**: Never read motor positions after user completes calibration
- **User Expectation**: After Enter, user should be able to release robot (positions will change)
- **Flow Testing**: Always validate against Python lerobot's exact behavior
### Development Process Requirements
#### CLI Build Process
- **Critical**: After TypeScript changes, MUST run `pnpm run build` to update CLI
- **Global CLI**: `lerobot` command uses compiled `dist/` files, not source
- **Testing Flow**: Edit source → Build → Test CLI → Repeat
- **Common Mistake**: Testing source changes without rebuilding CLI
#### Hardware Testing Priority
- **Real Hardware Required**: Simulation cannot catch hardware-specific issues
- **Baudrate Validation**: Only real devices will reveal communication problems
- **User Flow Testing**: Test complete calibration workflows with actual hardware
- **Port Management**: Ensure proper port cleanup between testing sessions
|