NERDDISCO commited on
Commit
24f0634
·
1 Parent(s): 5eb1bc0

feat: remove unused code, apply DRY

Browse files
docs/api-comparison.md DELETED
@@ -1,194 +0,0 @@
1
- # lerobot API Comparison: Python vs Node.js vs Web
2
-
3
- This document provides a comprehensive three-way comparison of lerobot APIs across Python lerobot (original), Node.js lerobot.js, and Web lerobot.js platforms.
4
-
5
- ## 🔄 Core Function Comparison
6
-
7
- | Function Category | Python lerobot (Original) | Node.js lerobot.js | Web Browser lerobot.js | Key Pattern |
8
- | ------------------ | ------------------------- | ----------------------------- | ------------------------------------------------------ | ------------------------------------------------------ |
9
- | **Port Discovery** | `find_port()` | `findPort()` | `findPortWeb(logger)` | Python → Node.js: Direct port, Web: Requires UI logger |
10
- | **Robot Creation** | `SO100Follower(config)` | `createSO100Follower(config)` | `createWebTeleoperationController(port, serialNumber)` | Python: Class, Node.js: Factory, Web: Pre-opened port |
11
- | **Calibration** | `calibrate(cfg)` | `calibrate(config)` | `createCalibrationController(armType, port)` | Python/Node.js: Function, Web: Controller pattern |
12
- | **Teleoperation** | `teleoperate(cfg)` | `teleoperate(config)` | `createWebTeleoperationController(port, serialNumber)` | Python/Node.js: Function, Web: Manual state management |
13
-
14
- ## 📋 Detailed API Reference
15
-
16
- ### Port Discovery
17
-
18
- | Aspect | Python lerobot | Node.js lerobot.js | Web Browser lerobot.js |
19
- | -------------------- | ----------------------------------------- | -------------------------------------------- | ---------------------------------------------- |
20
- | **Function** | `find_port()` | `findPort()` | `findPortWeb(logger)` |
21
- | **Import** | `from lerobot.find_port import find_port` | `import { findPort } from 'lerobot.js/node'` | `import { findPortWeb } from 'lerobot.js/web'` |
22
- | **Parameters** | None | None | `logger: (message: string) => void` |
23
- | **User Interaction** | Terminal prompts via `input()` | Terminal prompts via readline | Browser modals and buttons |
24
- | **Port Access** | Direct system access via pyserial | Direct system access | Web Serial API permissions |
25
- | **Return Value** | None (prints to console) | None (prints to console) | None (calls logger function) |
26
- | **Example** | `python<br>find_port()<br>` | `js<br>await findPort();<br>` | `js<br>await findPortWeb(console.log);<br>` |
27
-
28
- ### Robot Connection & Creation
29
-
30
- | Aspect | Python lerobot | Node.js lerobot.js | Web Browser lerobot.js |
31
- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
32
- | **Creation** | `SO100Follower(config)` or `make_robot_from_config(config)` | `createSO100Follower(config)` | `createWebTeleoperationController(port, serialNumber)` |
33
- | **Connection** | `robot.connect()` | `await robot.connect()` | Port already opened before creation |
34
- | **Port Parameter** | `RobotConfig(port='/dev/ttyUSB0')` | `{ port: 'COM4' }` (string) | `SerialPort` object |
35
- | **Baud Rate** | Handled internally | Handled internally | `await port.open({ baudRate: 1000000 })` |
36
- | **Factory Pattern** | `make_robot_from_config()` factory | `createSO100Follower()` factory | `createWebTeleoperationController()` factory |
37
- | **Example** | `python<br>from lerobot.common.robots.so100_follower import SO100Follower<br>robot = SO100Follower(config)<br>robot.connect()<br>` | `js<br>const robot = createSO100Follower({<br> type: 'so100_follower',<br> port: 'COM4',<br> id: 'my_robot'<br>});<br>await robot.connect();<br>` | `js<br>const port = await navigator.serial.requestPort();<br>await port.open({ baudRate: 1000000 });<br>const robot = await createWebTeleoperationController(<br> port, 'my_robot'<br>);<br>` |
38
-
39
- ### Calibration
40
-
41
- | Aspect | Python lerobot | Node.js lerobot.js | Web Browser lerobot.js |
42
- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
43
- | **Main Function** | `calibrate(cfg)` | `calibrate(config)` | `createCalibrationController(armType, port)` |
44
- | **Import** | `from lerobot.calibrate import calibrate` | `import { calibrate } from 'lerobot.js/node'` | `import { createCalibrationController } from 'lerobot.js/web'` |
45
- | **Configuration** | `CalibrateConfig` dataclass | Single config object | Controller with methods |
46
- | **Workflow** | All-in-one function calls `device.calibrate()` | All-in-one function | Step-by-step methods |
47
- | **Device Pattern** | Creates device, calls `device.calibrate()`, `device.disconnect()` | Automatic within calibrate() | Manual controller lifecycle |
48
- | **Homing** | Automatic within `device.calibrate()` | Automatic within calibrate() | `await controller.performHomingStep()` |
49
- | **Range Recording** | Automatic within `device.calibrate()` | Automatic within calibrate() | `await controller.performRangeRecordingStep()` |
50
- | **Completion** | Automatic save and disconnect | Automatic save | `await controller.finishCalibration()` |
51
- | **Data Storage** | File system (HF cache) | File system (HF cache) | localStorage + file download |
52
- | **Example** | `python<br>from lerobot.calibrate import calibrate<br>from lerobot.common.robots.so100_follower import SO100FollowerConfig<br>calibrate(CalibrateConfig(<br> robot=SO100FollowerConfig(<br> port='/dev/ttyUSB0', id='my_robot'<br> )<br>))<br>` | `js<br>await calibrate({<br> robot: {<br> type: 'so100_follower',<br> port: 'COM4',<br> id: 'my_robot'<br> }<br>});<br>` | `js<br>const controller = await createCalibrationController(<br> 'so100_follower', port<br>);<br>await controller.performHomingStep();<br>await controller.performRangeRecordingStep(stopCondition);<br>const results = await controller.finishCalibration();<br>` |
53
-
54
- ### Teleoperation
55
-
56
- | Aspect | Python lerobot | Node.js lerobot.js | Web Browser lerobot.js |
57
- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
58
- | **Main Function** | `teleoperate(cfg)` | `teleoperate(config)` | `createWebTeleoperationController(port, serialNumber)` |
59
- | **Import** | `from lerobot.teleoperate import teleoperate` | `import { teleoperate } from 'lerobot.js/node'` | `import { createWebTeleoperationController } from 'lerobot.js/web'` |
60
- | **Control Loop** | `teleop_loop()` with `get_action()` and `send_action()` | Automatic 60 FPS loop | Manual start/stop with `controller.start()` |
61
- | **Device Management** | Creates teleop and robot devices, connects both | Device creation handled internally | Port opened externally, controller manages state |
62
- | **Input Handling** | Teleoperator `get_action()` method | Terminal raw mode | Browser event listeners |
63
- | **Key State** | Handled by teleoperator device | Internal management | `controller.updateKeyState(key, pressed)` |
64
- | **Configuration** | `TeleoperateConfig` with separate robot/teleop configs | `js<br>{<br> robot: { type, port, id },<br> teleop: { type: 'keyboard' },<br> fps: 60,<br> step_size: 25<br>}<br>` | Built into controller |
65
- | **Example** | `python<br>from lerobot.teleoperate import teleoperate<br>teleoperate(TeleoperateConfig(<br> robot=SO100FollowerConfig(port='/dev/ttyUSB0'),<br> teleop=SO100LeaderConfig(port='/dev/ttyUSB1')<br>))<br>` | `js<br>await teleoperate({<br> robot: {<br> type: 'so100_follower',<br> port: 'COM4',<br> id: 'my_robot'<br> },<br> teleop: { type: 'keyboard' }<br>});<br>` | `js<br>const controller = await createWebTeleoperationController(<br> port, 'my_robot'<br>);<br>controller.start();<br>// Handle keyboard events manually<br>document.addEventListener('keydown', (e) => {<br> controller.updateKeyState(e.key, true);<br>});<br>` |
66
-
67
- ### Motor Control
68
-
69
- | Aspect | Python lerobot | Node.js lerobot.js | Web Browser lerobot.js |
70
- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
71
- | **Get Positions** | `robot.get_observation()` (includes motor positions) | `await robot.getMotorPositions()` | `controller.getMotorConfigs()` or `controller.getState()` |
72
- | **Set Positions** | `robot.send_action(action)` (dict format) | `await robot.setMotorPositions(positions)` | `await controller.setMotorPositions(positions)` |
73
- | **Position Format** | `dict[str, float]` (action format) | `Record<string, number>` | `Record<string, number>` ✅ Same |
74
- | **Data Flow** | `get_observation()` → `send_action()` loop | Direct position get/set methods | Controller state management |
75
- | **Action Features** | `robot.action_features` (motor names) | Motor names hardcoded in implementation | Motor configs with metadata |
76
- | **Calibration Limits** | Handled in robot implementation | `robot.getCalibrationLimits()` | `controller.getMotorConfigs()` (includes limits) |
77
- | **Home Position** | Manual via action dict | Manual calculation | `await controller.goToHomePosition()` |
78
- | **Example** | `python<br>obs = robot.get_observation()<br>action = {motor: value for motor in robot.action_features}<br>robot.send_action(action)<br>` | `js<br>const positions = await robot.getMotorPositions();<br>await robot.setMotorPositions({<br> shoulder_pan: 2047,<br> shoulder_lift: 1800<br>});<br>` | `js<br>const state = controller.getState();<br>await controller.setMotorPositions({<br> shoulder_pan: 2047,<br> shoulder_lift: 1800<br>});<br>` |
79
-
80
- ### Calibration Data Management
81
-
82
- | Aspect | Node.js | Web Browser |
83
- | -------------------- | ------------------------------------------- | ------------------------------------- |
84
- | **Storage Location** | `~/.cache/huggingface/lerobot/calibration/` | `localStorage` + file download |
85
- | **File Format** | JSON files on disk | JSON in browser storage |
86
- | **Loading** | Automatic during `robot.connect()` | Manual via `loadCalibrationConfig()` |
87
- | **Saving** | `robot.saveCalibration(results)` | `saveCalibrationResults()` + download |
88
- | **Persistence** | Permanent until deleted | Browser-specific, can be cleared |
89
- | **Sharing** | File system sharing | Manual file sharing |
90
-
91
- ### Error Handling & Debugging
92
-
93
- | Aspect | Node.js | Web Browser |
94
- | --------------------- | ------------------------ | ------------------------------------ |
95
- | **Connection Errors** | Standard Node.js errors | Web Serial API errors |
96
- | **Permission Issues** | File system permissions | User permission prompts |
97
- | **Port Conflicts** | "Port in use" errors | Silent failures or permission errors |
98
- | **Debugging** | Console.log + terminal | Browser DevTools console |
99
- | **Logging** | Built-in terminal output | Passed logger functions |
100
-
101
- ## 🎯 Usage Pattern Summary
102
-
103
- ### Python lerobot (Original) - Research & Production
104
-
105
- ```python
106
- # Configuration-driven, device-based approach
107
- from lerobot.find_port import find_port
108
- from lerobot.calibrate import calibrate, CalibrateConfig
109
- from lerobot.teleoperate import teleoperate, TeleoperateConfig
110
- from lerobot.common.robots.so100_follower import SO100FollowerConfig
111
- from lerobot.common.teleoperators.so100_leader import SO100LeaderConfig
112
-
113
- # Find port
114
- find_port()
115
-
116
- # Calibrate robot
117
- calibrate(CalibrateConfig(
118
- robot=SO100FollowerConfig(port='/dev/ttyUSB0', id='my_robot')
119
- ))
120
-
121
- # Teleoperation with leader-follower
122
- teleoperate(TeleoperateConfig(
123
- robot=SO100FollowerConfig(port='/dev/ttyUSB0'),
124
- teleop=SO100LeaderConfig(port='/dev/ttyUSB1')
125
- ))
126
- ```
127
-
128
- ### Node.js lerobot.js - Server/Desktop Applications
129
-
130
- ```javascript
131
- // High-level, all-in-one functions (mirrors Python closely)
132
- import { findPort, calibrate, teleoperate } from "lerobot.js/node";
133
-
134
- await findPort();
135
- await calibrate({
136
- robot: { type: "so100_follower", port: "COM4", id: "my_robot" },
137
- });
138
- await teleoperate({
139
- robot: { type: "so100_follower", port: "COM4" },
140
- teleop: { type: "keyboard" },
141
- });
142
- ```
143
-
144
- ### Web Browser lerobot.js - Interactive Applications
145
-
146
- ```javascript
147
- // Controller-based, step-by-step approach (browser constraints)
148
- import {
149
- findPortWeb,
150
- createCalibrationController,
151
- createWebTeleoperationController,
152
- } from "lerobot.js/web";
153
-
154
- // User interaction required
155
- const port = await navigator.serial.requestPort();
156
- await port.open({ baudRate: 1000000 });
157
-
158
- // Step-by-step calibration
159
- const calibrator = await createCalibrationController("so100_follower", port);
160
- await calibrator.performHomingStep();
161
- await calibrator.performRangeRecordingStep(() => stopRecording);
162
-
163
- // Manual teleoperation control
164
- const controller = await createWebTeleoperationController(port, "my_robot");
165
- controller.start();
166
- ```
167
-
168
- ## 🔑 Key Architectural Differences
169
-
170
- 1. **User Interaction Model**
171
-
172
- - **Node.js**: Terminal-based with readline prompts
173
- - **Web**: Browser UI with buttons and modals
174
-
175
- 2. **Permission Model**
176
-
177
- - **Node.js**: System-level permissions
178
- - **Web**: User-granted permissions per device
179
-
180
- 3. **State Management**
181
-
182
- - **Node.js**: Function-based, stateless
183
- - **Web**: Controller-based, stateful
184
-
185
- 4. **Data Persistence**
186
-
187
- - **Node.js**: File system with cross-session persistence
188
- - **Web**: Browser storage with limited persistence
189
-
190
- 5. **Platform Integration**
191
- - **Node.js**: Deep system integration
192
- - **Web**: Security-constrained browser environment
193
-
194
- This comparison helps developers choose the right platform and understand the API differences when porting between Node.js and Web implementations.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
src/demo/App.tsx CHANGED
@@ -1,7 +1,7 @@
1
  import React, { useState } from "react";
2
  import { Home } from "./pages/Home";
3
  import { ErrorBoundary } from "./components/ErrorBoundary";
4
- import type { RobotConnection } from "../lerobot/web/find_port.js";
5
 
6
  export function App() {
7
  const [connectedRobots, setConnectedRobots] = useState<RobotConnection[]>([]);
 
1
  import React, { useState } from "react";
2
  import { Home } from "./pages/Home";
3
  import { ErrorBoundary } from "./components/ErrorBoundary";
4
+ import type { RobotConnection } from "../lerobot/web/types/robot-connection.js";
5
 
6
  export function App() {
7
  const [connectedRobots, setConnectedRobots] = useState<RobotConnection[]>([]);
src/demo/components/CalibrationPanel.tsx CHANGED
@@ -18,7 +18,7 @@ import { releaseMotors } from "../../lerobot/web/utils/motor-communication.js";
18
  import { WebSerialPortWrapper } from "../../lerobot/web/utils/serial-port-wrapper.js";
19
  import { createSO100Config } from "../../lerobot/web/robots/so100_config.js";
20
  import { CalibrationModal } from "./CalibrationModal";
21
- import type { RobotConnection } from "../../lerobot/web/find_port.js";
22
 
23
  interface CalibrationPanelProps {
24
  robot: RobotConnection;
@@ -104,16 +104,8 @@ export function CalibrationPanel({ robot, onFinish }: CalibrationPanelProps) {
104
  setIsCalibrating(true);
105
  initializeMotorData();
106
 
107
- // Use the new unified calibrate API - pass the whole robot connection
108
- const robotConnection = {
109
- port: robot.port,
110
- robotType: robot.robotType!,
111
- robotId: robot.robotId || `${robot.robotType}_1`,
112
- serialNumber: robot.serialNumber || `unknown_${Date.now()}`,
113
- connected: robot.isConnected,
114
- } as any; // Type assertion to work around SerialPort type differences
115
-
116
- const process = await calibrate(robotConnection, {
117
  onLiveUpdate: (data) => {
118
  setMotorData(data);
119
  setStatus(
 
18
  import { WebSerialPortWrapper } from "../../lerobot/web/utils/serial-port-wrapper.js";
19
  import { createSO100Config } from "../../lerobot/web/robots/so100_config.js";
20
  import { CalibrationModal } from "./CalibrationModal";
21
+ import type { RobotConnection } from "../../lerobot/web/types/robot-connection.js";
22
 
23
  interface CalibrationPanelProps {
24
  robot: RobotConnection;
 
104
  setIsCalibrating(true);
105
  initializeMotorData();
106
 
107
+ // Use the simple calibrate API - just pass the robot connection
108
+ const process = await calibrate(robot, {
 
 
 
 
 
 
 
 
109
  onLiveUpdate: (data) => {
110
  setMotorData(data);
111
  setStatus(
src/demo/components/PortManager.tsx CHANGED
@@ -18,7 +18,7 @@ import {
18
  DialogTitle,
19
  } from "./ui/dialog";
20
  import { isWebSerialSupported } from "../../lerobot/web/calibrate";
21
- import type { RobotConnection } from "../../lerobot/web/find_port.js";
22
 
23
  /**
24
  * Type definitions for WebSerial API (missing from TypeScript)
 
18
  DialogTitle,
19
  } from "./ui/dialog";
20
  import { isWebSerialSupported } from "../../lerobot/web/calibrate";
21
+ import type { RobotConnection } from "../../lerobot/web/types/robot-connection.js";
22
 
23
  /**
24
  * Type definitions for WebSerial API (missing from TypeScript)
src/demo/components/TeleoperationPanel.tsx CHANGED
@@ -1,12 +1,17 @@
1
- import React, { useState } from "react";
2
  import { Button } from "./ui/button";
3
  import { Card, CardContent, CardHeader, CardTitle } from "./ui/card";
4
  import { Badge } from "./ui/badge";
5
  import { Alert, AlertDescription } from "./ui/alert";
6
  import { Progress } from "./ui/progress";
7
- import { useTeleoperation } from "../hooks/useTeleoperation";
8
- import type { RobotConnection } from "../../lerobot/web/find_port.js";
9
- import { KEYBOARD_CONTROLS } from "../../lerobot/web/teleoperate";
 
 
 
 
 
10
 
11
  interface TeleoperationPanelProps {
12
  robot: RobotConnection;
@@ -17,41 +22,171 @@ export function TeleoperationPanel({
17
  robot,
18
  onClose,
19
  }: TeleoperationPanelProps) {
20
- const [enabled, setEnabled] = useState(false);
21
-
22
- const {
23
- isConnected,
24
- isActive,
25
- motorConfigs,
26
- keyStates,
27
- error,
28
- start,
29
- stop,
30
- simulateKeyPress,
31
- simulateKeyRelease,
32
- moveMotorToPosition,
33
- } = useTeleoperation({
34
- robot,
35
- enabled,
36
- onError: (err: string) => console.error("Teleoperation error:", err),
37
- });
38
-
39
- const handleStart = async () => {
40
- setEnabled(true);
41
- await start();
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  };
43
 
44
  const handleStop = () => {
45
- stop();
46
- setEnabled(false);
 
 
47
  };
48
 
49
  const handleClose = () => {
50
- stop();
51
- setEnabled(false);
 
52
  onClose();
53
  };
54
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
55
  // Virtual keyboard component
56
  const VirtualKeyboard = () => {
57
  const isKeyPressed = (key: string) => {
@@ -70,7 +205,9 @@ export function TeleoperationPanel({
70
  size?: "default" | "sm" | "lg" | "icon";
71
  }) => {
72
  const control =
73
- KEYBOARD_CONTROLS[keyCode as keyof typeof KEYBOARD_CONTROLS];
 
 
74
  const pressed = isKeyPressed(keyCode);
75
 
76
  return (
 
1
+ import React, { useState, useEffect, useRef, useCallback } from "react";
2
  import { Button } from "./ui/button";
3
  import { Card, CardContent, CardHeader, CardTitle } from "./ui/card";
4
  import { Badge } from "./ui/badge";
5
  import { Alert, AlertDescription } from "./ui/alert";
6
  import { Progress } from "./ui/progress";
7
+ import {
8
+ teleoperate,
9
+ type TeleoperationProcess,
10
+ type TeleoperationState,
11
+ } from "../../lerobot/web/teleoperate.js";
12
+ import { getUnifiedRobotData } from "../lib/unified-storage";
13
+ import type { RobotConnection } from "../../lerobot/web/types/robot-connection.js";
14
+ import { SO100_KEYBOARD_CONTROLS } from "../../lerobot/web/robots/so100_config.js";
15
 
16
  interface TeleoperationPanelProps {
17
  robot: RobotConnection;
 
22
  robot,
23
  onClose,
24
  }: TeleoperationPanelProps) {
25
+ const [teleoperationState, setTeleoperationState] =
26
+ useState<TeleoperationState>({
27
+ isActive: false,
28
+ motorConfigs: [],
29
+ lastUpdate: 0,
30
+ keyStates: {},
31
+ });
32
+ const [error, setError] = useState<string | null>(null);
33
+ const [isInitialized, setIsInitialized] = useState(false);
34
+
35
+ const teleoperationProcessRef = useRef<TeleoperationProcess | null>(null);
36
+
37
+ // Initialize teleoperation process
38
+ useEffect(() => {
39
+ const initializeTeleoperation = async () => {
40
+ if (!robot || !robot.robotType) {
41
+ setError("No robot configuration available");
42
+ return;
43
+ }
44
+
45
+ try {
46
+ // Load calibration data from demo storage (app concern)
47
+ let calibrationData;
48
+ if (robot.serialNumber) {
49
+ const data = getUnifiedRobotData(robot.serialNumber);
50
+ calibrationData = data?.calibration;
51
+ if (calibrationData) {
52
+ console.log("✅ Loaded calibration data for", robot.serialNumber);
53
+ }
54
+ }
55
+
56
+ // Create teleoperation process using clean library API
57
+ const process = await teleoperate(robot, {
58
+ calibrationData,
59
+ onStateUpdate: (state: TeleoperationState) => {
60
+ setTeleoperationState(state);
61
+ },
62
+ });
63
+
64
+ teleoperationProcessRef.current = process;
65
+ setTeleoperationState(process.getState());
66
+ setIsInitialized(true);
67
+ setError(null);
68
+ } catch (error) {
69
+ const errorMessage =
70
+ error instanceof Error
71
+ ? error.message
72
+ : "Failed to initialize teleoperation";
73
+ setError(errorMessage);
74
+ console.error("❌ Failed to initialize teleoperation:", error);
75
+ }
76
+ };
77
+
78
+ initializeTeleoperation();
79
+
80
+ return () => {
81
+ // Cleanup on unmount
82
+ if (teleoperationProcessRef.current) {
83
+ teleoperationProcessRef.current.disconnect();
84
+ teleoperationProcessRef.current = null;
85
+ }
86
+ };
87
+ }, [robot]);
88
+
89
+ // Keyboard event handlers
90
+ const handleKeyDown = useCallback(
91
+ (event: KeyboardEvent) => {
92
+ if (!teleoperationState.isActive || !teleoperationProcessRef.current)
93
+ return;
94
+
95
+ const key = event.key;
96
+ event.preventDefault();
97
+ teleoperationProcessRef.current.updateKeyState(key, true);
98
+ },
99
+ [teleoperationState.isActive]
100
+ );
101
+
102
+ const handleKeyUp = useCallback(
103
+ (event: KeyboardEvent) => {
104
+ if (!teleoperationState.isActive || !teleoperationProcessRef.current)
105
+ return;
106
+
107
+ const key = event.key;
108
+ event.preventDefault();
109
+ teleoperationProcessRef.current.updateKeyState(key, false);
110
+ },
111
+ [teleoperationState.isActive]
112
+ );
113
+
114
+ // Register keyboard events
115
+ useEffect(() => {
116
+ if (teleoperationState.isActive) {
117
+ window.addEventListener("keydown", handleKeyDown);
118
+ window.addEventListener("keyup", handleKeyUp);
119
+
120
+ return () => {
121
+ window.removeEventListener("keydown", handleKeyDown);
122
+ window.removeEventListener("keyup", handleKeyUp);
123
+ };
124
+ }
125
+ }, [teleoperationState.isActive, handleKeyDown, handleKeyUp]);
126
+
127
+ const handleStart = () => {
128
+ if (!teleoperationProcessRef.current) {
129
+ setError("Teleoperation not initialized");
130
+ return;
131
+ }
132
+
133
+ try {
134
+ teleoperationProcessRef.current.start();
135
+ console.log("🎮 Teleoperation started");
136
+ } catch (error) {
137
+ const errorMessage =
138
+ error instanceof Error
139
+ ? error.message
140
+ : "Failed to start teleoperation";
141
+ setError(errorMessage);
142
+ }
143
  };
144
 
145
  const handleStop = () => {
146
+ if (!teleoperationProcessRef.current) return;
147
+
148
+ teleoperationProcessRef.current.stop();
149
+ console.log("🛑 Teleoperation stopped");
150
  };
151
 
152
  const handleClose = () => {
153
+ if (teleoperationProcessRef.current) {
154
+ teleoperationProcessRef.current.stop();
155
+ }
156
  onClose();
157
  };
158
 
159
+ const simulateKeyPress = (key: string) => {
160
+ if (!teleoperationProcessRef.current) return;
161
+ teleoperationProcessRef.current.updateKeyState(key, true);
162
+ };
163
+
164
+ const simulateKeyRelease = (key: string) => {
165
+ if (!teleoperationProcessRef.current) return;
166
+ teleoperationProcessRef.current.updateKeyState(key, false);
167
+ };
168
+
169
+ const moveMotorToPosition = async (motorIndex: number, position: number) => {
170
+ if (!teleoperationProcessRef.current) return;
171
+
172
+ try {
173
+ const motorName = teleoperationState.motorConfigs[motorIndex]?.name;
174
+ if (motorName) {
175
+ await teleoperationProcessRef.current.moveMotor(motorName, position);
176
+ }
177
+ } catch (error) {
178
+ console.warn(
179
+ `Failed to move motor ${motorIndex + 1} to position ${position}:`,
180
+ error
181
+ );
182
+ }
183
+ };
184
+
185
+ const isConnected = robot?.isConnected || false;
186
+ const isActive = teleoperationState.isActive;
187
+ const motorConfigs = teleoperationState.motorConfigs;
188
+ const keyStates = teleoperationState.keyStates;
189
+
190
  // Virtual keyboard component
191
  const VirtualKeyboard = () => {
192
  const isKeyPressed = (key: string) => {
 
205
  size?: "default" | "sm" | "lg" | "icon";
206
  }) => {
207
  const control =
208
+ SO100_KEYBOARD_CONTROLS[
209
+ keyCode as keyof typeof SO100_KEYBOARD_CONTROLS
210
+ ];
211
  const pressed = isKeyPressed(keyCode);
212
 
213
  return (
src/demo/hooks/useTeleoperation.ts DELETED
@@ -1,518 +0,0 @@
1
- import { useState, useEffect, useCallback, useRef } from "react";
2
- import { useRobotConnection } from "./useRobotConnection";
3
- import { getUnifiedRobotData } from "../lib/unified-storage";
4
- import type { RobotConnection } from "../../lerobot/web/find_port.js";
5
-
6
- export interface MotorConfig {
7
- name: string;
8
- minPosition: number;
9
- maxPosition: number;
10
- currentPosition: number;
11
- homePosition: number;
12
- }
13
-
14
- export interface KeyState {
15
- pressed: boolean;
16
- lastPressed: number;
17
- }
18
-
19
- export interface UseTeleoperationOptions {
20
- robot: RobotConnection;
21
- enabled: boolean;
22
- onError?: (error: string) => void;
23
- }
24
-
25
- export interface UseTeleoperationResult {
26
- // Connection state from singleton
27
- isConnected: boolean;
28
- isActive: boolean;
29
-
30
- // Motor state
31
- motorConfigs: MotorConfig[];
32
-
33
- // Keyboard state
34
- keyStates: Record<string, KeyState>;
35
-
36
- // Error state
37
- error: string | null;
38
-
39
- // Control methods
40
- start: () => void;
41
- stop: () => void;
42
- goToHome: () => Promise<void>;
43
- simulateKeyPress: (key: string) => void;
44
- simulateKeyRelease: (key: string) => void;
45
- moveMotorToPosition: (motorIndex: number, position: number) => Promise<void>;
46
- }
47
-
48
- const MOTOR_CONFIGS: MotorConfig[] = [
49
- {
50
- name: "shoulder_pan",
51
- minPosition: 0,
52
- maxPosition: 4095,
53
- currentPosition: 2048,
54
- homePosition: 2048,
55
- },
56
- {
57
- name: "shoulder_lift",
58
- minPosition: 1024,
59
- maxPosition: 3072,
60
- currentPosition: 2048,
61
- homePosition: 2048,
62
- },
63
- {
64
- name: "elbow_flex",
65
- minPosition: 1024,
66
- maxPosition: 3072,
67
- currentPosition: 2048,
68
- homePosition: 2048,
69
- },
70
- {
71
- name: "wrist_flex",
72
- minPosition: 1024,
73
- maxPosition: 3072,
74
- currentPosition: 2048,
75
- homePosition: 2048,
76
- },
77
- {
78
- name: "wrist_roll",
79
- minPosition: 0,
80
- maxPosition: 4095,
81
- currentPosition: 2048,
82
- homePosition: 2048,
83
- },
84
- {
85
- name: "gripper",
86
- minPosition: 1800,
87
- maxPosition: 2400,
88
- currentPosition: 2100,
89
- homePosition: 2100,
90
- },
91
- ];
92
-
93
- // PROVEN VALUES from Node.js implementation (conventions.md)
94
- const SMOOTH_CONTROL_CONFIG = {
95
- STEP_SIZE: 25, // Proven optimal from conventions.md
96
- CHANGE_THRESHOLD: 0.5, // Prevents micro-movements and unnecessary commands
97
- MOTOR_DELAY: 1, // Minimal delay between motor commands (from conventions.md)
98
- UPDATE_INTERVAL: 30, // 30ms = ~33Hz for responsive control (was 50ms = 20Hz)
99
- } as const;
100
-
101
- const KEYBOARD_CONTROLS = {
102
- ArrowUp: { motorIndex: 1, direction: 1, description: "Shoulder lift up" },
103
- ArrowDown: {
104
- motorIndex: 1,
105
- direction: -1,
106
- description: "Shoulder lift down",
107
- },
108
- ArrowLeft: { motorIndex: 0, direction: -1, description: "Shoulder pan left" },
109
- ArrowRight: {
110
- motorIndex: 0,
111
- direction: 1,
112
- description: "Shoulder pan right",
113
- },
114
- w: { motorIndex: 2, direction: 1, description: "Elbow flex up" },
115
- s: { motorIndex: 2, direction: -1, description: "Elbow flex down" },
116
- a: { motorIndex: 3, direction: -1, description: "Wrist flex left" },
117
- d: { motorIndex: 3, direction: 1, description: "Wrist flex right" },
118
- q: { motorIndex: 4, direction: -1, description: "Wrist roll left" },
119
- e: { motorIndex: 4, direction: 1, description: "Wrist roll right" },
120
- o: { motorIndex: 5, direction: 1, description: "Gripper open" },
121
- c: { motorIndex: 5, direction: -1, description: "Gripper close" },
122
- Escape: { motorIndex: -1, direction: 0, description: "Emergency stop" },
123
- };
124
-
125
- export function useTeleoperation({
126
- robot,
127
- enabled,
128
- onError,
129
- }: UseTeleoperationOptions): UseTeleoperationResult {
130
- const connection = useRobotConnection();
131
- const [isActive, setIsActive] = useState(false);
132
- const [motorConfigs, setMotorConfigs] =
133
- useState<MotorConfig[]>(MOTOR_CONFIGS);
134
- const [keyStates, setKeyStates] = useState<Record<string, KeyState>>({});
135
- const [error, setError] = useState<string | null>(null);
136
-
137
- const activeKeysRef = useRef<Set<string>>(new Set());
138
- const motorPositionsRef = useRef<number[]>(
139
- MOTOR_CONFIGS.map((m) => m.homePosition)
140
- );
141
- const movementIntervalRef = useRef<NodeJS.Timeout | null>(null);
142
-
143
- // Load calibration data
144
- useEffect(() => {
145
- const loadCalibration = async () => {
146
- try {
147
- if (!robot.serialNumber) {
148
- console.warn("No serial number available for calibration loading");
149
- return;
150
- }
151
-
152
- const data = getUnifiedRobotData(robot.serialNumber);
153
- if (data?.calibration) {
154
- // Map motor names to calibration data
155
- const motorNames = [
156
- "shoulder_pan",
157
- "shoulder_lift",
158
- "elbow_flex",
159
- "wrist_flex",
160
- "wrist_roll",
161
- "gripper",
162
- ];
163
- const calibratedConfigs = MOTOR_CONFIGS.map((config, index) => {
164
- const motorName = motorNames[index] as keyof NonNullable<
165
- typeof data.calibration
166
- >;
167
- const calibratedMotor = data.calibration![motorName];
168
- if (
169
- calibratedMotor &&
170
- typeof calibratedMotor === "object" &&
171
- "homing_offset" in calibratedMotor &&
172
- "range_min" in calibratedMotor &&
173
- "range_max" in calibratedMotor
174
- ) {
175
- // Use 2048 as default home position, adjusted by homing offset
176
- const homePosition = 2048 + (calibratedMotor.homing_offset || 0);
177
- return {
178
- ...config,
179
- homePosition,
180
- currentPosition: homePosition,
181
- // IMPORTANT: Use actual calibrated limits instead of hardcoded ones
182
- minPosition: calibratedMotor.range_min || config.minPosition,
183
- maxPosition: calibratedMotor.range_max || config.maxPosition,
184
- };
185
- }
186
- return config;
187
- });
188
- setMotorConfigs(calibratedConfigs);
189
- // DON'T set motorPositionsRef here - it will be set when teleoperation starts
190
- // motorPositionsRef.current = calibratedConfigs.map((m) => m.homePosition);
191
- console.log("✅ Loaded calibration data for", robot.serialNumber);
192
- }
193
- } catch (error) {
194
- console.warn("Failed to load calibration:", error);
195
- }
196
- };
197
-
198
- loadCalibration();
199
- }, [robot.serialNumber]);
200
-
201
- // Keyboard event handlers
202
- const handleKeyDown = useCallback(
203
- (event: KeyboardEvent) => {
204
- if (!isActive) return;
205
-
206
- const key = event.key;
207
- if (key in KEYBOARD_CONTROLS) {
208
- event.preventDefault();
209
-
210
- if (key === "Escape") {
211
- setIsActive(false);
212
- activeKeysRef.current.clear();
213
- return;
214
- }
215
-
216
- if (!activeKeysRef.current.has(key)) {
217
- activeKeysRef.current.add(key);
218
- setKeyStates((prev) => ({
219
- ...prev,
220
- [key]: { pressed: true, lastPressed: Date.now() },
221
- }));
222
- }
223
- }
224
- },
225
- [isActive]
226
- );
227
-
228
- const handleKeyUp = useCallback(
229
- (event: KeyboardEvent) => {
230
- if (!isActive) return;
231
-
232
- const key = event.key;
233
- if (key in KEYBOARD_CONTROLS) {
234
- event.preventDefault();
235
- activeKeysRef.current.delete(key);
236
- setKeyStates((prev) => ({
237
- ...prev,
238
- [key]: { pressed: false, lastPressed: Date.now() },
239
- }));
240
- }
241
- },
242
- [isActive]
243
- );
244
-
245
- // Register keyboard events
246
- useEffect(() => {
247
- if (enabled && isActive) {
248
- window.addEventListener("keydown", handleKeyDown);
249
- window.addEventListener("keyup", handleKeyUp);
250
-
251
- return () => {
252
- window.removeEventListener("keydown", handleKeyDown);
253
- window.removeEventListener("keyup", handleKeyUp);
254
- };
255
- }
256
- }, [enabled, isActive, handleKeyDown, handleKeyUp]);
257
-
258
- // CONTINUOUS MOVEMENT: For held keys with PROVEN smooth patterns from Node.js
259
- useEffect(() => {
260
- if (!isActive || !connection.isConnected) {
261
- if (movementIntervalRef.current) {
262
- clearInterval(movementIntervalRef.current);
263
- movementIntervalRef.current = null;
264
- }
265
- return;
266
- }
267
-
268
- const processMovement = async () => {
269
- if (activeKeysRef.current.size === 0) return;
270
-
271
- const activeKeys = Array.from(activeKeysRef.current);
272
- const changedMotors: Array<{ index: number; position: number }> = [];
273
-
274
- // PROVEN PATTERN: Process all active keys and collect changes
275
- for (const key of activeKeys) {
276
- const control =
277
- KEYBOARD_CONTROLS[key as keyof typeof KEYBOARD_CONTROLS];
278
- if (control && control.motorIndex >= 0) {
279
- const motorIndex = control.motorIndex;
280
- const direction = control.direction;
281
- const motor = motorConfigs[motorIndex];
282
-
283
- if (motor) {
284
- const currentPos = motorPositionsRef.current[motorIndex];
285
- let newPos =
286
- currentPos + direction * SMOOTH_CONTROL_CONFIG.STEP_SIZE;
287
-
288
- // Clamp to motor limits
289
- newPos = Math.max(
290
- motor.minPosition,
291
- Math.min(motor.maxPosition, newPos)
292
- );
293
-
294
- // PROVEN PATTERN: Only update if change is meaningful (0.5 unit threshold)
295
- if (
296
- Math.abs(newPos - currentPos) >
297
- SMOOTH_CONTROL_CONFIG.CHANGE_THRESHOLD
298
- ) {
299
- motorPositionsRef.current[motorIndex] = newPos;
300
- changedMotors.push({ index: motorIndex, position: newPos });
301
- }
302
- }
303
- }
304
- }
305
-
306
- // PROVEN PATTERN: Only send commands for motors that actually changed
307
- if (changedMotors.length > 0) {
308
- try {
309
- for (const { index, position } of changedMotors) {
310
- await connection.writeMotorPosition(index + 1, position);
311
-
312
- // PROVEN PATTERN: Minimal delay between motor commands (1ms)
313
- if (changedMotors.length > 1) {
314
- await new Promise((resolve) =>
315
- setTimeout(resolve, SMOOTH_CONTROL_CONFIG.MOTOR_DELAY)
316
- );
317
- }
318
- }
319
-
320
- // Update UI to reflect changes
321
- setMotorConfigs((prev) =>
322
- prev.map((config, index) => ({
323
- ...config,
324
- currentPosition: motorPositionsRef.current[index],
325
- }))
326
- );
327
- } catch (error) {
328
- console.warn("Failed to update robot positions:", error);
329
- }
330
- }
331
- };
332
-
333
- // PROVEN TIMING: 30ms interval (~33Hz) for responsive continuous movement
334
- movementIntervalRef.current = setInterval(
335
- processMovement,
336
- SMOOTH_CONTROL_CONFIG.UPDATE_INTERVAL
337
- );
338
-
339
- return () => {
340
- if (movementIntervalRef.current) {
341
- clearInterval(movementIntervalRef.current);
342
- movementIntervalRef.current = null;
343
- }
344
- };
345
- }, [
346
- isActive,
347
- connection.isConnected,
348
- connection.writeMotorPosition,
349
- motorConfigs,
350
- ]);
351
-
352
- // Control methods
353
- const start = useCallback(async () => {
354
- if (!connection.isConnected) {
355
- setError("Robot not connected");
356
- onError?.("Robot not connected");
357
- return;
358
- }
359
-
360
- try {
361
- console.log(
362
- "🎮 Starting teleoperation - reading current motor positions..."
363
- );
364
-
365
- // Read current positions of all motors using PROVEN utility
366
- const motorIds = [1, 2, 3, 4, 5, 6];
367
- const currentPositions = await connection.readAllMotorPositions(motorIds);
368
-
369
- // Log all positions (trust the utility's fallback handling)
370
- for (let i = 0; i < currentPositions.length; i++) {
371
- const position = currentPositions[i];
372
- console.log(`📍 Motor ${i + 1} current position: ${position}`);
373
- }
374
-
375
- // CRITICAL: Update positions BEFORE activating movement
376
- motorPositionsRef.current = currentPositions;
377
-
378
- // Update UI to show actual current positions
379
- setMotorConfigs((prev) =>
380
- prev.map((config, index) => ({
381
- ...config,
382
- currentPosition: currentPositions[index],
383
- }))
384
- );
385
-
386
- // IMPORTANT: Only activate AFTER positions are synchronized
387
- setIsActive(true);
388
- setError(null);
389
- console.log(
390
- "✅ Teleoperation started with synchronized positions:",
391
- currentPositions
392
- );
393
- } catch (error) {
394
- const errorMessage =
395
- error instanceof Error
396
- ? error.message
397
- : "Failed to start teleoperation";
398
- setError(errorMessage);
399
- onError?.(errorMessage);
400
- console.error("❌ Failed to start teleoperation:", error);
401
- }
402
- }, [
403
- connection.isConnected,
404
- connection.readAllMotorPositions,
405
- motorConfigs,
406
- onError,
407
- ]);
408
-
409
- const stop = useCallback(() => {
410
- setIsActive(false);
411
- activeKeysRef.current.clear();
412
- setKeyStates({});
413
- console.log("🛑 Teleoperation stopped");
414
- }, []);
415
-
416
- const goToHome = useCallback(async () => {
417
- if (!connection.isConnected) {
418
- setError("Robot not connected");
419
- return;
420
- }
421
-
422
- try {
423
- for (let i = 0; i < motorConfigs.length; i++) {
424
- const motor = motorConfigs[i];
425
- await connection.writeMotorPosition(i + 1, motor.homePosition);
426
- motorPositionsRef.current[i] = motor.homePosition;
427
- }
428
-
429
- setMotorConfigs((prev) =>
430
- prev.map((config) => ({
431
- ...config,
432
- currentPosition: config.homePosition,
433
- }))
434
- );
435
-
436
- console.log("🏠 Moved to home position");
437
- } catch (error) {
438
- const errorMessage =
439
- error instanceof Error ? error.message : "Failed to go to home";
440
- setError(errorMessage);
441
- onError?.(errorMessage);
442
- }
443
- }, [
444
- connection.isConnected,
445
- connection.writeMotorPosition,
446
- motorConfigs,
447
- onError,
448
- ]);
449
-
450
- const simulateKeyPress = useCallback(
451
- (key: string) => {
452
- if (!isActive) return;
453
-
454
- activeKeysRef.current.add(key);
455
- setKeyStates((prev) => ({
456
- ...prev,
457
- [key]: { pressed: true, lastPressed: Date.now() },
458
- }));
459
- },
460
- [isActive]
461
- );
462
-
463
- const simulateKeyRelease = useCallback(
464
- (key: string) => {
465
- if (!isActive) return;
466
-
467
- activeKeysRef.current.delete(key);
468
- setKeyStates((prev) => ({
469
- ...prev,
470
- [key]: { pressed: false, lastPressed: Date.now() },
471
- }));
472
- },
473
- [isActive]
474
- );
475
-
476
- const moveMotorToPosition = useCallback(
477
- async (motorIndex: number, position: number) => {
478
- if (!connection.isConnected) {
479
- return;
480
- }
481
-
482
- try {
483
- await connection.writeMotorPosition(motorIndex + 1, position);
484
-
485
- // Update internal state
486
- motorPositionsRef.current[motorIndex] = position;
487
-
488
- setMotorConfigs((prev) =>
489
- prev.map((config, index) => ({
490
- ...config,
491
- currentPosition:
492
- index === motorIndex ? position : config.currentPosition,
493
- }))
494
- );
495
- } catch (error) {
496
- console.warn(
497
- `Failed to move motor ${motorIndex + 1} to position ${position}:`,
498
- error
499
- );
500
- }
501
- },
502
- [connection.isConnected, connection.writeMotorPosition]
503
- );
504
-
505
- return {
506
- isConnected: connection.isConnected,
507
- isActive,
508
- motorConfigs,
509
- keyStates,
510
- error,
511
- start,
512
- stop,
513
- goToHome,
514
- simulateKeyPress,
515
- simulateKeyRelease,
516
- moveMotorToPosition,
517
- };
518
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
src/demo/pages/Home.tsx CHANGED
@@ -12,7 +12,7 @@ import { PortManager } from "../components/PortManager";
12
  import { CalibrationPanel } from "../components/CalibrationPanel";
13
  import { TeleoperationPanel } from "../components/TeleoperationPanel";
14
  import { isWebSerialSupported } from "../../lerobot/web/calibrate";
15
- import type { RobotConnection } from "../../lerobot/web/find_port.js";
16
 
17
  interface HomeProps {
18
  onGetStarted: () => void;
 
12
  import { CalibrationPanel } from "../components/CalibrationPanel";
13
  import { TeleoperationPanel } from "../components/TeleoperationPanel";
14
  import { isWebSerialSupported } from "../../lerobot/web/calibrate";
15
+ import type { RobotConnection } from "../../lerobot/web/types/robot-connection.js";
16
 
17
  interface HomeProps {
18
  onGetStarted: () => void;
src/lerobot/web/calibrate.ts CHANGED
@@ -1,14 +1,10 @@
1
  /**
2
  * Web calibration functionality using Web Serial API
3
- * Minimal library - does calibration, user handles storage/UI/etc.
4
  *
5
- * Currently supports SO-100 robots. Other robot types can be added by:
6
- * 1. Creating robot-specific config files in ./robots/
7
- * 2. Extending the calibrate() function to accept different robot types
8
- * 3. Adding robot-specific protocol configurations
9
  */
10
 
11
- import { createSO100Config } from "./robots/so100_config.js";
12
  import { WebSerialPortWrapper } from "./utils/serial-port-wrapper.js";
13
  import {
14
  readAllMotorPositions,
@@ -19,33 +15,11 @@ import {
19
  setHomingOffsets,
20
  writeHardwarePositionLimits,
21
  } from "./utils/motor-calibration.js";
22
- import type { RobotConnection } from "./find_port.js";
23
-
24
- /**
25
- * Device calibration configuration interface
26
- * Currently designed for SO-100, but can be extended for other robot types
27
- */
28
- interface WebCalibrationConfig {
29
- deviceType: "so100_follower" | "so100_leader";
30
- port: WebSerialPortWrapper;
31
- motorNames: string[];
32
- motorIds: number[];
33
- driveModes: number[];
34
 
35
- // Protocol-specific configuration
36
- protocol: {
37
- resolution: number;
38
- homingOffsetAddress: number;
39
- homingOffsetLength: number;
40
- presentPositionAddress: number;
41
- presentPositionLength: number;
42
- minPositionLimitAddress: number;
43
- minPositionLimitLength: number;
44
- maxPositionLimitAddress: number;
45
- maxPositionLimitLength: number;
46
- signMagnitudeBit: number;
47
- };
48
- }
49
 
50
  /**
51
  * Calibration results structure matching Python lerobot format exactly
@@ -149,25 +123,33 @@ async function recordRangesOfMotion(
149
  }
150
 
151
  /**
152
- * Create SO-100 web configuration
 
153
  */
154
- function createSO100WebConfig(
155
- deviceType: "so100_follower" | "so100_leader",
156
- port: WebSerialPortWrapper
157
- ): WebCalibrationConfig {
158
- const so100Config = createSO100Config(deviceType);
 
 
 
 
 
 
 
 
 
159
 
160
- return {
161
- ...so100Config,
162
- port,
163
- };
 
164
  }
165
 
166
  /**
167
- * Main calibrate function - minimal library API
168
- * Currently supports SO-100 robots (follower and leader)
169
- *
170
- * Takes a unified RobotConnection object from findPort()
171
  */
172
  export async function calibrate(
173
  robotConnection: RobotConnection,
@@ -187,8 +169,13 @@ export async function calibrate(
187
  const port = new WebSerialPortWrapper(robotConnection.port);
188
  await port.initialize();
189
 
190
- // Get SO-100 specific calibration configuration
191
- const config = createSO100WebConfig(robotConnection.robotType, port);
 
 
 
 
 
192
 
193
  let shouldStop = false;
194
  const stopFunction = () => shouldStop;
@@ -198,30 +185,31 @@ export async function calibrate(
198
  // Step 1: Set homing offsets (automatic)
199
  options?.onProgress?.("⚙️ Setting motor homing offsets");
200
  const homingOffsets = await setHomingOffsets(
201
- config.port,
202
  config.motorIds,
203
  config.motorNames
204
  );
205
 
206
  // Step 2: Record ranges of motion with live updates
207
  const { rangeMins, rangeMaxes } = await recordRangesOfMotion(
208
- config.port,
209
  config.motorIds,
210
  config.motorNames,
211
  stopFunction,
212
  options?.onLiveUpdate
213
  );
214
 
215
- // Step 3: Set special range for wrist_roll (full turn motor)
216
- // The wrist_roll is a continuous rotation motor that should use the full
217
- // 0-4095 range regardless of what the user recorded during calibration.
218
- // This matches the hardware specification and Python lerobot behavior.
219
- rangeMins["wrist_roll"] = 0;
220
- rangeMaxes["wrist_roll"] = 4095;
 
221
 
222
  // Step 4: Write hardware position limits to motors
223
  await writeHardwarePositionLimits(
224
- config.port,
225
  config.motorIds,
226
  config.motorNames,
227
  rangeMins,
 
1
  /**
2
  * Web calibration functionality using Web Serial API
3
+ * Simple library API - pass in robotConnection, get calibration results
4
  *
5
+ * Handles different robot types internally - users don't need to know about configs
 
 
 
6
  */
7
 
 
8
  import { WebSerialPortWrapper } from "./utils/serial-port-wrapper.js";
9
  import {
10
  readAllMotorPositions,
 
15
  setHomingOffsets,
16
  writeHardwarePositionLimits,
17
  } from "./utils/motor-calibration.js";
18
+ import { createSO100Config } from "./robots/so100_config.js";
19
+ import type { RobotConnection } from "./types/robot-connection.js";
 
 
 
 
 
 
 
 
 
 
20
 
21
+ // Import shared robot hardware configuration interface
22
+ import type { RobotHardwareConfig } from "./types/robot-config.js";
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
  /**
25
  * Calibration results structure matching Python lerobot format exactly
 
123
  }
124
 
125
  /**
126
+ * Apply robot-specific range adjustments
127
+ * Different robot types may have special cases (like continuous rotation motors)
128
  */
129
+ function applyRobotSpecificRangeAdjustments(
130
+ robotType: string,
131
+ protocol: { resolution: number },
132
+ rangeMins: { [motor: string]: number },
133
+ rangeMaxes: { [motor: string]: number }
134
+ ): void {
135
+ // SO-100 specific: wrist_roll is a continuous rotation motor
136
+ if (robotType.startsWith("so100") && rangeMins["wrist_roll"] !== undefined) {
137
+ // The wrist_roll is a continuous rotation motor that should use the full
138
+ // 0-4095 range regardless of what the user recorded during calibration.
139
+ // This matches the hardware specification and Python lerobot behavior.
140
+ rangeMins["wrist_roll"] = 0;
141
+ rangeMaxes["wrist_roll"] = protocol.resolution - 1;
142
+ }
143
 
144
+ // Future robot types can add their own specific adjustments here
145
+ // if (robotType.startsWith('newrobot') && rangeMins["special_joint"] !== undefined) {
146
+ // rangeMins["special_joint"] = 0;
147
+ // rangeMaxes["special_joint"] = 2048;
148
+ // }
149
  }
150
 
151
  /**
152
+ * Main calibrate function - simple API, handles robot types internally
 
 
 
153
  */
154
  export async function calibrate(
155
  robotConnection: RobotConnection,
 
169
  const port = new WebSerialPortWrapper(robotConnection.port);
170
  await port.initialize();
171
 
172
+ // Get robot-specific configuration (extensible - add new robot types here)
173
+ let config: RobotHardwareConfig;
174
+ if (robotConnection.robotType.startsWith("so100")) {
175
+ config = createSO100Config(robotConnection.robotType);
176
+ } else {
177
+ throw new Error(`Unsupported robot type: ${robotConnection.robotType}`);
178
+ }
179
 
180
  let shouldStop = false;
181
  const stopFunction = () => shouldStop;
 
185
  // Step 1: Set homing offsets (automatic)
186
  options?.onProgress?.("⚙️ Setting motor homing offsets");
187
  const homingOffsets = await setHomingOffsets(
188
+ port,
189
  config.motorIds,
190
  config.motorNames
191
  );
192
 
193
  // Step 2: Record ranges of motion with live updates
194
  const { rangeMins, rangeMaxes } = await recordRangesOfMotion(
195
+ port,
196
  config.motorIds,
197
  config.motorNames,
198
  stopFunction,
199
  options?.onLiveUpdate
200
  );
201
 
202
+ // Step 3: Apply robot-specific range adjustments
203
+ applyRobotSpecificRangeAdjustments(
204
+ robotConnection.robotType!,
205
+ config.protocol,
206
+ rangeMins,
207
+ rangeMaxes
208
+ );
209
 
210
  // Step 4: Write hardware position limits to motors
211
  await writeHardwarePositionLimits(
212
+ port,
213
  config.motorIds,
214
  config.motorNames,
215
  rangeMins,
src/lerobot/web/find_port.ts CHANGED
@@ -29,30 +29,15 @@
29
  */
30
 
31
  import { getRobotConnectionManager } from "./robot-connection.js";
 
 
 
 
 
32
 
33
  /**
34
- * Type definitions for WebSerial API (not yet in all TypeScript libs)
35
  */
36
- interface SerialPort {
37
- readonly readable: ReadableStream;
38
- readonly writable: WritableStream;
39
- getInfo(): SerialPortInfo;
40
- open(options: SerialOptions): Promise<void>;
41
- close(): Promise<void>;
42
- }
43
-
44
- interface SerialPortInfo {
45
- usbVendorId?: number;
46
- usbProductId?: number;
47
- }
48
-
49
- interface SerialOptions {
50
- baudRate: number;
51
- dataBits?: number;
52
- stopBits?: number;
53
- parity?: "none" | "even" | "odd";
54
- }
55
-
56
  interface Serial extends EventTarget {
57
  getPorts(): Promise<SerialPort[]>;
58
  requestPort(options?: SerialPortRequestOptions): Promise<SerialPort>;
@@ -73,43 +58,6 @@ declare global {
73
  }
74
  }
75
 
76
- /**
77
- * Unified robot connection interface used across all functions
78
- * This same object works for findPort, calibrate, teleoperate, etc.
79
- * Includes all fields needed by demo and other applications
80
- */
81
- export interface RobotConnection {
82
- port: SerialPort;
83
- name: string; // Display name for UI
84
- isConnected: boolean; // Connection status
85
- robotType?: "so100_follower" | "so100_leader"; // Optional until user configures
86
- robotId?: string; // Optional until user configures
87
- serialNumber: string; // Always required for identification
88
- error?: string; // Error message if connection failed
89
- usbMetadata?: {
90
- // USB device information
91
- vendorId: string;
92
- productId: string;
93
- serialNumber: string;
94
- manufacturerName: string;
95
- productName: string;
96
- usbVersionMajor?: number;
97
- usbVersionMinor?: number;
98
- deviceClass?: number;
99
- deviceSubclass?: number;
100
- deviceProtocol?: number;
101
- };
102
- }
103
-
104
- /**
105
- * Minimal robot config for finding/connecting to specific robots
106
- */
107
- export interface RobotConfig {
108
- robotType: "so100_follower" | "so100_leader";
109
- robotId: string;
110
- serialNumber: string;
111
- }
112
-
113
  /**
114
  * Options for findPort function
115
  */
@@ -366,6 +314,3 @@ export async function findPort(
366
  },
367
  };
368
  }
369
-
370
- // Export the main function (renamed from findPortWeb)
371
- export { findPort as findPortWeb }; // Backward compatibility alias
 
29
  */
30
 
31
  import { getRobotConnectionManager } from "./robot-connection.js";
32
+ import type {
33
+ RobotConnection,
34
+ RobotConfig,
35
+ SerialPort,
36
+ } from "./types/robot-connection.js";
37
 
38
  /**
39
+ * Extended WebSerial API type definitions
40
  */
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  interface Serial extends EventTarget {
42
  getPorts(): Promise<SerialPort[]>;
43
  requestPort(options?: SerialPortRequestOptions): Promise<SerialPort>;
 
58
  }
59
  }
60
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
  /**
62
  * Options for findPort function
63
  */
 
314
  },
315
  };
316
  }
 
 
 
src/lerobot/web/robots/so100_config.ts CHANGED
@@ -1,8 +1,10 @@
1
  /**
2
- * SO-100 specific configuration for web calibration
3
  * Matches Node.js SO-100 config structure and Python lerobot exactly
4
  */
5
 
 
 
6
  /**
7
  * STS3215 Protocol Configuration for SO-100 devices
8
  */
@@ -38,16 +40,60 @@ export const SO100_CONFIG = {
38
  };
39
 
40
  /**
41
- * Create SO-100 calibration configuration
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  */
43
  export function createSO100Config(
44
  deviceType: "so100_follower" | "so100_leader"
45
- ) {
46
  return {
47
  deviceType,
48
  motorNames: SO100_CONFIG.motorNames,
49
  motorIds: SO100_CONFIG.motorIds,
50
  driveModes: SO100_CONFIG.driveModes,
 
51
  protocol: WEB_STS3215_PROTOCOL,
52
  };
53
  }
 
1
  /**
2
+ * SO-100 specific hardware configuration
3
  * Matches Node.js SO-100 config structure and Python lerobot exactly
4
  */
5
 
6
+ import type { RobotHardwareConfig } from "../types/robot-config.js";
7
+
8
  /**
9
  * STS3215 Protocol Configuration for SO-100 devices
10
  */
 
40
  };
41
 
42
  /**
43
+ * SO-100 Keyboard Controls for Teleoperation
44
+ * Robot-specific mapping optimized for SO-100 joint layout
45
+ */
46
+ export const SO100_KEYBOARD_CONTROLS = {
47
+ // Shoulder controls
48
+ ArrowUp: { motor: "shoulder_lift", direction: 1, description: "Shoulder up" },
49
+ ArrowDown: {
50
+ motor: "shoulder_lift",
51
+ direction: -1,
52
+ description: "Shoulder down",
53
+ },
54
+ ArrowLeft: {
55
+ motor: "shoulder_pan",
56
+ direction: -1,
57
+ description: "Shoulder left",
58
+ },
59
+ ArrowRight: {
60
+ motor: "shoulder_pan",
61
+ direction: 1,
62
+ description: "Shoulder right",
63
+ },
64
+
65
+ // WASD controls
66
+ w: { motor: "elbow_flex", direction: 1, description: "Elbow flex" },
67
+ s: { motor: "elbow_flex", direction: -1, description: "Elbow extend" },
68
+ a: { motor: "wrist_flex", direction: -1, description: "Wrist down" },
69
+ d: { motor: "wrist_flex", direction: 1, description: "Wrist up" },
70
+
71
+ // Wrist roll and gripper
72
+ q: { motor: "wrist_roll", direction: -1, description: "Wrist roll left" },
73
+ e: { motor: "wrist_roll", direction: 1, description: "Wrist roll right" },
74
+ o: { motor: "gripper", direction: 1, description: "Gripper open" },
75
+ c: { motor: "gripper", direction: -1, description: "Gripper close" },
76
+
77
+ // Emergency stop
78
+ Escape: {
79
+ motor: "emergency_stop",
80
+ direction: 0,
81
+ description: "Emergency stop",
82
+ },
83
+ } as const;
84
+
85
+ /**
86
+ * Create SO-100 hardware configuration
87
  */
88
  export function createSO100Config(
89
  deviceType: "so100_follower" | "so100_leader"
90
+ ): RobotHardwareConfig {
91
  return {
92
  deviceType,
93
  motorNames: SO100_CONFIG.motorNames,
94
  motorIds: SO100_CONFIG.motorIds,
95
  driveModes: SO100_CONFIG.driveModes,
96
+ keyboardControls: SO100_KEYBOARD_CONTROLS,
97
  protocol: WEB_STS3215_PROTOCOL,
98
  };
99
  }
src/lerobot/web/teleoperate.ts CHANGED
@@ -3,7 +3,18 @@
3
  * Mirrors the Node.js implementation but adapted for browser environment
4
  */
5
 
6
- import type { UnifiedRobotData } from "../../demo/lib/unified-storage.js";
 
 
 
 
 
 
 
 
 
 
 
7
 
8
  /**
9
  * Motor position and limits for teleoperation
@@ -14,7 +25,6 @@ export interface MotorConfig {
14
  currentPosition: number;
15
  minPosition: number;
16
  maxPosition: number;
17
- homePosition: number;
18
  }
19
 
20
  /**
@@ -28,328 +38,103 @@ export interface TeleoperationState {
28
  }
29
 
30
  /**
31
- * Keyboard control mapping (matches Node.js version)
32
  */
33
- export const KEYBOARD_CONTROLS = {
34
- // Shoulder controls
35
- ArrowUp: { motor: "shoulder_lift", direction: 1, description: "Shoulder up" },
36
- ArrowDown: {
37
- motor: "shoulder_lift",
38
- direction: -1,
39
- description: "Shoulder down",
40
- },
41
- ArrowLeft: {
42
- motor: "shoulder_pan",
43
- direction: -1,
44
- description: "Shoulder left",
45
- },
46
- ArrowRight: {
47
- motor: "shoulder_pan",
48
- direction: 1,
49
- description: "Shoulder right",
50
- },
51
-
52
- // WASD controls
53
- w: { motor: "elbow_flex", direction: 1, description: "Elbow flex" },
54
- s: { motor: "elbow_flex", direction: -1, description: "Elbow extend" },
55
- a: { motor: "wrist_flex", direction: -1, description: "Wrist down" },
56
- d: { motor: "wrist_flex", direction: 1, description: "Wrist up" },
57
-
58
- // Wrist roll and gripper
59
- q: { motor: "wrist_roll", direction: -1, description: "Wrist roll left" },
60
- e: { motor: "wrist_roll", direction: 1, description: "Wrist roll right" },
61
- o: { motor: "gripper", direction: 1, description: "Gripper open" },
62
- c: { motor: "gripper", direction: -1, description: "Gripper close" },
63
-
64
- // Emergency stop
65
- Escape: {
66
- motor: "emergency_stop",
67
- direction: 0,
68
- description: "Emergency stop",
69
- },
70
- } as const;
71
 
72
  /**
73
- * Web Serial Port wrapper for teleoperation
74
- * Uses the same pattern as calibration - per-operation reader/writer access
75
  */
76
- class WebTeleoperationPort {
77
- private port: SerialPort;
78
-
79
- constructor(port: SerialPort) {
80
- this.port = port;
81
- }
82
-
83
- get isOpen(): boolean {
84
- return (
85
- this.port !== null &&
86
- this.port.readable !== null &&
87
- this.port.writable !== null
88
- );
89
- }
90
-
91
- async initialize(): Promise<void> {
92
- if (!this.port.readable || !this.port.writable) {
93
- throw new Error("Port is not open for teleoperation");
94
- }
95
- // Port is already open and ready - no need to grab persistent readers/writers
96
- }
97
-
98
- async writeMotorPosition(
99
- motorId: number,
100
- position: number
101
- ): Promise<boolean> {
102
- if (!this.port.writable) {
103
- throw new Error("Port not open for writing");
104
- }
105
-
106
- try {
107
- // STS3215 Write Goal_Position packet (matches Node.js exactly)
108
- const packet = new Uint8Array([
109
- 0xff,
110
- 0xff, // Header
111
- motorId, // Servo ID
112
- 0x05, // Length
113
- 0x03, // Instruction: WRITE_DATA
114
- 42, // Goal_Position register address
115
- position & 0xff, // Position low byte
116
- (position >> 8) & 0xff, // Position high byte
117
- 0x00, // Checksum placeholder
118
- ]);
119
-
120
- // Calculate checksum
121
- const checksum =
122
- ~(
123
- motorId +
124
- 0x05 +
125
- 0x03 +
126
- 42 +
127
- (position & 0xff) +
128
- ((position >> 8) & 0xff)
129
- ) & 0xff;
130
- packet[8] = checksum;
131
-
132
- // Use per-operation writer like calibration does
133
- const writer = this.port.writable.getWriter();
134
- try {
135
- await writer.write(packet);
136
- return true;
137
- } finally {
138
- writer.releaseLock();
139
- }
140
- } catch (error) {
141
- console.warn(`Failed to write motor ${motorId} position:`, error);
142
- return false;
143
- }
144
- }
145
-
146
- async readMotorPosition(motorId: number): Promise<number | null> {
147
- if (!this.port.writable || !this.port.readable) {
148
- throw new Error("Port not open for reading/writing");
149
- }
150
-
151
- const writer = this.port.writable.getWriter();
152
- const reader = this.port.readable.getReader();
153
-
154
- try {
155
- // STS3215 Read Present_Position packet
156
- const packet = new Uint8Array([
157
- 0xff,
158
- 0xff, // Header
159
- motorId, // Servo ID
160
- 0x04, // Length
161
- 0x02, // Instruction: READ_DATA
162
- 56, // Present_Position register address
163
- 0x02, // Data length (2 bytes)
164
- 0x00, // Checksum placeholder
165
- ]);
166
-
167
- const checksum = ~(motorId + 0x04 + 0x02 + 56 + 0x02) & 0xff;
168
- packet[7] = checksum;
169
-
170
- // Clear buffer first
171
- try {
172
- const { value, done } = await reader.read();
173
- if (done) return null;
174
- } catch (e) {
175
- // Buffer was empty, continue
176
- }
177
-
178
- await writer.write(packet);
179
- await new Promise((resolve) => setTimeout(resolve, 10));
180
-
181
- const { value: response, done } = await reader.read();
182
- if (done || !response || response.length < 7) {
183
- return null;
184
- }
185
-
186
- const id = response[2];
187
- const error = response[4];
188
-
189
- if (id === motorId && error === 0) {
190
- return response[5] | (response[6] << 8);
191
- }
192
-
193
- return null;
194
- } catch (error) {
195
- console.warn(`Failed to read motor ${motorId} position:`, error);
196
- return null;
197
- } finally {
198
- reader.releaseLock();
199
- writer.releaseLock();
200
- }
201
- }
202
-
203
- async disconnect(): Promise<void> {
204
- // Don't close the port itself - just cleanup wrapper
205
- // The port is managed by PortManager
206
- }
207
  }
208
 
209
  /**
210
- * Load calibration data from unified storage with fallback to defaults
211
- * Improved version that properly loads and applies calibration ranges
212
  */
213
- export function loadCalibrationConfig(serialNumber: string): MotorConfig[] {
214
- // Default SO-100 configuration (matches Node.js defaults)
215
- const defaultConfigs: MotorConfig[] = [
216
- {
217
- id: 1,
218
- name: "shoulder_pan",
219
- currentPosition: 2048,
220
- minPosition: 1024,
221
- maxPosition: 3072,
222
- homePosition: 2048,
223
- },
224
- {
225
- id: 2,
226
- name: "shoulder_lift",
227
- currentPosition: 2048,
228
- minPosition: 1024,
229
- maxPosition: 3072,
230
- homePosition: 2048,
231
- },
232
- {
233
- id: 3,
234
- name: "elbow_flex",
235
- currentPosition: 2048,
236
- minPosition: 1024,
237
- maxPosition: 3072,
238
- homePosition: 2048,
239
- },
240
- {
241
- id: 4,
242
- name: "wrist_flex",
243
- currentPosition: 2048,
244
- minPosition: 1024,
245
- maxPosition: 3072,
246
- homePosition: 2048,
247
- },
248
- {
249
- id: 5,
250
- name: "wrist_roll",
251
- currentPosition: 2048,
252
- minPosition: 1024,
253
- maxPosition: 3072,
254
- homePosition: 2048,
255
- },
256
- {
257
- id: 6,
258
- name: "gripper",
259
- currentPosition: 2048,
260
- minPosition: 1024,
261
- maxPosition: 3072,
262
- homePosition: 2048,
263
- },
264
- ];
265
-
266
- try {
267
- // Load from unified storage
268
- const unifiedKey = `lerobotjs-${serialNumber}`;
269
- const unifiedDataRaw = localStorage.getItem(unifiedKey);
270
-
271
- if (!unifiedDataRaw) {
272
- console.log(
273
- `No calibration data found for ${serialNumber}, using defaults`
274
- );
275
- return defaultConfigs;
276
- }
277
-
278
- const unifiedData: UnifiedRobotData = JSON.parse(unifiedDataRaw);
279
-
280
- if (!unifiedData.calibration) {
281
- console.log(
282
- `No calibration in unified data for ${serialNumber}, using defaults`
283
- );
284
- return defaultConfigs;
285
  }
286
 
287
- // Map calibration data to motor configs
288
- const calibratedConfigs: MotorConfig[] = defaultConfigs.map(
289
- (defaultConfig) => {
290
- const calibData = (unifiedData.calibration as any)?.[
291
- defaultConfig.name
292
- ];
293
-
294
- if (
295
- calibData &&
296
- typeof calibData === "object" &&
297
- "id" in calibData &&
298
- "range_min" in calibData &&
299
- "range_max" in calibData
300
- ) {
301
- // Use calibrated values but keep current position as default
302
- return {
303
- ...defaultConfig,
304
- id: calibData.id,
305
- minPosition: calibData.range_min,
306
- maxPosition: calibData.range_max,
307
- homePosition: Math.floor(
308
- (calibData.range_min + calibData.range_max) / 2
309
- ),
310
- };
311
- }
312
-
313
- return defaultConfig;
314
- }
315
- );
316
-
317
- console.log(`✅ Loaded calibration data for ${serialNumber}`);
318
- return calibratedConfigs;
319
- } catch (error) {
320
- console.warn(`Failed to load calibration for ${serialNumber}:`, error);
321
- return defaultConfigs;
322
- }
323
  }
324
 
325
  /**
326
  * Web teleoperation controller
 
327
  */
328
  export class WebTeleoperationController {
329
- private port: WebTeleoperationPort;
330
  private motorConfigs: MotorConfig[] = [];
 
331
  private isActive: boolean = false;
332
  private updateInterval: NodeJS.Timeout | null = null;
333
  private keyStates: {
334
  [key: string]: { pressed: boolean; timestamp: number };
335
  } = {};
 
336
 
337
  // Movement parameters (matches Node.js)
338
  private readonly STEP_SIZE = 8;
339
  private readonly UPDATE_RATE = 60; // 60 FPS
340
- private readonly KEY_TIMEOUT = 100; // ms
341
-
342
- constructor(port: SerialPort, serialNumber: string) {
343
- this.port = new WebTeleoperationPort(port);
344
- this.motorConfigs = loadCalibrationConfig(serialNumber);
 
 
 
 
 
 
 
345
  }
346
 
347
  async initialize(): Promise<void> {
348
- await this.port.initialize();
349
-
350
- // Read current positions
351
  for (const config of this.motorConfigs) {
352
- const position = await this.port.readMotorPosition(config.id);
353
  if (position !== null) {
354
  config.currentPosition = position;
355
  }
@@ -401,11 +186,16 @@ export class WebTeleoperationController {
401
  this.keyStates = {};
402
 
403
  console.log("⏹️ Web teleoperation stopped");
 
 
 
 
 
404
  }
405
 
406
  async disconnect(): Promise<void> {
407
  this.stop();
408
- await this.port.disconnect();
409
  }
410
 
411
  private updateMotorPositions(): void {
@@ -435,7 +225,7 @@ export class WebTeleoperationController {
435
  const targetPositions: { [motorName: string]: number } = {};
436
 
437
  for (const key of activeKeys) {
438
- const control = KEYBOARD_CONTROLS[key as keyof typeof KEYBOARD_CONTROLS];
439
  if (!control || control.motor === "emergency_stop") continue;
440
 
441
  const motorConfig = this.motorConfigs.find(
@@ -455,16 +245,23 @@ export class WebTeleoperationController {
455
  );
456
  }
457
 
458
- // Send motor commands
459
  Object.entries(targetPositions).forEach(([motorName, targetPosition]) => {
460
  const motorConfig = this.motorConfigs.find((m) => m.name === motorName);
461
  if (motorConfig && targetPosition !== motorConfig.currentPosition) {
462
- this.port
463
- .writeMotorPosition(motorConfig.id, Math.round(targetPosition))
464
- .then((success) => {
465
- if (success) {
466
- motorConfig.currentPosition = targetPosition;
467
- }
 
 
 
 
 
 
 
468
  });
469
  }
470
  });
@@ -480,15 +277,18 @@ export class WebTeleoperationController {
480
  Math.min(motorConfig.maxPosition, targetPosition)
481
  );
482
 
483
- const success = await this.port.writeMotorPosition(
484
- motorConfig.id,
485
- Math.round(clampedPosition)
486
- );
487
- if (success) {
 
488
  motorConfig.currentPosition = clampedPosition;
 
 
 
 
489
  }
490
-
491
- return success;
492
  }
493
 
494
  async setMotorPositions(positions: {
@@ -502,25 +302,81 @@ export class WebTeleoperationController {
502
 
503
  return results.every((result) => result);
504
  }
505
-
506
- async goToHomePosition(): Promise<boolean> {
507
- const homePositions = this.motorConfigs.reduce((acc, config) => {
508
- acc[config.name] = config.homePosition;
509
- return acc;
510
- }, {} as { [motorName: string]: number });
511
-
512
- return this.setMotorPositions(homePositions);
513
- }
514
  }
515
 
516
  /**
517
- * Create teleoperation controller for connected robot
 
518
  */
519
- export async function createWebTeleoperationController(
520
- port: SerialPort,
521
- serialNumber: string
522
- ): Promise<WebTeleoperationController> {
523
- const controller = new WebTeleoperationController(port, serialNumber);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
524
  await controller.initialize();
525
- return controller;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
526
  }
 
3
  * Mirrors the Node.js implementation but adapted for browser environment
4
  */
5
 
6
+ import { createSO100Config } from "./robots/so100_config.js";
7
+ import type {
8
+ RobotHardwareConfig,
9
+ KeyboardControl,
10
+ } from "./types/robot-config.js";
11
+ import type { RobotConnection } from "./types/robot-connection.js";
12
+ import { WebSerialPortWrapper } from "./utils/serial-port-wrapper.js";
13
+ import {
14
+ readMotorPosition,
15
+ writeMotorPosition,
16
+ type MotorCommunicationPort,
17
+ } from "./utils/motor-communication.js";
18
 
19
  /**
20
  * Motor position and limits for teleoperation
 
25
  currentPosition: number;
26
  minPosition: number;
27
  maxPosition: number;
 
28
  }
29
 
30
  /**
 
38
  }
39
 
40
  /**
41
+ * Teleoperation process control object (matches calibrate pattern)
42
  */
43
+ export interface TeleoperationProcess {
44
+ start(): void;
45
+ stop(): void;
46
+ updateKeyState(key: string, pressed: boolean): void;
47
+ getState(): TeleoperationState;
48
+ moveMotor(motorName: string, position: number): Promise<boolean>;
49
+ setMotorPositions(positions: {
50
+ [motorName: string]: number;
51
+ }): Promise<boolean>;
52
+ disconnect(): Promise<void>;
53
+ }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
 
55
  /**
56
+ * Create motor configurations from robot hardware config
57
+ * Pure function - converts robot specs to motor configs with defaults
58
  */
59
+ function createMotorConfigsFromRobotConfig(
60
+ robotConfig: RobotHardwareConfig
61
+ ): MotorConfig[] {
62
+ return robotConfig.motorNames.map((name: string, i: number) => ({
63
+ id: robotConfig.motorIds[i],
64
+ name,
65
+ currentPosition: 2048,
66
+ minPosition: 1024,
67
+ maxPosition: 3072,
68
+ }));
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
  }
70
 
71
  /**
72
+ * Apply calibration data to motor configurations
73
+ * Pure function - takes calibration data as parameter
74
  */
75
+ export function applyCalibrationToMotorConfigs(
76
+ defaultConfigs: MotorConfig[],
77
+ calibrationData: { [motorName: string]: any }
78
+ ): MotorConfig[] {
79
+ return defaultConfigs.map((defaultConfig) => {
80
+ const calibData = calibrationData[defaultConfig.name];
81
+
82
+ if (
83
+ calibData &&
84
+ typeof calibData === "object" &&
85
+ "id" in calibData &&
86
+ "range_min" in calibData &&
87
+ "range_max" in calibData
88
+ ) {
89
+ // Use calibrated values but keep current position as default
90
+ return {
91
+ ...defaultConfig,
92
+ id: calibData.id,
93
+ minPosition: calibData.range_min,
94
+ maxPosition: calibData.range_max,
95
+ };
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
96
  }
97
 
98
+ return defaultConfig;
99
+ });
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
100
  }
101
 
102
  /**
103
  * Web teleoperation controller
104
+ * Now uses shared utilities instead of custom port handling
105
  */
106
  export class WebTeleoperationController {
107
+ private port: MotorCommunicationPort;
108
  private motorConfigs: MotorConfig[] = [];
109
+ private keyboardControls: { [key: string]: KeyboardControl } = {};
110
  private isActive: boolean = false;
111
  private updateInterval: NodeJS.Timeout | null = null;
112
  private keyStates: {
113
  [key: string]: { pressed: boolean; timestamp: number };
114
  } = {};
115
+ private onStateUpdate?: (state: TeleoperationState) => void;
116
 
117
  // Movement parameters (matches Node.js)
118
  private readonly STEP_SIZE = 8;
119
  private readonly UPDATE_RATE = 60; // 60 FPS
120
+ private readonly KEY_TIMEOUT = 600; // ms - longer than browser keyboard repeat delay (~500ms)
121
+
122
+ constructor(
123
+ port: MotorCommunicationPort,
124
+ motorConfigs: MotorConfig[],
125
+ keyboardControls: { [key: string]: KeyboardControl },
126
+ onStateUpdate?: (state: TeleoperationState) => void
127
+ ) {
128
+ this.port = port;
129
+ this.motorConfigs = motorConfigs;
130
+ this.keyboardControls = keyboardControls;
131
+ this.onStateUpdate = onStateUpdate;
132
  }
133
 
134
  async initialize(): Promise<void> {
135
+ // Read current positions using proven utilities
 
 
136
  for (const config of this.motorConfigs) {
137
+ const position = await readMotorPosition(this.port, config.id);
138
  if (position !== null) {
139
  config.currentPosition = position;
140
  }
 
186
  this.keyStates = {};
187
 
188
  console.log("⏹️ Web teleoperation stopped");
189
+
190
+ // Notify UI of state change
191
+ if (this.onStateUpdate) {
192
+ this.onStateUpdate(this.getState());
193
+ }
194
  }
195
 
196
  async disconnect(): Promise<void> {
197
  this.stop();
198
+ // No need to manually disconnect - port wrapper handles this
199
  }
200
 
201
  private updateMotorPositions(): void {
 
225
  const targetPositions: { [motorName: string]: number } = {};
226
 
227
  for (const key of activeKeys) {
228
+ const control = this.keyboardControls[key];
229
  if (!control || control.motor === "emergency_stop") continue;
230
 
231
  const motorConfig = this.motorConfigs.find(
 
245
  );
246
  }
247
 
248
+ // Send motor commands using proven utilities
249
  Object.entries(targetPositions).forEach(([motorName, targetPosition]) => {
250
  const motorConfig = this.motorConfigs.find((m) => m.name === motorName);
251
  if (motorConfig && targetPosition !== motorConfig.currentPosition) {
252
+ writeMotorPosition(
253
+ this.port,
254
+ motorConfig.id,
255
+ Math.round(targetPosition)
256
+ )
257
+ .then(() => {
258
+ motorConfig.currentPosition = targetPosition;
259
+ })
260
+ .catch((error) => {
261
+ console.warn(
262
+ `Failed to write motor ${motorConfig.id} position:`,
263
+ error
264
+ );
265
  });
266
  }
267
  });
 
277
  Math.min(motorConfig.maxPosition, targetPosition)
278
  );
279
 
280
+ try {
281
+ await writeMotorPosition(
282
+ this.port,
283
+ motorConfig.id,
284
+ Math.round(clampedPosition)
285
+ );
286
  motorConfig.currentPosition = clampedPosition;
287
+ return true;
288
+ } catch (error) {
289
+ console.warn(`Failed to move motor ${motorName}:`, error);
290
+ return false;
291
  }
 
 
292
  }
293
 
294
  async setMotorPositions(positions: {
 
302
 
303
  return results.every((result) => result);
304
  }
 
 
 
 
 
 
 
 
 
305
  }
306
 
307
  /**
308
+ * Main teleoperate function - simple API matching calibrate pattern
309
+ * Handles robot types internally, creates appropriate motor configurations
310
  */
311
+ export async function teleoperate(
312
+ robotConnection: RobotConnection,
313
+ options?: {
314
+ calibrationData?: { [motorName: string]: any };
315
+ onStateUpdate?: (state: TeleoperationState) => void;
316
+ }
317
+ ): Promise<TeleoperationProcess> {
318
+ // Validate required fields
319
+ if (!robotConnection.robotType) {
320
+ throw new Error(
321
+ "Robot type is required for teleoperation. Please configure the robot first."
322
+ );
323
+ }
324
+
325
+ // Create web serial port wrapper (same pattern as calibrate.ts)
326
+ const port = new WebSerialPortWrapper(robotConnection.port);
327
+ await port.initialize();
328
+
329
+ // Get robot-specific configuration (same pattern as calibrate.ts)
330
+ let config: RobotHardwareConfig;
331
+ if (robotConnection.robotType.startsWith("so100")) {
332
+ config = createSO100Config(robotConnection.robotType);
333
+ } else {
334
+ throw new Error(`Unsupported robot type: ${robotConnection.robotType}`);
335
+ }
336
+
337
+ // Create motor configs from robot hardware specs (single call, no duplication)
338
+ const defaultMotorConfigs = createMotorConfigsFromRobotConfig(config);
339
+
340
+ // Apply calibration data if provided
341
+ const motorConfigs = options?.calibrationData
342
+ ? applyCalibrationToMotorConfigs(
343
+ defaultMotorConfigs,
344
+ options.calibrationData
345
+ )
346
+ : defaultMotorConfigs;
347
+
348
+ // Create and initialize controller using shared utilities
349
+ const controller = new WebTeleoperationController(
350
+ port,
351
+ motorConfigs,
352
+ config.keyboardControls,
353
+ options?.onStateUpdate
354
+ );
355
  await controller.initialize();
356
+
357
+ // Wrap controller in process object (matches calibrate pattern)
358
+ return {
359
+ start: () => {
360
+ controller.start();
361
+ // Optional state update callback
362
+ if (options?.onStateUpdate) {
363
+ const updateLoop = () => {
364
+ if (controller.getState().isActive) {
365
+ options.onStateUpdate!(controller.getState());
366
+ setTimeout(updateLoop, 100); // 10fps state updates
367
+ }
368
+ };
369
+ updateLoop();
370
+ }
371
+ },
372
+ stop: () => controller.stop(),
373
+ updateKeyState: (key: string, pressed: boolean) =>
374
+ controller.updateKeyState(key, pressed),
375
+ getState: () => controller.getState(),
376
+ moveMotor: (motorName: string, position: number) =>
377
+ controller.moveMotor(motorName, position),
378
+ setMotorPositions: (positions: { [motorName: string]: number }) =>
379
+ controller.setMotorPositions(positions),
380
+ disconnect: () => controller.disconnect(),
381
+ };
382
  }
src/lerobot/web/types/robot-config.ts ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * Shared robot hardware configuration types
3
+ * Used across calibration, teleoperation, and other robot operations
4
+ */
5
+
6
+ /**
7
+ * Keyboard control mapping for teleoperation
8
+ */
9
+ export interface KeyboardControl {
10
+ motor: string;
11
+ direction: number;
12
+ description: string;
13
+ }
14
+
15
+ /**
16
+ * Robot hardware configuration interface
17
+ * Defines the contract that all robot configurations must implement
18
+ */
19
+ export interface RobotHardwareConfig {
20
+ deviceType: string;
21
+ motorNames: string[];
22
+ motorIds: number[];
23
+ driveModes: number[];
24
+
25
+ // Keyboard controls for teleoperation (robot-specific)
26
+ keyboardControls: { [key: string]: KeyboardControl };
27
+
28
+ protocol: {
29
+ resolution: number;
30
+ homingOffsetAddress: number;
31
+ homingOffsetLength: number;
32
+ presentPositionAddress: number;
33
+ presentPositionLength: number;
34
+ minPositionLimitAddress: number;
35
+ minPositionLimitLength: number;
36
+ maxPositionLimitAddress: number;
37
+ maxPositionLimitLength: number;
38
+ signMagnitudeBit: number;
39
+ };
40
+ }
src/lerobot/web/types/robot-connection.ts ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * Core robot connection types used across the lerobot.js web library
3
+ * These types are shared between findPort, calibrate, teleoperate, and other modules
4
+ */
5
+
6
+ /**
7
+ * Type definitions for WebSerial API (not yet in all TypeScript libs)
8
+ */
9
+ export interface SerialPort {
10
+ readonly readable: ReadableStream;
11
+ readonly writable: WritableStream;
12
+ getInfo(): SerialPortInfo;
13
+ open(options: SerialOptions): Promise<void>;
14
+ close(): Promise<void>;
15
+ }
16
+
17
+ export interface SerialPortInfo {
18
+ usbVendorId?: number;
19
+ usbProductId?: number;
20
+ }
21
+
22
+ export interface SerialOptions {
23
+ baudRate: number;
24
+ dataBits?: number;
25
+ stopBits?: number;
26
+ parity?: "none" | "even" | "odd";
27
+ }
28
+
29
+ /**
30
+ * Unified robot connection interface used across all functions
31
+ * This same object works for findPort, calibrate, teleoperate, etc.
32
+ * Includes all fields needed by demo and other applications
33
+ */
34
+ export interface RobotConnection {
35
+ port: SerialPort;
36
+ name: string; // Display name for UI
37
+ isConnected: boolean; // Connection status
38
+ robotType?: "so100_follower" | "so100_leader"; // Optional until user configures
39
+ robotId?: string; // Optional until user configures
40
+ serialNumber: string; // Always required for identification
41
+ error?: string; // Error message if connection failed
42
+ usbMetadata?: {
43
+ // USB device information
44
+ vendorId: string;
45
+ productId: string;
46
+ serialNumber: string;
47
+ manufacturerName: string;
48
+ productName: string;
49
+ usbVersionMajor?: number;
50
+ usbVersionMinor?: number;
51
+ deviceClass?: number;
52
+ deviceSubclass?: number;
53
+ deviceProtocol?: number;
54
+ };
55
+ }
56
+
57
+ /**
58
+ * Minimal robot config for finding/connecting to specific robots
59
+ */
60
+ export interface RobotConfig {
61
+ robotType: "so100_follower" | "so100_leader";
62
+ robotId: string;
63
+ serialNumber: string;
64
+ }