nbugs commited on
Commit
d119685
·
verified ·
1 Parent(s): 933511b

Upload 12 files

Browse files
Files changed (12) hide show
  1. Dockerfile +26 -0
  2. README.md +92 -5
  3. app.py +643 -0
  4. app.py.dockerignore +23 -0
  5. docker-compose.yml +14 -0
  6. dockerignore +22 -0
  7. env.example +7 -0
  8. gitattributes +35 -0
  9. gitignore +43 -0
  10. requirements.txt +5 -0
  11. run.sh +7 -0
  12. static/index.html +104 -0
Dockerfile ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM python:3.11-slim
2
+
3
+ WORKDIR /app
4
+
5
+ # Copy requirements first for better caching
6
+ COPY requirements.txt .
7
+
8
+ # Install dependencies
9
+ RUN pip install --no-cache-dir -r requirements.txt
10
+
11
+ # Copy application code
12
+ COPY . .
13
+
14
+ # Make run script executable
15
+ RUN chmod +x run.sh
16
+
17
+ # Expose port
18
+ EXPOSE 7860
19
+
20
+ # Environment variables
21
+ ENV PORT=7860
22
+ ENV API_BASE_URL=https://fragments.e2b.dev
23
+ ENV API_KEY=sk-123456
24
+
25
+ # Command to run the application
26
+ CMD ["./run.sh"]
README.md CHANGED
@@ -1,10 +1,97 @@
1
  ---
2
- title: Eb2
3
- emoji: 😻
4
- colorFrom: green
5
- colorTo: green
6
  sdk: docker
7
  pinned: false
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: eb2
3
+ emoji: 🚀
4
+ colorFrom: blue
5
+ colorTo: indigo
6
  sdk: docker
7
  pinned: false
8
+ app_port: 7860
9
  ---
10
 
11
+ # E2B API Proxy with FastAPI
12
+
13
+ This project is a FastAPI implementation of an API proxy for E2B (fragments.e2b.dev). It provides a compatible interface for various AI model providers including OpenAI, Google, and Anthropic.
14
+
15
+ ## Description
16
+
17
+ The E2B API Proxy acts as a middleware between your application and the E2B service, providing:
18
+
19
+ - Proxy API requests to E2B service
20
+ - Support for multiple AI models (OpenAI, Google Vertex AI, Anthropic)
21
+ - Streaming and non-streaming response handling
22
+ - CORS support for cross-origin requests
23
+
24
+ ## Deployment on Hugging Face Spaces
25
+
26
+ This application is ready to be deployed on Hugging Face Spaces:
27
+
28
+ 1. Create a new Space on Hugging Face with Docker SDK
29
+ 2. Upload these files to your Space
30
+ 3. Set the environment variables in the Space settings:
31
+ - `API_KEY`: Your API key for authentication (default: sk-123456)
32
+ - `API_BASE_URL`: The base URL for the E2B service (default: https://fragments.e2b.dev)
33
+
34
+ ## API Endpoints
35
+
36
+ - `GET /hf/v1/models`: List available models
37
+ - `POST /hf/v1/chat/completions`: Send chat completion requests
38
+ - `GET /`: Root endpoint (health check)
39
+
40
+ ## Configuration
41
+
42
+ The main configuration is in the `app.py` file. You can customize:
43
+
44
+ - API key for authentication
45
+ - Base URL for the E2B service
46
+ - Model configurations
47
+ - Default headers for requests
48
+
49
+ ## Local Development
50
+
51
+ ### Prerequisites
52
+
53
+ - Docker and Docker Compose
54
+
55
+ ### Running the Application Locally
56
+
57
+ 1. Clone this repository
58
+ 2. Update the API key in docker-compose.yml (replace `sk-123456` with your actual key)
59
+ 3. Build and start the container:
60
+
61
+ ```bash
62
+ docker-compose up -d
63
+ ```
64
+
65
+ 4. The API will be available at http://localhost:7860
66
+
67
+ ### Testing the API
68
+
69
+ You can test the API using curl:
70
+
71
+ ```bash
72
+ # Get available models
73
+ curl http://localhost:7860/hf/v1/models
74
+
75
+ # Send a chat completion request
76
+ curl -X POST http://localhost:7860/hf/v1/chat/completions \
77
+ -H "Content-Type: application/json" \
78
+ -H "Authorization: Bearer sk-123456" \
79
+ -d '{
80
+ "model": "gpt-4o",
81
+ "messages": [
82
+ {"role": "user", "content": "Hello, how are you?"}
83
+ ]
84
+ }'
85
+ ```
86
+
87
+ ## Supported Models
88
+
89
+ The API supports various models from different providers:
90
+
91
+ - **OpenAI**: o1-preview, o3-mini, gpt-4o, gpt-4.5-preview, gpt-4-turbo
92
+ - **Google**: gemini-1.5-pro, gemini-2.5-pro-exp-03-25, gemini-exp-1121, gemini-2.0-flash-exp
93
+ - **Anthropic**: claude-3-5-sonnet-latest, claude-3-7-sonnet-latest, claude-3-5-haiku-latest
94
+
95
+ ## License
96
+
97
+ This project is open source and available under the MIT License.
app.py ADDED
@@ -0,0 +1,643 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import uuid
3
+ import time
4
+ import asyncio
5
+ import logging
6
+ from datetime import datetime
7
+ from typing import Dict, List, Optional, Union, Any
8
+
9
+ import httpx
10
+ from fastapi import FastAPI, Request, Response, Depends, HTTPException, status
11
+ from fastapi.middleware.cors import CORSMiddleware
12
+ from fastapi.responses import StreamingResponse, JSONResponse, HTMLResponse
13
+ from fastapi.staticfiles import StaticFiles
14
+ from pydantic import BaseModel, Field
15
+
16
+ # Configure logging
17
+ logging.basicConfig(
18
+ level=logging.INFO,
19
+ format='[%(asctime)s] %(levelname)s: %(message)s',
20
+ datefmt='%Y-%m-%dT%H:%M:%S'
21
+ )
22
+ logger = logging.getLogger(__name__)
23
+
24
+ # Import os for environment variables
25
+ import os
26
+
27
+ # Configuration constants
28
+ CONFIG = {
29
+ "API": {
30
+ "BASE_URL": os.environ.get("API_BASE_URL", "https://fragments.e2b.dev"),
31
+ "API_KEY": os.environ.get("API_KEY", "sk-123456") # Customize your own authentication key
32
+ },
33
+ "RETRY": {
34
+ "MAX_ATTEMPTS": 1,
35
+ "DELAY_BASE": 1000
36
+ },
37
+ "MODEL_CONFIG": {
38
+ "o1-preview": {
39
+ "id": "o1",
40
+ "provider": "OpenAI",
41
+ "providerId": "openai",
42
+ "name": "o1",
43
+ "multiModal": True,
44
+ "Systemprompt": "",
45
+ "opt_max": {
46
+ "temperatureMax": 2,
47
+ "max_tokensMax": 0,
48
+ "presence_penaltyMax": 2,
49
+ "frequency_penaltyMax": 2,
50
+ "top_pMax": 1,
51
+ "top_kMax": 500
52
+ }
53
+ },
54
+ "o3-mini": {
55
+ "id": "o3-mini",
56
+ "provider": "OpenAI",
57
+ "providerId": "openai",
58
+ "name": "o3 Mini",
59
+ "multiModal": True,
60
+ "Systemprompt": "",
61
+ "opt_max": {
62
+ "temperatureMax": 2,
63
+ "max_tokensMax": 4096,
64
+ "presence_penaltyMax": 2,
65
+ "frequency_penaltyMax": 2,
66
+ "top_pMax": 1,
67
+ "top_kMax": 500
68
+ }
69
+ },
70
+ "gpt-4o": {
71
+ "id": "gpt-4o",
72
+ "provider": "OpenAI",
73
+ "providerId": "openai",
74
+ "name": "GPT-4o",
75
+ "multiModal": True,
76
+ "Systemprompt": "",
77
+ "opt_max": {
78
+ "temperatureMax": 2,
79
+ "max_tokensMax": 16380,
80
+ "presence_penaltyMax": 2,
81
+ "frequency_penaltyMax": 2,
82
+ "top_pMax": 1,
83
+ "top_kMax": 500
84
+ }
85
+ },
86
+ "gpt-4.5-preview": {
87
+ "id": "gpt-4.5-preview",
88
+ "provider": "OpenAI",
89
+ "providerId": "openai",
90
+ "name": "GPT-4.5",
91
+ "multiModal": True,
92
+ "Systemprompt": "",
93
+ "opt_max": {
94
+ "temperatureMax": 2,
95
+ "max_tokensMax": 16380,
96
+ "presence_penaltyMax": 2,
97
+ "frequency_penaltyMax": 2,
98
+ "top_pMax": 1,
99
+ "top_kMax": 500
100
+ }
101
+ },
102
+ "gpt-4-turbo": {
103
+ "id": "gpt-4-turbo",
104
+ "provider": "OpenAI",
105
+ "providerId": "openai",
106
+ "name": "GPT-4 Turbo",
107
+ "multiModal": True,
108
+ "Systemprompt": "",
109
+ "opt_max": {
110
+ "temperatureMax": 2,
111
+ "max_tokensMax": 16380,
112
+ "presence_penaltyMax": 2,
113
+ "frequency_penaltyMax": 2,
114
+ "top_pMax": 1,
115
+ "top_kMax": 500
116
+ }
117
+ },
118
+ "gemini-1.5-pro": {
119
+ "id": "gemini-1.5-pro-002",
120
+ "provider": "Google Vertex AI",
121
+ "providerId": "vertex",
122
+ "name": "Gemini 1.5 Pro",
123
+ "multiModal": True,
124
+ "Systemprompt": "",
125
+ "opt_max": {
126
+ "temperatureMax": 2,
127
+ "max_tokensMax": 8192,
128
+ "presence_penaltyMax": 2,
129
+ "frequency_penaltyMax": 2,
130
+ "top_pMax": 1,
131
+ "top_kMax": 500
132
+ }
133
+ },
134
+ "gemini-2.5-pro-exp-03-25": {
135
+ "id": "gemini-2.5-pro-exp-03-25",
136
+ "provider": "Google Generative AI",
137
+ "providerId": "google",
138
+ "name": "Gemini 2.5 Pro Experimental 03-25",
139
+ "multiModal": True,
140
+ "Systemprompt": "",
141
+ "opt_max": {
142
+ "temperatureMax": 2,
143
+ "max_tokensMax": 8192,
144
+ "presence_penaltyMax": 2,
145
+ "frequency_penaltyMax": 2,
146
+ "top_pMax": 1,
147
+ "top_kMax": 40
148
+ }
149
+ },
150
+ "gemini-exp-1121": {
151
+ "id": "gemini-exp-1121",
152
+ "provider": "Google Generative AI",
153
+ "providerId": "google",
154
+ "name": "Gemini Experimental 1121",
155
+ "multiModal": True,
156
+ "Systemprompt": "",
157
+ "opt_max": {
158
+ "temperatureMax": 2,
159
+ "max_tokensMax": 8192,
160
+ "presence_penaltyMax": 2,
161
+ "frequency_penaltyMax": 2,
162
+ "top_pMax": 1,
163
+ "top_kMax": 40
164
+ }
165
+ },
166
+ "gemini-2.0-flash-exp": {
167
+ "id": "models/gemini-2.0-flash-exp",
168
+ "provider": "Google Generative AI",
169
+ "providerId": "google",
170
+ "name": "Gemini 2.0 Flash",
171
+ "multiModal": True,
172
+ "Systemprompt": "",
173
+ "opt_max": {
174
+ "temperatureMax": 2,
175
+ "max_tokensMax": 8192,
176
+ "presence_penaltyMax": 2,
177
+ "frequency_penaltyMax": 2,
178
+ "top_pMax": 1,
179
+ "top_kMax": 40
180
+ }
181
+ },
182
+ "claude-3-5-sonnet-latest": {
183
+ "id": "claude-3-5-sonnet-latest",
184
+ "provider": "Anthropic",
185
+ "providerId": "anthropic",
186
+ "name": "Claude 3.5 Sonnet",
187
+ "multiModal": True,
188
+ "Systemprompt": "",
189
+ "opt_max": {
190
+ "temperatureMax": 1,
191
+ "max_tokensMax": 8192,
192
+ "presence_penaltyMax": 2,
193
+ "frequency_penaltyMax": 2,
194
+ "top_pMax": 1,
195
+ "top_kMax": 500
196
+ }
197
+ },
198
+ "claude-3-7-sonnet-latest": {
199
+ "id": "claude-3-7-sonnet-latest",
200
+ "provider": "Anthropic",
201
+ "providerId": "anthropic",
202
+ "name": "Claude 3.7 Sonnet",
203
+ "multiModal": True,
204
+ "Systemprompt": "",
205
+ "opt_max": {
206
+ "temperatureMax": 1,
207
+ "max_tokensMax": 8192,
208
+ "presence_penaltyMax": 2,
209
+ "frequency_penaltyMax": 2,
210
+ "top_pMax": 1,
211
+ "top_kMax": 500
212
+ }
213
+ },
214
+ "claude-3-5-haiku-latest": {
215
+ "id": "claude-3-5-haiku-latest",
216
+ "provider": "Anthropic",
217
+ "providerId": "anthropic",
218
+ "name": "Claude 3.5 Haiku",
219
+ "multiModal": False,
220
+ "Systemprompt": "",
221
+ "opt_max": {
222
+ "temperatureMax": 1,
223
+ "max_tokensMax": 8192,
224
+ "presence_penaltyMax": 2,
225
+ "frequency_penaltyMax": 2,
226
+ "top_pMax": 1,
227
+ "top_kMax": 500
228
+ }
229
+ }
230
+ },
231
+ "DEFAULT_HEADERS": {
232
+ "accept": "*/*",
233
+ "accept-language": "zh-CN,zh;q=0.9",
234
+ "content-type": "application/json",
235
+ "priority": "u=1, i",
236
+ "sec-ch-ua": "\"Microsoft Edge\";v=\"131\", \"Chromium\";v=\"131\", \"Not_A Brand\";v=\"24\"",
237
+ "sec-ch-ua-mobile": "?0",
238
+ "sec-ch-ua-platform": "\"Windows\"",
239
+ "sec-fetch-dest": "empty",
240
+ "sec-fetch-mode": "cors",
241
+ "sec-fetch-site": "same-origin",
242
+ "Referer": "https://fragments.e2b.dev/",
243
+ "Referrer-Policy": "strict-origin-when-cross-origin"
244
+ },
245
+ "MODEL_PROMPT": "Chatting with users and starting role-playing, the most important thing is to pay attention to their latest messages, use only 'text' to output the chat text reply content generated for user messages, and finally output it in code"
246
+ }
247
+
248
+
249
+ # Utility functions
250
+ def generate_uuid():
251
+ """Generate a UUID v4 string."""
252
+ return str(uuid.uuid4())
253
+
254
+
255
+ async def config_opt(params: Dict[str, Any], model_config: Dict[str, Any]) -> Optional[Dict[str, Any]]:
256
+ """Constrain parameters based on model configuration."""
257
+ if not model_config.get("opt_max"):
258
+ return None
259
+
260
+ options_map = {
261
+ "temperature": "temperatureMax",
262
+ "max_tokens": "max_tokensMax",
263
+ "presence_penalty": "presence_penaltyMax",
264
+ "frequency_penalty": "frequency_penaltyMax",
265
+ "top_p": "top_pMax",
266
+ "top_k": "top_kMax"
267
+ }
268
+
269
+ constrained_params = {}
270
+ for key, value in params.items():
271
+ max_key = options_map.get(key)
272
+ if (max_key and
273
+ max_key in model_config["opt_max"] and
274
+ value is not None):
275
+ constrained_params[key] = min(value, model_config["opt_max"][max_key])
276
+
277
+ return constrained_params
278
+
279
+
280
+ # API client class
281
+ class ApiClient:
282
+ def __init__(self, model_id: str, request_id: str = ""):
283
+ if model_id not in CONFIG["MODEL_CONFIG"]:
284
+ raise ValueError(f"Unsupported model: {model_id}")
285
+ self.model_config = CONFIG["MODEL_CONFIG"][model_id]
286
+ self.request_id = request_id
287
+
288
+ def process_message_content(self, content: Any) -> Optional[str]:
289
+ """Process message content to extract text."""
290
+ if isinstance(content, str):
291
+ return content
292
+ if isinstance(content, list):
293
+ return "\n".join([item.get("text", "") for item in content if item.get("type") == "text"])
294
+ if isinstance(content, dict):
295
+ return content.get("text")
296
+ return None
297
+
298
+ async def prepare_chat_request(self, request: Dict[str, Any], config: Optional[Dict[str, Any]]) -> Dict[str, Any]:
299
+ """Prepare chat request for E2B API."""
300
+ logger.info(f"[{self.request_id}] Preparing chat request, model: {self.model_config['name']}, messages count: {len(request.get('messages', []))}")
301
+
302
+ opt_config = config or {"model": self.model_config["id"]}
303
+ transformed_messages = await self.transform_messages(request)
304
+
305
+ logger.info(f"[{self.request_id}] Transformed messages count: {len(transformed_messages)}")
306
+
307
+ return {
308
+ "userID": generate_uuid(),
309
+ "messages": transformed_messages,
310
+ "template": {
311
+ "text": {
312
+ "name": CONFIG["MODEL_PROMPT"],
313
+ "lib": [""],
314
+ "file": "pages/ChatWithUsers.txt",
315
+ "instructions": self.model_config["Systemprompt"],
316
+ "port": None
317
+ }
318
+ },
319
+ "model": {
320
+ "id": self.model_config["id"],
321
+ "provider": self.model_config["provider"],
322
+ "providerId": self.model_config["providerId"],
323
+ "name": self.model_config["name"],
324
+ "multiModal": self.model_config["multiModal"]
325
+ },
326
+ "config": opt_config
327
+ }
328
+
329
+ async def transform_messages(self, request: Dict[str, Any]) -> List[Dict[str, Any]]:
330
+ """Transform and merge messages for E2B API."""
331
+ messages = request.get("messages", [])
332
+ merged_messages = []
333
+
334
+ for current in messages:
335
+ current_content = self.process_message_content(current.get("content"))
336
+ if current_content is None:
337
+ continue
338
+
339
+ if (merged_messages and
340
+ current.get("role") == merged_messages[-1].get("role")):
341
+ last_content = self.process_message_content(merged_messages[-1].get("content"))
342
+ if last_content is not None:
343
+ merged_messages[-1]["content"] = f"{last_content}\n{current_content}"
344
+ continue
345
+
346
+ merged_messages.append(current)
347
+
348
+ result = []
349
+ for msg in merged_messages:
350
+ role = msg.get("role", "")
351
+ content = msg.get("content", "")
352
+
353
+ if role in ["system", "user"]:
354
+ result.append({
355
+ "role": "user",
356
+ "content": [{"type": "text", "text": content}]
357
+ })
358
+ elif role == "assistant":
359
+ result.append({
360
+ "role": "assistant",
361
+ "content": [{"type": "text", "text": content}]
362
+ })
363
+ else:
364
+ result.append(msg)
365
+
366
+ return result
367
+
368
+
369
+ # Response handler class
370
+ class ResponseHandler:
371
+ @staticmethod
372
+ async def handle_stream_response(chat_message: str, model: str, request_id: str):
373
+ """Handle streaming response."""
374
+ logger.info(f"[{request_id}] Handling streaming response, content length: {len(chat_message)} characters")
375
+
376
+ async def generate():
377
+ index = 0
378
+ while index < len(chat_message):
379
+ # Simulate chunking similar to the Deno implementation
380
+ chunk_size = min(15 + int(15 * (0.5 - (0.5 * (index / len(chat_message))))), 30)
381
+ chunk = chat_message[index:index + chunk_size]
382
+
383
+ event_data = {
384
+ "id": generate_uuid(),
385
+ "object": "chat.completion.chunk",
386
+ "created": int(time.time()),
387
+ "model": model,
388
+ "choices": [{
389
+ "index": 0,
390
+ "delta": {"content": chunk},
391
+ "finish_reason": "stop" if index + chunk_size >= len(chat_message) else None
392
+ }]
393
+ }
394
+
395
+ yield f"data: {json.dumps(event_data)}\n\n"
396
+
397
+ index += chunk_size
398
+ await asyncio.sleep(0.05) # 50ms delay between chunks
399
+
400
+ yield "data: [DONE]\n\n"
401
+ logger.info(f"[{request_id}] Streaming response completed")
402
+
403
+ return StreamingResponse(
404
+ generate(),
405
+ media_type="text/event-stream",
406
+ headers={
407
+ "Cache-Control": "no-cache",
408
+ "Connection": "keep-alive",
409
+ }
410
+ )
411
+
412
+ @staticmethod
413
+ async def handle_normal_response(chat_message: str, model: str, request_id: str):
414
+ """Handle normal (non-streaming) response."""
415
+ logger.info(f"[{request_id}] Handling normal response, content length: {len(chat_message)} characters")
416
+
417
+ response_data = {
418
+ "id": generate_uuid(),
419
+ "object": "chat.completion",
420
+ "created": int(time.time()),
421
+ "model": model,
422
+ "choices": [{
423
+ "index": 0,
424
+ "message": {
425
+ "role": "assistant",
426
+ "content": chat_message
427
+ },
428
+ "finish_reason": "stop"
429
+ }],
430
+ "usage": None
431
+ }
432
+
433
+ return JSONResponse(content=response_data)
434
+
435
+
436
+ # Pydantic models for request validation
437
+ class Message(BaseModel):
438
+ role: str
439
+ content: Union[str, List[Dict[str, Any]], Dict[str, Any]]
440
+
441
+
442
+ class ChatCompletionRequest(BaseModel):
443
+ model: str
444
+ messages: List[Message]
445
+ temperature: Optional[float] = None
446
+ max_tokens: Optional[int] = None
447
+ presence_penalty: Optional[float] = None
448
+ frequency_penalty: Optional[float] = None
449
+ top_p: Optional[float] = None
450
+ top_k: Optional[int] = None
451
+ stream: Optional[bool] = False
452
+
453
+
454
+ # Create FastAPI app
455
+ app = FastAPI(title="E2B API Proxy")
456
+
457
+ # Add CORS middleware
458
+ app.add_middleware(
459
+ CORSMiddleware,
460
+ allow_origins=["*"],
461
+ allow_credentials=True,
462
+ allow_methods=["*"],
463
+ allow_headers=["*"],
464
+ )
465
+
466
+ # Mount static files directory
467
+ app.mount("/static", StaticFiles(directory="static"), name="static")
468
+
469
+
470
+ # Dependency for API key validation
471
+ async def verify_api_key(request: Request):
472
+ auth_header = request.headers.get("authorization")
473
+ if not auth_header:
474
+ raise HTTPException(
475
+ status_code=status.HTTP_401_UNAUTHORIZED,
476
+ detail="Missing API key"
477
+ )
478
+
479
+ token = auth_header.replace("Bearer ", "")
480
+ if token != CONFIG["API"]["API_KEY"]:
481
+ logger.error(f"Authentication failed, provided token: {token[:8]}...")
482
+ raise HTTPException(
483
+ status_code=status.HTTP_401_UNAUTHORIZED,
484
+ detail="Invalid API key"
485
+ )
486
+
487
+ return token
488
+
489
+
490
+ # API endpoints
491
+ @app.get("/hf/v1/models")
492
+ async def get_models():
493
+ """Get available models."""
494
+ logger.info("Getting model list")
495
+
496
+ models = [
497
+ {
498
+ "id": model_id,
499
+ "object": "model",
500
+ "created": int(time.time()),
501
+ "owned_by": "e2b"
502
+ }
503
+ for model_id in CONFIG["MODEL_CONFIG"].keys()
504
+ ]
505
+
506
+ logger.info(f"Model list returned successfully, model count: {len(models)}")
507
+ return {"object": "list", "data": models}
508
+
509
+
510
+ @app.post("/hf/v1/chat/completions")
511
+ async def chat_completions(
512
+ request: ChatCompletionRequest,
513
+ api_key: str = Depends(verify_api_key)
514
+ ):
515
+ """Handle chat completions."""
516
+ request_id = generate_uuid()
517
+ logger.info(f"[{request_id}] Processing chat completion request")
518
+
519
+ try:
520
+ logger.info(f"[{request_id}] User request body:", {
521
+ "model": request.model,
522
+ "messages_count": len(request.messages),
523
+ "stream": request.stream,
524
+ "temperature": request.temperature,
525
+ "max_tokens": request.max_tokens
526
+ })
527
+
528
+ # Configure options based on model limits
529
+ config_options = await config_opt(
530
+ {
531
+ "temperature": request.temperature,
532
+ "max_tokens": request.max_tokens,
533
+ "presence_penalty": request.presence_penalty,
534
+ "frequency_penalty": request.frequency_penalty,
535
+ "top_p": request.top_p,
536
+ "top_k": request.top_k
537
+ },
538
+ CONFIG["MODEL_CONFIG"][request.model]
539
+ )
540
+
541
+ # Prepare request for E2B API
542
+ api_client = ApiClient(request.model, request_id)
543
+ request_payload = await api_client.prepare_chat_request(
544
+ request.dict(),
545
+ config_options
546
+ )
547
+
548
+ logger.info(f"[{request_id}] Sending request to E2B:", {
549
+ "model": request_payload["model"]["name"],
550
+ "messages_count": len(request_payload["messages"]),
551
+ "config": request_payload["config"]
552
+ })
553
+
554
+ # Send request to E2B API
555
+ fetch_start_time = time.time()
556
+ async with httpx.AsyncClient() as client:
557
+ fetch_response = await client.post(
558
+ f"{CONFIG['API']['BASE_URL']}/api/chat",
559
+ headers=CONFIG["DEFAULT_HEADERS"],
560
+ json=request_payload,
561
+ timeout=60.0
562
+ )
563
+ fetch_end_time = time.time()
564
+
565
+ print(fetch_response.text)
566
+
567
+ # Process response
568
+ # response_data = fetch_response.json()
569
+ # logger.info(
570
+ # f"[{request_id}] Received E2B response: {fetch_response.status_code}, "
571
+ # f"time: {(fetch_end_time - fetch_start_time) * 1000:.0f}ms",
572
+ # {
573
+ # "status": fetch_response.status_code,
574
+ # "has_code": bool(response_data.get("code")),
575
+ # "has_text": bool(response_data.get("text")),
576
+ # "response_preview": (response_data.get("code", "") or
577
+ # response_data.get("text", "") or
578
+ # "")[:100] + "..."
579
+ # }
580
+ # )
581
+
582
+ # # Extract message content
583
+ # chat_message = (
584
+ # response_data.get("code", "").strip() or
585
+ # response_data.get("text", "").strip() or
586
+ # (response_data.strip() if isinstance(response_data, str) else None)
587
+ # )
588
+
589
+ chat_message = fetch_response.text
590
+
591
+ if not chat_message:
592
+ logger.error(f"[{request_id}] E2B did not return a valid response")
593
+ raise ValueError("No response from upstream service")
594
+
595
+ # Return response based on streaming preference
596
+ if request.stream:
597
+ return await ResponseHandler.handle_stream_response(
598
+ chat_message,
599
+ request.model,
600
+ request_id
601
+ )
602
+ else:
603
+ return await ResponseHandler.handle_normal_response(
604
+ chat_message,
605
+ request.model,
606
+ request_id
607
+ )
608
+
609
+ except Exception as e:
610
+ logger.error(f"[{request_id}] Error processing request:", exc_info=e)
611
+
612
+ return JSONResponse(
613
+ status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
614
+ content={
615
+ "error": {
616
+ "message": f"{str(e)} Request failed, possibly due to context limit exceeded or other error. Please try again later.",
617
+ "type": "server_error",
618
+ "param": None,
619
+ "code": None
620
+ }
621
+ }
622
+ )
623
+
624
+
625
+ @app.get("/", response_class=HTMLResponse)
626
+ async def root():
627
+ """Root endpoint that serves the HTML UI."""
628
+ with open("static/index.html", "r") as f:
629
+ html_content = f.read()
630
+ return HTMLResponse(content=html_content)
631
+
632
+
633
+ @app.get("/health")
634
+ async def health_check():
635
+ """Health check endpoint for Hugging Face."""
636
+ return {"status": "ok", "message": "E2B API Proxy is running"}
637
+
638
+
639
+ if __name__ == "__main__":
640
+ import uvicorn
641
+ import os
642
+ port = int(os.environ.get("PORT", 7860))
643
+ uvicorn.run("app:app", host="0.0.0.0", port=port, reload=True)
app.py.dockerignore ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Ignore git and version control
2
+ .git
3
+ .gitignore
4
+
5
+ # Ignore Docker files
6
+ Dockerfile
7
+ docker-compose.yml
8
+
9
+ # Ignore Python cache
10
+ __pycache__/
11
+ *.py[cod]
12
+ *$py.class
13
+
14
+ # Ignore environment files
15
+ .env
16
+ .env.example
17
+
18
+ # Ignore development files
19
+ .vscode/
20
+ .idea/
21
+
22
+ # Ignore logs
23
+ *.log
docker-compose.yml ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ version: '3.8'
2
+
3
+ services:
4
+ e2b-api-proxy:
5
+ build: .
6
+ container_name: e2b-api-proxy
7
+ ports:
8
+ - "7860:7860"
9
+ environment:
10
+ - API_BASE_URL=https://fragments.e2b.dev
11
+ - API_KEY=sk-123456 # Replace with your actual API key
12
+ restart: unless-stopped
13
+ volumes:
14
+ - ./app.py:/app/app.py # For development - allows code changes without rebuilding
dockerignore ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Ignore git and version control
2
+ .git
3
+ .gitignore
4
+
5
+ # Ignore Docker files
6
+ docker-compose.yml
7
+
8
+ # Ignore Python cache
9
+ __pycache__/
10
+ *.py[cod]
11
+ *$py.class
12
+
13
+ # Ignore environment files
14
+ .env
15
+ .env.example
16
+
17
+ # Ignore development files
18
+ .vscode/
19
+ .idea/
20
+
21
+ # Ignore logs
22
+ *.log
env.example ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ # E2B API Configuration
2
+ API_BASE_URL=https://fragments.e2b.dev
3
+ API_KEY=your-api-key-here
4
+
5
+ # Server Configuration
6
+ PORT=8000
7
+ LOG_LEVEL=INFO
gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
gitignore ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Python
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+ *.so
6
+ .Python
7
+ env/
8
+ build/
9
+ develop-eggs/
10
+ dist/
11
+ downloads/
12
+ eggs/
13
+ .eggs/
14
+ lib/
15
+ lib64/
16
+ parts/
17
+ sdist/
18
+ var/
19
+ *.egg-info/
20
+ .installed.cfg
21
+ *.egg
22
+
23
+ # Virtual Environment
24
+ venv/
25
+ ENV/
26
+ .env
27
+
28
+ # IDE
29
+ .idea/
30
+ .vscode/
31
+ *.swp
32
+ *.swo
33
+
34
+ # Logs
35
+ logs/
36
+ *.log
37
+
38
+ # Docker
39
+ .docker/
40
+
41
+ # OS specific
42
+ .DS_Store
43
+ Thumbs.db
requirements.txt ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ fastapi==0.110.0
2
+ uvicorn==0.27.1
3
+ python-dotenv==1.0.1
4
+ httpx==0.27.0
5
+ pydantic==2.6.1
run.sh ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+
3
+ # Set default port for Hugging Face
4
+ export PORT=${PORT:-7860}
5
+
6
+ # Start the FastAPI application
7
+ exec uvicorn app:app --host 0.0.0.0 --port $PORT
static/index.html ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
6
+ <title>E2B API Proxy</title>
7
+ <style>
8
+ body {
9
+ font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
10
+ line-height: 1.6;
11
+ color: #333;
12
+ max-width: 800px;
13
+ margin: 0 auto;
14
+ padding: 20px;
15
+ }
16
+ h1 {
17
+ color: #2c3e50;
18
+ border-bottom: 2px solid #3498db;
19
+ padding-bottom: 10px;
20
+ }
21
+ h2 {
22
+ color: #2980b9;
23
+ margin-top: 30px;
24
+ }
25
+ pre {
26
+ background-color: #f8f9fa;
27
+ border: 1px solid #e9ecef;
28
+ border-radius: 4px;
29
+ padding: 15px;
30
+ overflow-x: auto;
31
+ }
32
+ code {
33
+ font-family: 'Courier New', Courier, monospace;
34
+ }
35
+ .endpoint {
36
+ background-color: #e9f7fe;
37
+ border-left: 4px solid #3498db;
38
+ padding: 10px 15px;
39
+ margin: 20px 0;
40
+ }
41
+ .method {
42
+ font-weight: bold;
43
+ color: #2980b9;
44
+ }
45
+ .url {
46
+ color: #27ae60;
47
+ }
48
+ .button {
49
+ display: inline-block;
50
+ background-color: #3498db;
51
+ color: white;
52
+ padding: 10px 15px;
53
+ text-decoration: none;
54
+ border-radius: 4px;
55
+ margin-top: 20px;
56
+ }
57
+ .button:hover {
58
+ background-color: #2980b9;
59
+ }
60
+ </style>
61
+ </head>
62
+ <body>
63
+ <h1>E2B API Proxy</h1>
64
+ <p>This is a FastAPI implementation of an API proxy for E2B (fragments.e2b.dev). It provides a compatible interface for various AI model providers including OpenAI, Google, and Anthropic.</p>
65
+
66
+ <h2>API Endpoints</h2>
67
+
68
+ <div class="endpoint">
69
+ <p><span class="method">GET</span> <span class="url">/hf/v1/models</span></p>
70
+ <p>List available models</p>
71
+ <pre><code>curl http://localhost:7860/hf/v1/models</code></pre>
72
+ </div>
73
+
74
+ <div class="endpoint">
75
+ <p><span class="method">POST</span> <span class="url">/hf/v1/chat/completions</span></p>
76
+ <p>Send chat completion requests</p>
77
+ <pre><code>curl -X POST http://localhost:7860/hf/v1/chat/completions \
78
+ -H "Content-Type: application/json" \
79
+ -H "Authorization: Bearer sk-123456" \
80
+ -d '{
81
+ "model": "gpt-4o",
82
+ "messages": [
83
+ {"role": "user", "content": "Hello, how are you?"}
84
+ ]
85
+ }'</code></pre>
86
+ </div>
87
+
88
+ <div class="endpoint">
89
+ <p><span class="method">GET</span> <span class="url">/health</span></p>
90
+ <p>Health check endpoint</p>
91
+ <pre><code>curl http://localhost:7860/health</code></pre>
92
+ </div>
93
+
94
+ <h2>Supported Models</h2>
95
+ <p>The API supports various models from different providers:</p>
96
+ <ul>
97
+ <li><strong>OpenAI</strong>: o1-preview, o3-mini, gpt-4o, gpt-4.5-preview, gpt-4-turbo</li>
98
+ <li><strong>Google</strong>: gemini-1.5-pro, gemini-2.5-pro-exp-03-25, gemini-exp-1121, gemini-2.0-flash-exp</li>
99
+ <li><strong>Anthropic</strong>: claude-3-5-sonnet-latest, claude-3-7-sonnet-latest, claude-3-5-haiku-latest</li>
100
+ </ul>
101
+
102
+ <a href="https://github.com/yourusername/e2b-api-proxy" class="button">View on GitHub</a>
103
+ </body>
104
+ </html>