Hugging Face's logo

Spaces:
CatPtain
/
GeminiBalance
Running

App Files Community
CatPtain commited on
Commit
7865ae2
·
verified ·
1 Parent(s): 33204ed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +292 -0
README.md CHANGED
@@ -9,3 +9,295 @@ license: mit
9
  ---
10
 
11
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  ---
10
 
11
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
12
+ [Read this document in Chinese](README_ZH.md)
13
+
14
+ # Gemini Balance - Gemini API Proxy and Load Balancer
15
+
16
+ <p align="center">
17
+ <a href="https://trendshift.io/repositories/13692" target="_blank">
18
+ <img src="https://trendshift.io/api/badge/repositories/13692" alt="snailyp%2Fgemini-balance | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/>
19
+ </a>
20
+ </p>
21
+
22
+ > ⚠️ This project is licensed under the CC BY-NC 4.0 (Attribution-NonCommercial) license. Any form of commercial resale service is prohibited. See the LICENSE file for details.
23
+
24
+ > I have never sold this service on any platform. If you encounter someone selling this service, they are definitely a reseller. Please be careful not to be deceived.
25
+
26
+ [![Python](https://img.shields.io/badge/Python-3.9%2B-blue.svg)](https://www.python.org/)
27
+ [![FastAPI](https://img.shields.io/badge/FastAPI-0.100%2B-green.svg)](https://fastapi.tiangolo.com/)
28
+ [![Uvicorn](https://img.shields.io/badge/Uvicorn-running-purple.svg)](https://www.uvicorn.org/)
29
+ [![Telegram Group](https://img.shields.io/badge/Telegram-Group-blue.svg?logo=telegram)](https://t.me/+soaHax5lyI0wZDVl)
30
+
31
+ > Telegram Group: <https://t.me/+soaHax5lyI0wZDVl>
32
+
33
+ ## Project Introduction
34
+
35
+ Gemini Balance is an application built with Python FastAPI, designed to provide proxy and load balancing functions for the Google Gemini API. It allows you to manage multiple Gemini API Keys and implement key rotation, authentication, model filtering, and status monitoring through simple configuration. Additionally, the project integrates image generation and multiple image hosting upload functions, and supports proxying in the OpenAI API format.
36
+
37
+ **Project Structure:**
38
+
39
+ ```plaintext
40
+ app/
41
+ ├── config/ # Configuration management
42
+ ├── core/ # Core application logic (FastAPI instance creation, middleware, etc.)
43
+ ├── database/ # Database models and connections
44
+ ├── domain/ # Business domain objects (optional)
45
+ ├── exception/ # Custom exceptions
46
+ ├── handler/ # Request handlers (optional, or handled in router)
47
+ ├── log/ # Logging configuration
48
+ ├── main.py # Application entry point
49
+ ├── middleware/ # FastAPI middleware
50
+ ├── router/ # API routes (Gemini, OpenAI, status page, etc.)
51
+ ├── scheduler/ # Scheduled tasks (e.g., Key status check)
52
+ ├── service/ # Business logic services (chat, Key management, statistics, etc.)
53
+ ├── static/ # Static files (CSS, JS)
54
+ ├── templates/ # HTML templates (e.g., Key status page)
55
+ ├── utils/ # Utility functions
56
+ ```
57
+
58
+ ## ✨ Feature Highlights
59
+
60
+ * **Multi-Key Load Balancing**: Supports configuring multiple Gemini API Keys (`API_KEYS`) for automatic sequential polling, improving availability and concurrency.
61
+ * **Visual Configuration Takes Effect Immediately**: Configurations modified through the admin backend take effect without restarting the service. Remember to click save for changes to apply.
62
+ ![Configuration Panel](files/image4.png)
63
+ * **Dual Protocol API Compatibility**: Supports forwarding CHAT API requests in both Gemini and OpenAI formats.
64
+
65
+ ```plaintext
66
+ openai baseurl `http://localhost:8000(/hf)/v1`
67
+ gemini baseurl `http://localhost:8000(/gemini)/v1beta`
68
+ ```
69
+
70
+ * **Supports Image-Text Chat and Image Modification**: `IMAGE_MODELS` configures which models can perform image-text chat and image editing. When actually calling, use the `configured_model-image` model name to use this feature.
71
+ ![Chat with Image Generation](files/image6.png)
72
+ ![Modify Image](files/image7.png)
73
+ * **Supports Web Search**: Supports web search. `SEARCH_MODELS` configures which models can perform web searches. When actually calling, use the `configured_model-search` model name to use this feature.
74
+ ![Web Search](files/image8.png)
75
+ * **Key Status Monitoring**: Provides a `/keys_status` page (requires authentication) to view the status and usage of each Key in real-time.
76
+ ![Monitoring Panel](files/image.png)
77
+ * **Detailed Logging**: Provides detailed error logs for easy troubleshooting.
78
+ ![Call Details](files/image1.png)
79
+ ![Log List](files/image2.png)
80
+ ![Log Details](files/image3.png)
81
+ * **Support for Custom Gemini Proxy**: Supports custom Gemini proxies, such as those built on Deno or Cloudflare.
82
+ * **OpenAI Image Generation API Compatibility**: Adapts the `imagen-3.0-generate-002` model interface to be compatible with the OpenAI image generation API, supporting client calls.
83
+ * **Flexible Key Addition**: Flexible way to add keys using regex matching for `gemini_key`, with key deduplication.
84
+ ![Add Key](files/image5.png)
85
+ * **OpenAI Format Embeddings API Compatibility**: Perfectly adapts to the OpenAI format `embeddings` interface, usable for local document vectorization.
86
+ * **Streamlined Response Optimization**: Optional stream output optimizer (`STREAM_OPTIMIZER_ENABLED`) to improve the experience of long-text stream responses.
87
+ * **Failure Retry and Key Management**: Automatically handles API request failures, retries (`MAX_RETRIES`), automatically disables Keys after too many failures (`MAX_FAILURES`), and periodically checks for recovery (`CHECK_INTERVAL_HOURS`).
88
+ * **Docker Support**: Supports AMD and ARM architecture Docker deployments. You can also build your own Docker image.
89
+ > Image address: docker pull ghcr.io/snailyp/gemini-balance:latest
90
+ * **Automatic Model List Maintenance**: Supports fetching OpenAI and Gemini model lists, perfectly compatible with NewAPI's automatic model list fetching, no manual entry required.
91
+ * **Support for Removing Unused Models**: Too many default models are provided, many of which are not used. You can filter them out using `FILTERED_MODELS`.
92
+ * **Proxy Support**: Supports configuring HTTP/SOCKS5 proxy servers (`PROXIES`) for accessing the Gemini API, convenient for use in special network environments. Supports batch adding proxies.
93
+
94
+ ## 🚀 Quick Start
95
+
96
+ ### Build Docker Yourself (Recommended)
97
+
98
+ #### a) Build with Dockerfile
99
+
100
+ 1. **Build Image**:
101
+
102
+ ```bash
103
+ docker build -t gemini-balance .
104
+ ```
105
+
106
+ 2. **Run Container**:
107
+
108
+ ```bash
109
+ docker run -d -p 8000:8000 --env-file .env gemini-balance
110
+ ```
111
+
112
+ * `-d`: Run in detached mode.
113
+ * `-p 8000:8000`: Map port 8000 of the container to port 8000 of the host.
114
+ * `--env-file .env`: Use the `.env` file to set environment variables.
115
+
116
+ > Note: If using an SQLite database, you need to mount a data volume to persist
117
+ >
118
+ > ```bash
119
+ > docker run -d -p 8000:8000 --env-file .env -v /path/to/data:/app/data gemini-balance
120
+ > ```
121
+ >
122
+ > Where `/path/to/data` is the data storage path on the host, and `/app/data` is the data directory inside the container.
123
+
124
+ #### b) Deploy with an Existing Docker Image
125
+
126
+ 1. **Pull Image**:
127
+
128
+ ```bash
129
+ docker pull ghcr.io/snailyp/gemini-balance:latest
130
+ ```
131
+
132
+ 2. **Run Container**:
133
+
134
+ ```bash
135
+ docker run -d -p 8000:8000 --env-file .env ghcr.io/snailyp/gemini-balance:latest
136
+ ```
137
+
138
+ * `-d`: Run in detached mode.
139
+ * `-p 8000:8000`: Map port 8000 of the container to port 8000 of the host (adjust as needed).
140
+ * `--env-file .env`: Use the `.env` file to set environment variables (ensure the `.env` file exists in the directory where the command is executed).
141
+
142
+ > Note: If using an SQLite database, you need to mount a data volume to persist
143
+ >
144
+ > ```bash
145
+ > docker run -d -p 8000:8000 --env-file .env -v /path/to/data:/app/data ghcr.io/snailyp/gemini-balance:latest
146
+ > ```
147
+ >
148
+ > Where `/path/to/data` is the data storage path on the host, and `/app/data` is the data directory inside the container.
149
+
150
+ ### Run Locally (Suitable for Development and Testing)
151
+
152
+ If you want to run the source code directly locally for development or testing, follow these steps:
153
+
154
+ 1. **Ensure Prerequisites are Met**:
155
+ * Clone the repository locally.
156
+ * Install Python 3.9 or higher.
157
+ * Create and configure the `.env` file in the project root directory (refer to the "Configure Environment Variables" section above).
158
+ * Install project dependencies:
159
+
160
+ ```bash
161
+ pip install -r requirements.txt
162
+ ```
163
+
164
+ 2. **Start Application**:
165
+ Run the following command in the project root directory:
166
+
167
+ ```bash
168
+ uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
169
+ ```
170
+
171
+ * `app.main:app`: Specifies the location of the FastAPI application instance (the `app` object in the `main.py` file within the `app` module).
172
+ * `--host 0.0.0.0`: Makes the application accessible from any IP address on the local network.
173
+ * `--port 8000`: Specifies the port number the application listens on (you can change this as needed).
174
+ * `--reload`: Enables automatic reloading. When you modify the code, the service will automatically restart, which is very suitable for development environments (remove this option in production environments).
175
+
176
+ 3. **Access Application**:
177
+ After the application starts, you can access `http://localhost:8000` (or the host and port you specified) through a browser or API tool.
178
+
179
+ ### Complete Configuration List
180
+
181
+ | Configuration Item | Description | Default Value |
182
+ | :----------------------------- | :-------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
183
+ | **Database Configuration** | | |
184
+ | `DATABASE_TYPE` | Optional, database type, supports `mysql` or `sqlite` | `mysql` |
185
+ | `SQLITE_DATABASE` | Optional, required when using `sqlite`, SQLite database file path | `default_db` |
186
+ | `MYSQL_HOST` | Required when using `mysql`, MySQL database host address | `localhost` |
187
+ | `MYSQL_SOCKET` | Optional, MySQL database socket address | `/var/run/mysqld/mysqld.sock` |
188
+ | `MYSQL_PORT` | Required when using `mysql`, MySQL database port | `3306` |
189
+ | `MYSQL_USER` | Required when using `mysql`, MySQL database username | `your_db_user` |
190
+ | `MYSQL_PASSWORD` | Required when using `mysql`, MySQL database password | `your_db_password` |
191
+ | `MYSQL_DATABASE` | Required when using `mysql`, MySQL database name | `defaultdb` |
192
+ | **API Related Configuration** | | |
193
+ | `API_KEYS` | Required, list of Gemini API keys for load balancing | `["your-gemini-api-key-1", "your-gemini-api-key-2"]` |
194
+ | `ALLOWED_TOKENS` | Required, list of tokens allowed to access | `["your-access-token-1", "your-access-token-2"]` |
195
+ | `AUTH_TOKEN` | Optional, super admin token with all permissions, defaults to the first of `ALLOWED_TOKENS` if not set | `sk-123456` |
196
+ | `TEST_MODEL` | Optional, model name used to test if a key is usable | `gemini-1.5-flash` |
197
+ | `IMAGE_MODELS` | Optional, list of models that support drawing functions | `["gemini-2.0-flash-exp"]` |
198
+ | `SEARCH_MODELS` | Optional, list of models that support search functions | `["gemini-2.0-flash-exp"]` |
199
+ | `FILTERED_MODELS` | Optional, list of disabled models | `["gemini-1.0-pro-vision-latest", ...]` |
200
+ | `TOOLS_CODE_EXECUTION_ENABLED` | Optional, whether to enable the code execution tool | `false` |
201
+ | `SHOW_SEARCH_LINK` | Optional, whether to display search result links in the response | `true` |
202
+ | `SHOW_THINKING_PROCESS` | Optional, whether to display the model's thinking process | `true` |
203
+ | `THINKING_MODELS` | Optional, list of models that support thinking functions | `[]` |
204
+ | `THINKING_BUDGET_MAP` | Optional, thinking function budget mapping (model_name:budget_value) | `{}` |
205
+ | `URL_NORMALIZATION_ENABLED` | Optional, whether to enable intelligent URL routing mapping | `false` |
206
+ | `BASE_URL` | Optional, Gemini API base URL, no modification needed by default | `https://generativelanguage.googleapis.com/v1beta` |
207
+ | `MAX_FAILURES` | Optional, number of times a single key is allowed to fail | `3` |
208
+ | `MAX_RETRIES` | Optional, maximum number of retries for failed API requests | `3` |
209
+ | `CHECK_INTERVAL_HOURS` | Optional, time interval (hours) to check if a disabled Key has recovered | `1` |
210
+ | `TIMEZONE` | Optional, timezone used by the application | `Asia/Shanghai` |
211
+ | `TIME_OUT` | Optional, request timeout (seconds) | `300` |
212
+ | `PROXIES` | Optional, list of proxy servers (e.g., `http://user:pass@host:port`, `socks5://host:port`) | `[]` |
213
+ | `LOG_LEVEL` | Optional, log level, e.g., DEBUG, INFO, WARNING, ERROR, CRITICAL | `INFO` |
214
+ | `AUTO_DELETE_ERROR_LOGS_ENABLED` | Optional, whether to enable automatic deletion of error logs | `true` |
215
+ | `AUTO_DELETE_ERROR_LOGS_DAYS` | Optional, automatically delete error logs older than this many days (e.g., 1, 7, 30) | `7` |
216
+ | `AUTO_DELETE_REQUEST_LOGS_ENABLED`| Optional, whether to enable automatic deletion of request logs | `false` |
217
+ | `AUTO_DELETE_REQUEST_LOGS_DAYS` | Optional, automatically delete request logs older than this many days (e.g., 1, 7, 30) | `30` |
218
+ | `SAFETY_SETTINGS` | Optional, safety settings (JSON string format), used to configure content safety thresholds. Example values may need adjustment based on actual model support. | `[{"category": "HARM_CATEGORY_HARASSMENT", "threshold": "OFF"}, {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "OFF"}, {"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "OFF"}, {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "OFF"}, {"category": "HARM_CATEGORY_CIVIC_INTEGRITY", "threshold": "BLOCK_NONE"}]` |
219
+ | **TTS Related** | | |
220
+ | `TTS_MODEL` | Optional, TTS model name | `gemini-2.5-flash-preview-tts` |
221
+ | `TTS_VOICE_NAME` | Optional, TTS voice name | `Zephyr` |
222
+ | `TTS_SPEED` | Optional, TTS speed | `normal` |
223
+ | **Image Generation Related** | | |
224
+ | `PAID_KEY` | Optional, paid API Key for advanced features like image generation | `your-paid-api-key` |
225
+ | `CREATE_IMAGE_MODEL` | Optional, image generation model | `imagen-3.0-generate-002` |
226
+ | `UPLOAD_PROVIDER` | Optional, image upload provider: `smms`, `picgo`, `cloudflare_imgbed` | `smms` |
227
+ | `SMMS_SECRET_TOKEN` | Optional, API Token for SM.MS image hosting | `your-smms-token` |
228
+ | `PICGO_API_KEY` | Optional, API Key for [PicoGo](https://www.picgo.net/) image hosting | `your-picogo-apikey` |
229
+ | `CLOUDFLARE_IMGBED_URL` | Optional, [CloudFlare](https://github.com/MarSeventh/CloudFlare-ImgBed) image hosting upload address | `https://xxxxxxx.pages.dev/upload` |
230
+ | `CLOUDFLARE_IMGBED_AUTH_CODE` | Optional, authentication key for CloudFlare image hosting | `your-cloudflare-imgber-auth-code` |
231
+ | **Stream Optimizer Related** | | |
232
+ | `STREAM_OPTIMIZER_ENABLED` | Optional, whether to enable stream output optimization | `false` |
233
+ | `STREAM_MIN_DELAY` | Optional, minimum delay for stream output | `0.016` |
234
+ | `STREAM_MAX_DELAY` | Optional, maximum delay for stream output | `0.024` |
235
+ | `STREAM_SHORT_TEXT_THRESHOLD` | Optional, short text threshold | `10` |
236
+ | `STREAM_LONG_TEXT_THRESHOLD` | Optional, long text threshold | `50` |
237
+ | `STREAM_CHUNK_SIZE` | Optional, stream output chunk size | `5` |
238
+ | **Fake Stream Related** | | |
239
+ | `FAKE_STREAM_ENABLED` | Optional, whether to enable fake streaming for models or scenarios that don't support streaming | `false` |
240
+ | `FAKE_STREAM_EMPTY_DATA_INTERVAL_SECONDS` | Optional, interval in seconds for sending heartbeat empty data during fake streaming | `5` |
241
+
242
+ ## ⚙️ API Endpoints
243
+
244
+ The following are the main API endpoints provided by the service:
245
+
246
+ ### Gemini API Related (`(/gemini)/v1beta`)
247
+
248
+ * `GET /models`: List available Gemini models.
249
+ * `POST /models/{model_name}:generateContent`: Generate content using the specified Gemini model.
250
+ * `POST /models/{model_name}:streamGenerateContent`: Stream content generation using the specified Gemini model.
251
+
252
+ ### OpenAI API Related
253
+
254
+ * `GET (/hf)/v1/models`: List available models (uses Gemini format underneath).
255
+ * `POST (/hf)/v1/chat/completions`: Perform chat completion (uses Gemini format underneath, supports streaming).
256
+ * `POST (/hf)/v1/embeddings`: Create text embeddings (uses Gemini format underneath).
257
+ * `POST (/hf)/v1/images/generations`: Generate images (uses Gemini format underneath).
258
+ * `GET /openai/v1/models`: List available models (uses OpenAI format underneath).
259
+ * `POST /openai/v1/chat/completions`: Perform chat completion (uses OpenAI format underneath, supports streaming, can prevent truncation, and is faster).
260
+ * `POST /openai/v1/embeddings`: Create text embeddings (uses OpenAI format underneath).
261
+ * `POST /openai/v1/images/generations`: Generate images (uses OpenAI format underneath).
262
+
263
+ ## 🤝 Contributing
264
+
265
+ Pull Requests or Issues are welcome.
266
+
267
+ ## 🎉 Special Thanks
268
+
269
+ Special thanks to the following projects and platforms for providing image hosting services for this project:
270
+
271
+ * [PicGo](https://www.picgo.net/)
272
+ * [SM.MS](https://smms.app/)
273
+ * [CloudFlare-ImgBed](https://github.com/MarSeventh/CloudFlare-ImgBed) open source project
274
+
275
+ ## 🙏 Thanks to Contributors
276
+
277
+ Thanks to all developers who contributed to this project!
278
+
279
+ [![Contributors](https://contrib.rocks/image?repo=snailyp/gemini-balance)](https://github.com/snailyp/gemini-balance/graphs/contributors)
280
+
281
+ ## Thanks to Our Supporters
282
+
283
+ A special shout-out to DigitalOcean for providing the rock-solid and dependable cloud infrastructure that keeps this project humming!
284
+ [![DigitalOcean Logo](files/dataocean.svg)](https://m.do.co/c/b249dd7f3b4c)
285
+
286
+ CDN acceleration and security protection for this project are sponsored by Tencent EdgeOne.
287
+ [![EdgeOne Logo](https://edgeone.ai/media/34fe3a45-492d-4ea4-ae5d-ea1087ca7b4b.png)](https://edgeone.ai/?from=github)
288
+
289
+ ## ⭐ Star History
290
+
291
+ [![Star History Chart](https://api.star-history.com/svg?repos=snailyp/gemini-balance&type=Date)](https://star-history.com/#snailyp/gemini-balance&Date)
292
+
293
+ ## 💖 Friendly Projects
294
+
295
+ * **[OneLine](https://github.com/chengtx809/OneLine)** by [chengtx809](https://github.com/chengtx809) - OneLine: AI-driven hot event timeline generation tool
296
+
297
+ ## 🎁 Project Support
298
+
299
+ If you find this project helpful, consider supporting me via [Afdian](https://afdian.com/a/snaily).
300
+
301
+ ## License
302
+
303
+ This project is licensed under the CC BY-NC 4.0 (Attribution-NonCommercial) license. Any form of commercial resale service is prohibited. See the LICENSE file for details.