Spaces:
Running
Running
DVampire
commited on
Commit
·
f7470ea
1
Parent(s):
78f6650
update readme
Browse files
README.md
CHANGED
@@ -11,15 +11,29 @@ pinned: false
|
|
11 |
|
12 |
# Paper Index - AI Paper Evaluation System
|
13 |
|
14 |
-
A comprehensive system for evaluating AI research papers using advanced language models.
|
15 |
|
16 |
## Features
|
17 |
|
18 |
- **Daily Paper Crawling**: Automatically fetches papers from Hugging Face daily
|
19 |
- **AI Evaluation**: Uses Claude Sonnet to evaluate papers across multiple dimensions
|
|
|
|
|
|
|
20 |
- **Interactive Dashboard**: Beautiful web interface for browsing and evaluating papers
|
21 |
-
- **Database
|
22 |
- **Smart Navigation**: Intelligent date navigation with fallback mechanisms
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
23 |
|
24 |
## Hugging Face Spaces Deployment
|
25 |
|
@@ -93,7 +107,9 @@ PORT=7860 # Optional, defaults to 7860
|
|
93 |
### Evaluation Endpoints
|
94 |
|
95 |
- `POST /api/papers/evaluate/{arxiv_id}` - Start paper evaluation
|
|
|
96 |
- `GET /api/papers/evaluate/{arxiv_id}/status` - Get evaluation status
|
|
|
97 |
|
98 |
### Cache Management
|
99 |
|
@@ -105,19 +121,23 @@ PORT=7860 # Optional, defaults to 7860
|
|
105 |
|
106 |
### Frontend
|
107 |
- **HTML/CSS/JavaScript**: Modern, responsive interface
|
108 |
-
- **Real-time Updates**: Dynamic content loading
|
109 |
- **Theme Support**: Light/dark mode toggle
|
|
|
|
|
110 |
|
111 |
### Backend
|
112 |
- **FastAPI**: High-performance web framework
|
113 |
-
- **SQLite**:
|
114 |
-
- **Async Processing**: Background evaluation tasks
|
|
|
115 |
- **Caching**: Intelligent caching system for performance
|
116 |
|
117 |
### AI Integration
|
118 |
-
- **
|
119 |
- **Multi-dimensional Analysis**: Comprehensive evaluation criteria
|
120 |
- **Structured Output**: JSON-based evaluation results
|
|
|
121 |
|
122 |
## Database Schema
|
123 |
|
@@ -142,6 +162,11 @@ CREATE TABLE papers (
|
|
142 |
);
|
143 |
```
|
144 |
|
|
|
|
|
|
|
|
|
|
|
145 |
## Evaluation Dimensions
|
146 |
|
147 |
The system evaluates papers across 12 key dimensions:
|
@@ -159,6 +184,52 @@ The system evaluates papers across 12 key dimensions:
|
|
159 |
11. **3-Year Feasibility** - Short-term potential
|
160 |
12. **Overall Automatability** - Comprehensive assessment
|
161 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
162 |
## Contributing
|
163 |
|
164 |
1. Fork the repository
|
|
|
11 |
|
12 |
# Paper Index - AI Paper Evaluation System
|
13 |
|
14 |
+
A comprehensive system for evaluating AI research papers using advanced language models with asynchronous processing and concurrent evaluation capabilities.
|
15 |
|
16 |
## Features
|
17 |
|
18 |
- **Daily Paper Crawling**: Automatically fetches papers from Hugging Face daily
|
19 |
- **AI Evaluation**: Uses Claude Sonnet to evaluate papers across multiple dimensions
|
20 |
+
- **Concurrent Processing**: True asynchronous evaluation with multiple papers processed simultaneously
|
21 |
+
- **Re-evaluation**: Ability to re-run evaluations for papers with updated results
|
22 |
+
- **Batch Evaluation**: "Evaluate All" feature to process multiple papers at once
|
23 |
- **Interactive Dashboard**: Beautiful web interface for browsing and evaluating papers
|
24 |
+
- **Asynchronous Database**: High-performance SQLite with WAL mode for concurrent operations
|
25 |
- **Smart Navigation**: Intelligent date navigation with fallback mechanisms
|
26 |
+
- **Real-time Status Updates**: Live progress tracking and notifications
|
27 |
+
|
28 |
+
## Recent Updates
|
29 |
+
|
30 |
+
### v0.1.0 - Asynchronous & Concurrent Features
|
31 |
+
- **Asynchronous Database**: Migrated from `sqlite3` to `aiosqlite` for better performance
|
32 |
+
- **Concurrent Evaluation**: Multiple papers can be evaluated simultaneously
|
33 |
+
- **Re-evaluation**: Added "Re-evaluate" button for papers to update evaluation results
|
34 |
+
- **Batch Processing**: "Evaluate All" button to process all un-evaluated papers
|
35 |
+
- **Enhanced UI**: Improved progress indicators and real-time notifications
|
36 |
+
- **Database Optimization**: WAL mode and performance pragmas for better concurrency
|
37 |
|
38 |
## Hugging Face Spaces Deployment
|
39 |
|
|
|
107 |
### Evaluation Endpoints
|
108 |
|
109 |
- `POST /api/papers/evaluate/{arxiv_id}` - Start paper evaluation
|
110 |
+
- `POST /api/papers/reevaluate/{arxiv_id}` - Re-evaluate a paper
|
111 |
- `GET /api/papers/evaluate/{arxiv_id}/status` - Get evaluation status
|
112 |
+
- `GET /api/papers/evaluate/active-tasks` - Get currently running evaluations
|
113 |
|
114 |
### Cache Management
|
115 |
|
|
|
121 |
|
122 |
### Frontend
|
123 |
- **HTML/CSS/JavaScript**: Modern, responsive interface
|
124 |
+
- **Real-time Updates**: Dynamic content loading with polling
|
125 |
- **Theme Support**: Light/dark mode toggle
|
126 |
+
- **Progress Indicators**: Visual feedback for evaluation status
|
127 |
+
- **Batch Operations**: "Evaluate All" functionality with sequential processing
|
128 |
|
129 |
### Backend
|
130 |
- **FastAPI**: High-performance web framework
|
131 |
+
- **Async SQLite**: `aiosqlite` with WAL mode for concurrent operations
|
132 |
+
- **Async Processing**: Background evaluation tasks with task tracking
|
133 |
+
- **Concurrent Evaluation**: Multiple papers evaluated simultaneously
|
134 |
- **Caching**: Intelligent caching system for performance
|
135 |
|
136 |
### AI Integration
|
137 |
+
- **Async Anthropic**: Non-blocking API calls with `AsyncAnthropic`
|
138 |
- **Multi-dimensional Analysis**: Comprehensive evaluation criteria
|
139 |
- **Structured Output**: JSON-based evaluation results
|
140 |
+
- **Error Handling**: Robust error handling and retry mechanisms
|
141 |
|
142 |
## Database Schema
|
143 |
|
|
|
162 |
);
|
163 |
```
|
164 |
|
165 |
+
### Database Optimizations
|
166 |
+
- **WAL Mode**: `PRAGMA journal_mode=WAL` for better concurrency
|
167 |
+
- **Performance Pragmas**: Optimized settings for concurrent access
|
168 |
+
- **Asynchronous Operations**: All database calls are async/await
|
169 |
+
|
170 |
## Evaluation Dimensions
|
171 |
|
172 |
The system evaluates papers across 12 key dimensions:
|
|
|
184 |
11. **3-Year Feasibility** - Short-term potential
|
185 |
12. **Overall Automatability** - Comprehensive assessment
|
186 |
|
187 |
+
## Key Features
|
188 |
+
|
189 |
+
### Concurrent Evaluation
|
190 |
+
- Multiple papers can be evaluated simultaneously
|
191 |
+
- Global task tracking prevents duplicate evaluations
|
192 |
+
- Real-time status updates via polling
|
193 |
+
- Automatic error handling and recovery
|
194 |
+
|
195 |
+
### Re-evaluation System
|
196 |
+
- "Re-evaluate" button appears after initial evaluation
|
197 |
+
- Updates existing evaluation results in database
|
198 |
+
- Maintains evaluation history and timestamps
|
199 |
+
- Same comprehensive evaluation criteria
|
200 |
+
|
201 |
+
### Batch Processing
|
202 |
+
- "Evaluate All" button processes all un-evaluated papers
|
203 |
+
- Sequential processing with delays to prevent API overload
|
204 |
+
- Progress tracking and real-time notifications
|
205 |
+
- Automatic button state management
|
206 |
+
|
207 |
+
### Enhanced UI/UX
|
208 |
+
- Progress circles with proper layering
|
209 |
+
- Bottom-right notification system
|
210 |
+
- Dynamic button states and text updates
|
211 |
+
- Responsive design with modern styling
|
212 |
+
|
213 |
+
## Performance Optimizations
|
214 |
+
|
215 |
+
### Database
|
216 |
+
- Asynchronous operations with `aiosqlite`
|
217 |
+
- WAL mode for better concurrency
|
218 |
+
- Optimized SQLite pragmas
|
219 |
+
- Connection pooling and management
|
220 |
+
|
221 |
+
### API Calls
|
222 |
+
- Non-blocking Anthropic API calls
|
223 |
+
- Concurrent evaluation processing
|
224 |
+
- Task tracking and management
|
225 |
+
- Error handling and retry logic
|
226 |
+
|
227 |
+
### Frontend
|
228 |
+
- Efficient DOM manipulation
|
229 |
+
- Polling with appropriate intervals
|
230 |
+
- Memory management for log entries
|
231 |
+
- Optimized event handling
|
232 |
+
|
233 |
## Contributing
|
234 |
|
235 |
1. Fork the repository
|