Add comprehensive service breakdown: Nebius AI vs Modal.com usage
Browse filesβ
Clear Service Division:
- Nebius AI: 'The Brain' - Language intelligence, AI reasoning, content analysis
- Modal.com: 'The Engine' - Heavy computation, OCR, vector operations, batch processing
β
Specific Use Cases:
- Document analysis, query enhancement, research synthesis β Nebius AI
- OCR processing, FAISS indexing, distributed computing β Modal.com
β
Integration Workflows:
- Step-by-step document processing pipeline showing which service handles each step
- Search workflow demonstrating coordinated service usage
β
Feature Comparison Table:
- Clear checkmarks showing exactly which service handles which operations
- Notes on overlapping areas (embeddings: real-time vs batch)
β
Specific API Endpoints:
- Listed exact endpoints that use each service
- Removed confusion about service responsibilities
This section provides crystal clear understanding of why both services are needed
and how they complement each other in the architecture.
@@ -343,18 +343,79 @@ npm run build
|
|
343 |
- High-performance semantic similarity search
|
344 |
- Persistent storage across sessions
|
345 |
|
346 |
-
### **Service Integration**
|
347 |
-
|
348 |
-
|
349 |
-
|
350 |
-
|
351 |
-
-
|
352 |
-
|
353 |
-
|
354 |
-
-
|
355 |
-
-
|
356 |
-
|
357 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
358 |
|
359 |
### **Intelligent Fallbacks**
|
360 |
- **Modal Unavailable**: Local processing for text files, basic search
|
|
|
343 |
- High-performance semantic similarity search
|
344 |
- Persistent storage across sessions
|
345 |
|
346 |
+
### **Service Integration & Division of Responsibilities**
|
347 |
+
|
348 |
+
## **π§ Nebius AI: Language Intelligence & AI Reasoning**
|
349 |
+
|
350 |
+
### **Used For:**
|
351 |
+
- **π Document Analysis**: Classification, summarization, key points extraction, quality scoring
|
352 |
+
- **π Search Intelligence**: Query enhancement, intent understanding, relevance scoring
|
353 |
+
- **π AI Reasoning**: Research synthesis, explanations, conversational responses
|
354 |
+
- **π― Embeddings**: Real-time text-to-vector conversion using BAAI/bge-en-icl model
|
355 |
+
- **π Content Understanding**: All language comprehension and semantic analysis
|
356 |
+
|
357 |
+
### **Specific Endpoints:**
|
358 |
+
- `/api/analyze-document` - Document analysis with DeepSeek-R1 model
|
359 |
+
- `/api/enhance-query` - AI-powered query improvement
|
360 |
+
- `/api/embeddings` - Generate vector embeddings
|
361 |
+
- `/api/research-synthesis` - Combine insights from multiple sources
|
362 |
+
- `/api/ai-search` - Enhanced semantic search
|
363 |
+
|
364 |
+
---
|
365 |
+
|
366 |
+
## **β‘ Modal.com: Heavy Computation & Distributed Processing**
|
367 |
+
|
368 |
+
### **Used For:**
|
369 |
+
- **π OCR Processing**: PDF and image text extraction using Tesseract
|
370 |
+
- **π§ Vector Operations**: FAISS index building and high-performance search
|
371 |
+
- **π¦ Batch Processing**: Concurrent processing of large document collections
|
372 |
+
- **πΎ Infrastructure**: Serverless scaling, persistent storage, distributed compute
|
373 |
+
- **π Heavy Workloads**: All computationally intensive operations
|
374 |
+
|
375 |
+
### **Specific Endpoints:**
|
376 |
+
- `/api/documents/process/:id` - OCR text extraction via Modal
|
377 |
+
- `/api/documents/index/build` - FAISS vector index creation
|
378 |
+
- `/api/documents/search/vector` - High-performance vector search
|
379 |
+
- `/api/documents/process/batch` - Distributed batch processing
|
380 |
+
|
381 |
+
### **Live Deployment**: [Modal App](https://fazeelusmani18--knowledgebridge-main-fastapi-app.modal.run)
|
382 |
+
|
383 |
+
---
|
384 |
+
|
385 |
+
## **π How They Work Together**
|
386 |
+
|
387 |
+
### **Document Processing Pipeline:**
|
388 |
+
1. **Upload** β Local file storage
|
389 |
+
2. **OCR** β **Modal** extracts text from PDFs/images
|
390 |
+
3. **Analysis** β **Nebius** analyzes content and generates embeddings
|
391 |
+
4. **Indexing** β **Modal** builds FAISS vector index
|
392 |
+
5. **Search** β **Modal** performs vector search, **Nebius** scores relevance
|
393 |
+
|
394 |
+
### **Search Workflow:**
|
395 |
+
1. **Query Enhancement** β **Nebius** improves user queries
|
396 |
+
2. **Vector Search** β **Modal** finds similar documents
|
397 |
+
3. **Traditional Search** β Local database + external APIs
|
398 |
+
4. **Ranking** β **Nebius** scores and ranks combined results
|
399 |
+
5. **Synthesis** β **Nebius** generates insights
|
400 |
+
|
401 |
+
---
|
402 |
+
|
403 |
+
## **π Clear Division:**
|
404 |
+
|
405 |
+
| Feature | Nebius AI | Modal.com |
|
406 |
+
|---------|-----------|-----------|
|
407 |
+
| **OCR Processing** | β | β
|
|
408 |
+
| **Document Analysis** | β
| β |
|
409 |
+
| **Vector Search** | β | β
|
|
410 |
+
| **Query Enhancement** | β
| β |
|
411 |
+
| **Batch Processing** | β | β
|
|
412 |
+
| **Embeddings** | β
| β
* |
|
413 |
+
| **Research Synthesis** | β
| β |
|
414 |
+
|
415 |
+
*Modal only for batch embeddings, Nebius for real-time
|
416 |
+
|
417 |
+
**Nebius = "The Brain"** (AI intelligence)
|
418 |
+
**Modal = "The Engine"** (computational power)
|
419 |
|
420 |
### **Intelligent Fallbacks**
|
421 |
- **Modal Unavailable**: Local processing for text files, basic search
|