README.md

---
license: mit
title: Data Extractor Using Gemini
sdk: docker
emoji: 🏃
colorFrom: yellow
colorTo: blue
---
# 📊 Financial Data Extractor Using Gemini

A powerful AI-driven financial document analysis system that automatically extracts, organizes, and generates professional Excel reports from financial documents using Google's Gemini AI models.

## 🚀 Features

### Core Functionality

- **📄 Multi-format Document Support**: PDF, DOCX, TXT, and image files
- **🔍 Intelligent Data Extraction**: AI-powered extraction of financial data points
- **📊 Smart Data Organization**: Automatic categorization into 12+ financial categories
- **💻 Excel Report Generation**: Professional multi-worksheet Excel reports with charts
- **🎯 Real-time Processing**: Live streaming interface with progress tracking

### Advanced Capabilities

- **🤖 Multi-Agent Workflow**: Specialized AI agents for extraction, arrangement, and code generation
- **💾 Session Management**: Persistent storage with SQLite caching
- **🔄 Auto-shutdown**: Intelligent resource management for cloud deployments
- **📱 Modern UI**: Beautiful Gradio-based web interface
- **🌐 Cross-platform**: Works on Windows, Mac, and Linux
- **🐳 Docker Support**: Containerized deployment ready

## 🏗️ Architecture

The system uses a sophisticated multi-agent workflow powered by the Agno framework:

```
📄 Document Upload
    ↓
🔍 Data Extractor Agent
    ↓ (Structured Financial Data)
📊 Data Arranger Agent  
    ↓ (Organized Categories)
💻 Code Generator Agent
    ↓ (Python Excel Code)
📊 Excel Report Output
```

### Agent Specialization

- **Data Extractor**: Extracts financial data points with confidence scoring
- **Data Arranger**: Organizes data into 12+ professional categories
- **Code Generator**: Creates Python code for Excel report generation

## 📋 Requirements

### System Requirements

- Python 3.8+
- Google API Key (for Gemini models)
- 2GB+ RAM recommended
- Cross-platform compatible

### Dependencies

```
agno>=1.7.4
gradio
google-generativeai
PyPDF2
Pillow
python-dotenv
pandas
matplotlib
openpyxl
python-docx
lxml
markdown
requests
seaborn
sqlalchemy
websockets
```

## 🚀 Quick Start

### 1. Clone the Repository

```bash
git clone <repository-url>
cd Data_Extractor_Using_Gemini
```

### 2. Install Dependencies

```bash
pip install -r requirements.txt
```

### 3. Configure Environment

Create a `.env` file:

```env
GOOGLE_API_KEY=your_gemini_api_key_here
```

### 4. Run the Application

```bash
python app.py
```

The application will be available at `http://localhost:7860`

## 🐳 Docker Deployment

### Build and Run

```bash
docker build -t financial-extractor .
docker run -p 7860:7860 --env-file .env financial-extractor
```

### Environment Variables

- `GOOGLE_API_KEY`: Your Google Gemini API key
- `INACTIVITY_TIMEOUT_MINUTES`: Auto-shutdown timeout (default: 30)

## 📖 Usage Guide

### 1. Upload Document

- Drag and drop or select your financial document
- Supported formats: PDF, DOCX, TXT, PNG, JPG, JPEG

### 2. Select Processing Mode

- **Quick Analysis**: Standard extraction and organization
- **Custom Prompts**: Use predefined prompt templates for specific document types

### 3. Monitor Progress

- Real-time streaming interface shows each processing step
- Progress indicators for all workflow stages
- Live terminal output for code execution

### 4. Download Results

- Professional Excel report with multiple worksheets
- Organized data categories with charts and formatting
- All intermediate files available for download

## 📊 Output Structure

The generated Excel reports include:

### Worksheets

- **Summary**: Executive overview with key metrics
- **Revenue**: Income and revenue streams
- **Expenses**: Operating and non-operating expenses
- **Assets**: Current and non-current assets
- **Liabilities**: Short-term and long-term liabilities
- **Equity**: Shareholder equity components
- **Cash Flow**: Cash flow statements
- **Ratios**: Financial ratio analysis
- **Charts**: Visual representations of key data
- **Raw Data**: Original extracted data points

### Features

- Professional formatting with consistent styling
- Interactive charts and visualizations
- Dynamic period handling (auto-detects years/quarters)
- Cross-referenced data validation
- Print-ready layouts

## 🔧 Configuration

### Model Settings

Configure AI models in `config/settings.py`:

- Data Extractor Model
- Data Arranger Model
- Code Generator Model
- Thinking budgets and retry settings

### Prompt Customization

Customize agent instructions in `instructions/agents/`:

- `data_extractor.md`: Data extraction prompts
- `data_arranger.md`: Data organization prompts
- `code_generator.md`: Excel generation prompts

### Workflow Configuration

Modify workflow behavior in `workflow/financial_workflow.py`:

- Agent configurations
- Tool assignments
- Output formats

## 🛠️ Development

### Project Structure

```
├── app.py                 # Main Gradio application
├── workflow/              # Core workflow implementation
├── instructions/          # Agent instruction templates
├── prompts/              # Prompt gallery configurations
├── config/               # Application settings
├── utils/                # Utility functions
├── static/               # Static assets
├── models/               # Data models
└── terminal_stream.py    # Real-time terminal streaming
```

### Key Components

- **WorkflowUI**: Main interface controller
- **FinancialDocumentWorkflow**: Core processing pipeline
- **AutoShutdownManager**: Resource management
- **TerminalLogHandler**: Real-time logging
- **PromptGallery**: Template management

## 🔒 Security & Privacy

- **Local Processing**: All document processing happens locally
- **No Data Storage**: Documents are processed and cleaned up automatically
- **API Key Security**: Environment-based configuration
- **Session Isolation**: Each session has isolated temporary directories

## 🌐 Deployment Options

### Local Development

```bash
python app.py
```

### Production (Gunicorn)

```bash
gunicorn -w 4 -b 0.0.0.0:7860 app:app
```

### Cloud Platforms

- **Hugging Face Spaces**: Ready for deployment
- **Google Cloud Run**: Containerized deployment
- **AWS/Azure**: Standard container deployment

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

## 📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

## 🆘 Support

### Common Issues

- **API Key Errors**: Ensure your Google API key is valid and has Gemini access
- **Memory Issues**: Increase system RAM or reduce document size
- **Processing Timeouts**: Check network connectivity and API quotas