Hugging Face's logo

Spaces:
fireworks-ai
/
scout-claims
Running

App Files Community
scout-claims
File size: 5,009 Bytes 
2684adc
 
 
 
 
 
b384e24
5d8e07e
2684adc
b384e24
2684adc
 
e954acb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
 
---
title: Scout AI Claims Assistant
emoji: 🔍
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: "5.38.0"
app_file: src/app.py
pinned: false
python_version: "3.11"
---

![Scout Demo](assets/fireworks_logo.png)


# Scout | AI Claims Assistant

> **Automated Insurance Claims Processing powered by Fireworks AI**

Scout is an intelligent claims processing assistant that uses advanced AI to analyze vehicle damage, process incident descriptions, and generate comprehensive claim reports. It showcases cutting-edge AI capabilities including computer vision, speech transcription, natural language processing, and autonomous function calling.

## Key Features

- **Automated Damage Analysis**: AI-powered computer vision analyzes damage photos
- **Real-time Speech Transcription**: Live audio processing with Fireworks AI
- **Intelligent Incident Processing**: Advanced NLP extracts structured claim data
- **Autonomous Function Calling**: AI automatically gathers additional context (weather, driver records)
- **Professional PDF Generation**: Comprehensive claim reports with all evidence
- **Real-time Processing**: Sub-30 second end-to-end claim processing

## High-Level Architecture

### Core AI Components (Powered by Fireworks AI)

#### 1. Vision Analysis Module (`image_analysis.py`)
- **Function**: Analyzes damage photos to determine severity, location, and repair estimates
- **Output**: Structured JSON with damage classification and cost estimates

#### 2. Speech Transcription Service (`transcription.py`)
- **Function**: Converts live audio to text with 500ms updates
- **Features**: Automatic speech recognition with live feedback

#### 3. Incident Processing Engine (`incident_processing.py`)
- **Function**: Extracts structured claim data from transcribed incident descriptions
- **Advanced Feature**: **Autonomous Function Calling** (see details below)

#### 4. Report Generation System (`claim_processing.py`)
- **Technology**: AI-driven professional document generation
- **Output**: Comprehensive PDF reports with evidence, analysis, and recommendations
- **Features**: Professional formatting, appendices, and legal disclaimers

### Autonomous Function Calling System
One of Scout's most advanced features is its **autonomous function calling capability**. The AI automatically determines when additional context would be helpful and calls external functions without human intervention.

#### How It Works:

1. **Analysis Phase**: AI analyzes the incident transcript
2. **Decision Making**: Determines which external data would improve assessment accuracy
3. **Autonomous Execution**: Automatically calls relevant functions
4. **Context Integration**: Incorporates results into final claim analysis

#### Available Functions:

| Function | Purpose | Data Retrieved |
|----------|---------|----------------|
| `weather_lookup` | Get weather conditions for incident date/location | Temperature, visibility, precipitation, conditions |
| `driver_record_check` | Verify other party's driving record | License status, insurance status, violation history, risk assessment |

#### Example Function Call Flow:

```
User describes incident → AI extracts date/location → AI calls weather_lookup() →
AI finds other driver name → AI calls driver_record_check() →
AI incorporates weather + driver data into fault assessment
```

This autonomous approach means **no manual intervention required** - the AI intelligently gathers the exact context needed for each unique claim.

## Setup Instructions

### Prerequisites

- Python 3.11+
- Fireworks AI API key ([Get one here](https://fireworks.ai))
- OpenSSL (for HTTPS/microphone access)

### Quick Start

1. **Clone the repository**
   ```bash
   git clone <repository-url>
   cd scout-claims-assistant
   ```

2. **Run automated setup**
   ```bash
   make setup
   ```
   This will:
   - Install `uv` package manager
   - Create Python 3.11 virtual environment
   - Install all dependencies
   - Generate SSL certificates for HTTPS

3. **Set your API key**: add FIREWORKS_API_KEY to .env

4. **Launch the application**
   ```bash
   make run
   ```

5. **Open in browser**
   ```
   https://localhost:7860
   ```
   > **Note**: Accept the security warning for self-signed certificates

## How to Use

### Step 1: Upload Damage Photos
- Upload clear photos of vehicle damage
- AI analyzes damage severity, location, and repair costs
- Results appear in real-time

### Step 2: Record Incident Description
- Click the microphone to start recording
- Describe the incident including:
  - **When & Where**: Date, time, location
  - **Who**: Other parties, witnesses
  - **What**: How the accident happened
  - **Injuries**: Any medical concerns
- Watch live transcription appear as you speak

### Step 3: Generate Professional Report
- AI processes all information + calls external functions
- Generates comprehensive PDF claim report
- Download or submit directly

**Powered by Fireworks AI** | **Built for intelligent claims processing**