Transcendental-Programmer
commited on
Commit
Β·
0af9146
1
Parent(s):
b407fad
fix: fixed server error
Browse files- DEPLOYMENT.md +297 -1
- app.py +172 -114
- webapp/streamlit_app.py +172 -114
DEPLOYMENT.md
CHANGED
@@ -117,4 +117,300 @@ After deployment, you'll have:
|
|
117 |
- β
Professional presentation of your project
|
118 |
- β
Educational value for visitors
|
119 |
|
120 |
-
**Your federated learning demo will be live and working!** π
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
117 |
- β
Professional presentation of your project
|
118 |
- β
Educational value for visitors
|
119 |
|
120 |
+
**Your federated learning demo will be live and working!** π
|
121 |
+
|
122 |
+
# FinFedRAG Deployment Guide
|
123 |
+
|
124 |
+
## Overview
|
125 |
+
|
126 |
+
This project implements a federated learning framework with RAG capabilities for financial data. The system can be deployed using Docker Compose for local development or Kubernetes for production environments.
|
127 |
+
|
128 |
+
## Debugging and Monitoring
|
129 |
+
|
130 |
+
### Enhanced Debugging Features
|
131 |
+
|
132 |
+
The web application now includes comprehensive debugging capabilities:
|
133 |
+
|
134 |
+
1. **Debug Information Panel**: Located in the sidebar, shows:
|
135 |
+
- Real-time server health status
|
136 |
+
- Recent debug messages and logs
|
137 |
+
- Connection error details
|
138 |
+
- Client simulator status
|
139 |
+
|
140 |
+
2. **Detailed Error Logging**: All operations are logged with:
|
141 |
+
- Connection attempts and failures
|
142 |
+
- Server response details
|
143 |
+
- Timeout and network error handling
|
144 |
+
- Client registration and training status updates
|
145 |
+
|
146 |
+
3. **Real-time Status Monitoring**:
|
147 |
+
- Server health checks
|
148 |
+
- Training progress tracking
|
149 |
+
- Client connection status
|
150 |
+
- Error message history
|
151 |
+
|
152 |
+
### Using the Debug Features
|
153 |
+
|
154 |
+
1. **Enable Debug Mode**: Uncheck "Demo Mode" in the sidebar
|
155 |
+
2. **View Debug Information**: Expand the "Debug Information" section in the sidebar
|
156 |
+
3. **Monitor Logs**: Check the "Recent Logs" section for real-time updates
|
157 |
+
4. **Clear Logs**: Use the "Clear Debug Logs" button to reset the log history
|
158 |
+
|
159 |
+
## Local Development Setup
|
160 |
+
|
161 |
+
### Prerequisites
|
162 |
+
|
163 |
+
- Python 3.8+
|
164 |
+
- Docker and Docker Compose
|
165 |
+
- Required Python packages (see requirements.txt)
|
166 |
+
|
167 |
+
### Quick Start
|
168 |
+
|
169 |
+
1. **Clone and Setup**:
|
170 |
+
```bash
|
171 |
+
git clone <repository-url>
|
172 |
+
cd FinFedRAG-Financial-Federated-RAG
|
173 |
+
python -m venv venv
|
174 |
+
source venv/bin/activate # On Windows: venv\Scripts\activate
|
175 |
+
pip install -r requirements.txt
|
176 |
+
```
|
177 |
+
|
178 |
+
2. **Start the Federated Server**:
|
179 |
+
```bash
|
180 |
+
python src/main.py --mode server
|
181 |
+
```
|
182 |
+
|
183 |
+
3. **Start Multiple Clients** (in separate terminals):
|
184 |
+
```bash
|
185 |
+
python src/main.py --mode client --client-id client1
|
186 |
+
python src/main.py --mode client --client-id client2
|
187 |
+
python src/main.py --mode client --client-id client3
|
188 |
+
```
|
189 |
+
|
190 |
+
4. **Run the Web Application**:
|
191 |
+
```bash
|
192 |
+
streamlit run app.py
|
193 |
+
```
|
194 |
+
|
195 |
+
### Docker Compose Deployment
|
196 |
+
|
197 |
+
For containerized deployment:
|
198 |
+
|
199 |
+
```bash
|
200 |
+
cd docker
|
201 |
+
docker-compose up --build
|
202 |
+
```
|
203 |
+
|
204 |
+
This will start:
|
205 |
+
- 1 federated server on port 8000
|
206 |
+
- 3 federated clients
|
207 |
+
- All services connected via Docker network
|
208 |
+
|
209 |
+
## Kubernetes Deployment
|
210 |
+
|
211 |
+
### Architecture Overview
|
212 |
+
|
213 |
+
The Kubernetes setup provides a production-ready deployment with:
|
214 |
+
|
215 |
+
- **Server Deployment**: Single federated learning server
|
216 |
+
- **Client Deployment**: Multiple federated learning clients (3 replicas)
|
217 |
+
- **Service Layer**: Internal service discovery
|
218 |
+
- **ConfigMaps**: Configuration management
|
219 |
+
- **Namespace Isolation**: Dedicated `federated-learning` namespace
|
220 |
+
|
221 |
+
### Components
|
222 |
+
|
223 |
+
#### 1. Server Deployment (`kubernetes/deployments/server.yaml`)
|
224 |
+
```yaml
|
225 |
+
- Replicas: 1 (single server instance)
|
226 |
+
- Port: 8000 (internal)
|
227 |
+
- Config: Mounted from ConfigMap
|
228 |
+
- Image: fl-server:latest
|
229 |
+
```
|
230 |
+
|
231 |
+
#### 2. Client Deployment (`kubernetes/deployments/client.yaml`)
|
232 |
+
```yaml
|
233 |
+
- Replicas: 3 (multiple client instances)
|
234 |
+
- Environment: SERVER_HOST=fl-server-service
|
235 |
+
- Config: Mounted from ConfigMap
|
236 |
+
- Image: fl-client:latest
|
237 |
+
```
|
238 |
+
|
239 |
+
#### 3. Service (`kubernetes/services/service.yaml`)
|
240 |
+
```yaml
|
241 |
+
- Type: ClusterIP (internal communication)
|
242 |
+
- Port: 8000
|
243 |
+
- Selector: app=fl-server
|
244 |
+
```
|
245 |
+
|
246 |
+
### Deployment Steps
|
247 |
+
|
248 |
+
1. **Build Docker Images**:
|
249 |
+
```bash
|
250 |
+
docker build -f docker/Dockerfile.server -t fl-server:latest .
|
251 |
+
docker build -f docker/Dockerfile.client -t fl-client:latest .
|
252 |
+
```
|
253 |
+
|
254 |
+
2. **Create Namespace**:
|
255 |
+
```bash
|
256 |
+
kubectl create namespace federated-learning
|
257 |
+
```
|
258 |
+
|
259 |
+
3. **Create ConfigMaps**:
|
260 |
+
```bash
|
261 |
+
kubectl create configmap server-config --from-file=config/server_config.yaml -n federated-learning
|
262 |
+
kubectl create configmap client-config --from-file=config/client_config.yaml -n federated-learning
|
263 |
+
```
|
264 |
+
|
265 |
+
4. **Deploy Services**:
|
266 |
+
```bash
|
267 |
+
kubectl apply -f kubernetes/services/service.yaml
|
268 |
+
kubectl apply -f kubernetes/deployments/server.yaml
|
269 |
+
kubectl apply -f kubernetes/deployments/client.yaml
|
270 |
+
```
|
271 |
+
|
272 |
+
5. **Verify Deployment**:
|
273 |
+
```bash
|
274 |
+
kubectl get pods -n federated-learning
|
275 |
+
kubectl get services -n federated-learning
|
276 |
+
```
|
277 |
+
|
278 |
+
### Accessing the Application
|
279 |
+
|
280 |
+
#### Option 1: Port Forwarding
|
281 |
+
```bash
|
282 |
+
kubectl port-forward service/fl-server-service 8080:8000 -n federated-learning
|
283 |
+
```
|
284 |
+
|
285 |
+
#### Option 2: Load Balancer
|
286 |
+
Modify the service to use LoadBalancer type:
|
287 |
+
```yaml
|
288 |
+
apiVersion: v1
|
289 |
+
kind: Service
|
290 |
+
metadata:
|
291 |
+
name: fl-server-service
|
292 |
+
namespace: federated-learning
|
293 |
+
spec:
|
294 |
+
type: LoadBalancer # Changed from ClusterIP
|
295 |
+
selector:
|
296 |
+
app: fl-server
|
297 |
+
ports:
|
298 |
+
- port: 8080
|
299 |
+
targetPort: 8000
|
300 |
+
```
|
301 |
+
|
302 |
+
#### Option 3: Ingress Controller
|
303 |
+
Create an ingress resource for external access:
|
304 |
+
```yaml
|
305 |
+
apiVersion: networking.k8s.io/v1
|
306 |
+
kind: Ingress
|
307 |
+
metadata:
|
308 |
+
name: fl-ingress
|
309 |
+
namespace: federated-learning
|
310 |
+
spec:
|
311 |
+
rules:
|
312 |
+
- host: fl.example.com
|
313 |
+
http:
|
314 |
+
paths:
|
315 |
+
- path: /
|
316 |
+
pathType: Prefix
|
317 |
+
backend:
|
318 |
+
service:
|
319 |
+
name: fl-server-service
|
320 |
+
port:
|
321 |
+
number: 8000
|
322 |
+
```
|
323 |
+
|
324 |
+
### Monitoring and Debugging in Kubernetes
|
325 |
+
|
326 |
+
1. **View Pod Logs**:
|
327 |
+
```bash
|
328 |
+
kubectl logs -f deployment/fl-server -n federated-learning
|
329 |
+
kubectl logs -f deployment/fl-client -n federated-learning
|
330 |
+
```
|
331 |
+
|
332 |
+
2. **Check Pod Status**:
|
333 |
+
```bash
|
334 |
+
kubectl describe pods -n federated-learning
|
335 |
+
```
|
336 |
+
|
337 |
+
3. **Access Pod Shell**:
|
338 |
+
```bash
|
339 |
+
kubectl exec -it <pod-name> -n federated-learning -- /bin/bash
|
340 |
+
```
|
341 |
+
|
342 |
+
4. **Monitor Resource Usage**:
|
343 |
+
```bash
|
344 |
+
kubectl top pods -n federated-learning
|
345 |
+
```
|
346 |
+
|
347 |
+
## Troubleshooting
|
348 |
+
|
349 |
+
### Common Issues
|
350 |
+
|
351 |
+
1. **Connection Refused Errors**:
|
352 |
+
- Check if server is running: `kubectl get pods -n federated-learning`
|
353 |
+
- Verify service exists: `kubectl get services -n federated-learning`
|
354 |
+
- Check pod logs for startup errors
|
355 |
+
|
356 |
+
2. **Client Registration Failures**:
|
357 |
+
- Ensure server is healthy before starting clients
|
358 |
+
- Check network connectivity between pods
|
359 |
+
- Verify ConfigMap configurations
|
360 |
+
|
361 |
+
3. **Training Status Issues**:
|
362 |
+
- Monitor server logs for aggregation errors
|
363 |
+
- Check client participation in training rounds
|
364 |
+
- Verify model update sharing
|
365 |
+
|
366 |
+
### Debug Commands
|
367 |
+
|
368 |
+
```bash
|
369 |
+
# Check all resources in namespace
|
370 |
+
kubectl get all -n federated-learning
|
371 |
+
|
372 |
+
# View detailed pod information
|
373 |
+
kubectl describe pod <pod-name> -n federated-learning
|
374 |
+
|
375 |
+
# Check service endpoints
|
376 |
+
kubectl get endpoints -n federated-learning
|
377 |
+
|
378 |
+
# View ConfigMap contents
|
379 |
+
kubectl get configmap server-config -n federated-learning -o yaml
|
380 |
+
```
|
381 |
+
|
382 |
+
## Production Considerations
|
383 |
+
|
384 |
+
1. **Resource Limits**: Add resource requests and limits to deployments
|
385 |
+
2. **Health Checks**: Implement liveness and readiness probes
|
386 |
+
3. **Secrets Management**: Use Kubernetes secrets for sensitive data
|
387 |
+
4. **Persistent Storage**: Add persistent volumes for model storage
|
388 |
+
5. **Monitoring**: Integrate with Prometheus/Grafana for metrics
|
389 |
+
6. **Logging**: Use centralized logging (ELK stack, Fluentd)
|
390 |
+
|
391 |
+
## Scaling
|
392 |
+
|
393 |
+
### Horizontal Pod Autoscaling
|
394 |
+
```yaml
|
395 |
+
apiVersion: autoscaling/v2
|
396 |
+
kind: HorizontalPodAutoscaler
|
397 |
+
metadata:
|
398 |
+
name: fl-client-hpa
|
399 |
+
namespace: federated-learning
|
400 |
+
spec:
|
401 |
+
scaleTargetRef:
|
402 |
+
apiVersion: apps/v1
|
403 |
+
kind: Deployment
|
404 |
+
name: fl-client
|
405 |
+
minReplicas: 3
|
406 |
+
maxReplicas: 10
|
407 |
+
metrics:
|
408 |
+
- type: Resource
|
409 |
+
resource:
|
410 |
+
name: cpu
|
411 |
+
target:
|
412 |
+
type: Utilization
|
413 |
+
averageUtilization: 70
|
414 |
+
```
|
415 |
+
|
416 |
+
This deployment guide provides comprehensive information for both local development and production Kubernetes deployment, with enhanced debugging capabilities for better monitoring and troubleshooting.
|
app.py
CHANGED
@@ -4,9 +4,14 @@ import numpy as np
|
|
4 |
import time
|
5 |
import threading
|
6 |
import json
|
|
|
7 |
from datetime import datetime
|
8 |
|
9 |
-
#
|
|
|
|
|
|
|
|
|
10 |
class ClientSimulator:
|
11 |
def __init__(self, server_url):
|
12 |
self.server_url = server_url
|
@@ -14,18 +19,21 @@ class ClientSimulator:
|
|
14 |
self.is_running = False
|
15 |
self.thread = None
|
16 |
self.last_update = "Never"
|
|
|
17 |
|
18 |
def start(self):
|
19 |
self.is_running = True
|
20 |
self.thread = threading.Thread(target=self._run_client, daemon=True)
|
21 |
self.thread.start()
|
|
|
22 |
|
23 |
def stop(self):
|
24 |
self.is_running = False
|
|
|
25 |
|
26 |
def _run_client(self):
|
27 |
try:
|
28 |
-
|
29 |
client_info = {
|
30 |
'dataset_size': 100,
|
31 |
'model_params': 10000,
|
@@ -33,9 +41,11 @@ class ClientSimulator:
|
|
33 |
}
|
34 |
|
35 |
resp = requests.post(f"{self.server_url}/register",
|
36 |
-
json={'client_id': self.client_id, 'client_info': client_info}
|
|
|
37 |
|
38 |
if resp.status_code == 200:
|
|
|
39 |
st.session_state.training_history.append({
|
40 |
'round': 0,
|
41 |
'active_clients': 1,
|
@@ -43,15 +53,13 @@ class ClientSimulator:
|
|
43 |
'timestamp': datetime.now()
|
44 |
})
|
45 |
|
46 |
-
# Simulate client participation
|
47 |
while self.is_running:
|
48 |
try:
|
49 |
-
|
50 |
-
status = requests.get(f"{self.server_url}/training_status")
|
51 |
if status.status_code == 200:
|
52 |
data = status.json()
|
53 |
-
|
54 |
-
# Update training history
|
55 |
st.session_state.training_history.append({
|
56 |
'round': data.get('current_round', 0),
|
57 |
'active_clients': data.get('active_clients', 0),
|
@@ -59,56 +67,127 @@ class ClientSimulator:
|
|
59 |
'timestamp': datetime.now()
|
60 |
})
|
61 |
|
62 |
-
# Keep only last 50 entries
|
63 |
if len(st.session_state.training_history) > 50:
|
64 |
st.session_state.training_history = st.session_state.training_history[-50:]
|
|
|
|
|
65 |
|
66 |
-
time.sleep(5)
|
67 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
68 |
except Exception as e:
|
69 |
-
|
|
|
70 |
time.sleep(10)
|
71 |
|
|
|
|
|
|
|
|
|
72 |
except Exception as e:
|
73 |
-
|
|
|
74 |
self.is_running = False
|
75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
76 |
st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
|
77 |
-
st.title("Federated Credit Scoring Demo
|
78 |
|
79 |
# Sidebar configuration
|
80 |
st.sidebar.header("Configuration")
|
81 |
SERVER_URL = st.sidebar.text_input("Server URL", value="http://localhost:8080")
|
82 |
-
DEMO_MODE = st.sidebar.checkbox("Demo Mode
|
83 |
|
84 |
# Initialize session state
|
85 |
if 'client_simulator' not in st.session_state:
|
86 |
st.session_state.client_simulator = None
|
87 |
if 'training_history' not in st.session_state:
|
88 |
st.session_state.training_history = []
|
|
|
|
|
89 |
|
90 |
-
|
91 |
-
|
92 |
-
|
93 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
94 |
|
95 |
-
#
|
96 |
st.sidebar.header("Client Simulator")
|
97 |
-
if st.sidebar.button("Start Client
|
98 |
if not DEMO_MODE:
|
99 |
-
|
100 |
-
|
101 |
-
|
|
|
|
|
|
|
|
|
|
|
102 |
else:
|
103 |
-
st.sidebar.warning("
|
104 |
|
105 |
-
if st.sidebar.button("Stop Client
|
106 |
if st.session_state.client_simulator:
|
107 |
st.session_state.client_simulator.stop()
|
108 |
st.session_state.client_simulator = None
|
109 |
-
st.sidebar.success("Client
|
|
|
110 |
|
111 |
-
#
|
112 |
st.header("Enter Customer Features")
|
113 |
with st.form("feature_form"):
|
114 |
features = []
|
@@ -119,124 +198,103 @@ with st.form("feature_form"):
|
|
119 |
features.append(val)
|
120 |
submitted = st.form_submit_button("Predict Credit Score")
|
121 |
|
122 |
-
#
|
123 |
if submitted:
|
|
|
124 |
if DEMO_MODE:
|
125 |
-
|
126 |
-
|
127 |
-
|
128 |
-
|
129 |
-
|
130 |
-
demo_prediction = sum(features) / len(features) * 100 + 500 # Scale to credit score range
|
131 |
-
st.success(f"Demo Prediction: Credit Score = {demo_prediction:.2f}")
|
132 |
-
st.info("π‘ This is a demo prediction. In a real federated system, this would come from the trained model.")
|
133 |
-
|
134 |
-
# Show what would happen in real mode
|
135 |
-
st.markdown("---")
|
136 |
-
st.markdown("**What happens in real federated learning:**")
|
137 |
-
st.markdown("1. Your features are sent to the federated server")
|
138 |
-
st.markdown("2. Server uses the global model (trained by multiple banks)")
|
139 |
-
st.markdown("3. Prediction is returned without exposing any bank's data")
|
140 |
-
|
141 |
else:
|
142 |
-
# Real mode - connect to server
|
143 |
try:
|
144 |
-
|
|
|
145 |
resp = requests.post(f"{SERVER_URL}/predict", json={"features": features}, timeout=10)
|
146 |
|
147 |
if resp.status_code == 200:
|
148 |
prediction = resp.json().get("prediction")
|
149 |
st.success(f"Predicted Credit Score: {prediction:.2f}")
|
150 |
-
st.
|
|
|
151 |
else:
|
152 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
153 |
except Exception as e:
|
154 |
-
|
155 |
-
st.
|
156 |
-
|
157 |
-
|
158 |
-
st.header("Federated Training Progress")
|
159 |
|
|
|
|
|
160 |
if DEMO_MODE:
|
161 |
-
# Demo training progress
|
162 |
col1, col2, col3, col4 = st.columns(4)
|
163 |
with col1:
|
164 |
-
st.metric("
|
165 |
with col2:
|
166 |
-
st.metric("
|
167 |
with col3:
|
168 |
-
st.metric("
|
169 |
with col4:
|
170 |
-
st.metric("
|
171 |
-
|
172 |
-
st.info("π‘ Demo mode showing simulated training progress. In real federated learning, multiple banks would be training collaboratively.")
|
173 |
-
|
174 |
else:
|
175 |
-
# Real training progress
|
176 |
try:
|
|
|
177 |
status = requests.get(f"{SERVER_URL}/training_status", timeout=5)
|
178 |
if status.status_code == 200:
|
179 |
data = status.json()
|
|
|
180 |
col1, col2, col3, col4 = st.columns(4)
|
181 |
with col1:
|
182 |
-
st.metric("
|
183 |
with col2:
|
184 |
-
st.metric("
|
185 |
with col3:
|
186 |
-
st.metric("
|
187 |
with col4:
|
188 |
-
st.metric("
|
189 |
-
|
190 |
-
# Show training history
|
191 |
-
if st.session_state.training_history:
|
192 |
-
st.subheader("Training History")
|
193 |
-
history_df = st.session_state.training_history
|
194 |
-
st.line_chart(history_df.set_index('round')[['active_clients', 'clients_ready']])
|
195 |
else:
|
196 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
197 |
except Exception as e:
|
198 |
-
|
|
|
|
|
|
|
199 |
|
200 |
-
#
|
201 |
-
if not DEMO_MODE:
|
202 |
-
st.header("Server Health")
|
203 |
-
try:
|
204 |
-
health = requests.get(f"{SERVER_URL}/health", timeout=5)
|
205 |
-
if health.status_code == 200:
|
206 |
-
health_data = health.json()
|
207 |
-
st.success(f"β
Server is healthy")
|
208 |
-
st.json(health_data)
|
209 |
-
else:
|
210 |
-
st.error("β Server health check failed")
|
211 |
-
except Exception as e:
|
212 |
-
st.error(f"β Cannot connect to server: {e}")
|
213 |
-
|
214 |
-
# --- How it works ---
|
215 |
-
st.header("How Federated Learning Works")
|
216 |
-
st.markdown("""
|
217 |
-
**Traditional ML:** All banks send their data to a central server β Privacy risk β
|
218 |
-
|
219 |
-
**Federated Learning:**
|
220 |
-
1. Each bank keeps their data locally β
|
221 |
-
2. Banks train models on their own data β
|
222 |
-
3. Only model updates (not data) are shared β
|
223 |
-
4. Server aggregates updates to create global model β
|
224 |
-
5. Global model is distributed back to all banks β
|
225 |
-
|
226 |
-
**Result:** Collaborative learning without data sharing! π―
|
227 |
-
""")
|
228 |
-
|
229 |
-
# --- Client Simulator Status ---
|
230 |
if st.session_state.client_simulator and not DEMO_MODE:
|
231 |
-
st.header("Client
|
232 |
if st.session_state.client_simulator.is_running:
|
233 |
-
st.success("
|
234 |
-
st.info(f"
|
235 |
-
|
|
|
236 |
else:
|
237 |
-
st.warning("
|
238 |
-
|
239 |
-
st.markdown("---")
|
240 |
-
st.markdown("""
|
241 |
-
*This is a demonstration of federated learning concepts. For full functionality, run the federated server and clients locally.*
|
242 |
-
""")
|
|
|
4 |
import time
|
5 |
import threading
|
6 |
import json
|
7 |
+
import logging
|
8 |
from datetime import datetime
|
9 |
|
10 |
+
# Configure logging
|
11 |
+
logging.basicConfig(level=logging.DEBUG)
|
12 |
+
logger = logging.getLogger(__name__)
|
13 |
+
|
14 |
+
# Client Simulator Class
|
15 |
class ClientSimulator:
|
16 |
def __init__(self, server_url):
|
17 |
self.server_url = server_url
|
|
|
19 |
self.is_running = False
|
20 |
self.thread = None
|
21 |
self.last_update = "Never"
|
22 |
+
self.last_error = None
|
23 |
|
24 |
def start(self):
|
25 |
self.is_running = True
|
26 |
self.thread = threading.Thread(target=self._run_client, daemon=True)
|
27 |
self.thread.start()
|
28 |
+
logger.info(f"Client simulator started for {self.server_url}")
|
29 |
|
30 |
def stop(self):
|
31 |
self.is_running = False
|
32 |
+
logger.info("Client simulator stopped")
|
33 |
|
34 |
def _run_client(self):
|
35 |
try:
|
36 |
+
logger.info(f"Attempting to register client {self.client_id} with server {self.server_url}")
|
37 |
client_info = {
|
38 |
'dataset_size': 100,
|
39 |
'model_params': 10000,
|
|
|
41 |
}
|
42 |
|
43 |
resp = requests.post(f"{self.server_url}/register",
|
44 |
+
json={'client_id': self.client_id, 'client_info': client_info},
|
45 |
+
timeout=10)
|
46 |
|
47 |
if resp.status_code == 200:
|
48 |
+
logger.info(f"Successfully registered client {self.client_id}")
|
49 |
st.session_state.training_history.append({
|
50 |
'round': 0,
|
51 |
'active_clients': 1,
|
|
|
53 |
'timestamp': datetime.now()
|
54 |
})
|
55 |
|
|
|
56 |
while self.is_running:
|
57 |
try:
|
58 |
+
logger.debug(f"Checking training status from {self.server_url}/training_status")
|
59 |
+
status = requests.get(f"{self.server_url}/training_status", timeout=5)
|
60 |
if status.status_code == 200:
|
61 |
data = status.json()
|
62 |
+
logger.debug(f"Training status: {data}")
|
|
|
63 |
st.session_state.training_history.append({
|
64 |
'round': data.get('current_round', 0),
|
65 |
'active_clients': data.get('active_clients', 0),
|
|
|
67 |
'timestamp': datetime.now()
|
68 |
})
|
69 |
|
|
|
70 |
if len(st.session_state.training_history) > 50:
|
71 |
st.session_state.training_history = st.session_state.training_history[-50:]
|
72 |
+
else:
|
73 |
+
logger.warning(f"Training status returned {status.status_code}: {status.text}")
|
74 |
|
75 |
+
time.sleep(5)
|
76 |
|
77 |
+
except requests.exceptions.Timeout:
|
78 |
+
logger.warning("Timeout while checking training status")
|
79 |
+
self.last_error = "Timeout connecting to server"
|
80 |
+
time.sleep(10)
|
81 |
+
except requests.exceptions.ConnectionError as e:
|
82 |
+
logger.error(f"Connection error while checking training status: {e}")
|
83 |
+
self.last_error = f"Connection error: {e}"
|
84 |
+
time.sleep(10)
|
85 |
except Exception as e:
|
86 |
+
logger.error(f"Unexpected error in client simulator: {e}")
|
87 |
+
self.last_error = f"Unexpected error: {e}"
|
88 |
time.sleep(10)
|
89 |
|
90 |
+
except requests.exceptions.ConnectionError as e:
|
91 |
+
logger.error(f"Failed to connect to server {self.server_url}: {e}")
|
92 |
+
self.last_error = f"Failed to connect to server: {e}"
|
93 |
+
self.is_running = False
|
94 |
except Exception as e:
|
95 |
+
logger.error(f"Failed to start client simulator: {e}")
|
96 |
+
self.last_error = f"Failed to start: {e}"
|
97 |
self.is_running = False
|
98 |
|
99 |
+
def check_server_health(server_url):
|
100 |
+
"""Check if server is reachable and healthy"""
|
101 |
+
try:
|
102 |
+
logger.debug(f"Checking server health at {server_url}/health")
|
103 |
+
resp = requests.get(f"{server_url}/health", timeout=5)
|
104 |
+
if resp.status_code == 200:
|
105 |
+
logger.info("Server is healthy")
|
106 |
+
return True, resp.json()
|
107 |
+
else:
|
108 |
+
logger.warning(f"Server health check returned {resp.status_code}")
|
109 |
+
return False, f"HTTP {resp.status_code}: {resp.text}"
|
110 |
+
except requests.exceptions.Timeout:
|
111 |
+
logger.error("Server health check timeout")
|
112 |
+
return False, "Timeout"
|
113 |
+
except requests.exceptions.ConnectionError as e:
|
114 |
+
logger.error(f"Server health check connection error: {e}")
|
115 |
+
return False, f"Connection refused: {e}"
|
116 |
+
except Exception as e:
|
117 |
+
logger.error(f"Server health check unexpected error: {e}")
|
118 |
+
return False, f"Unexpected error: {e}"
|
119 |
+
|
120 |
st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
|
121 |
+
st.title("Federated Credit Scoring Demo")
|
122 |
|
123 |
# Sidebar configuration
|
124 |
st.sidebar.header("Configuration")
|
125 |
SERVER_URL = st.sidebar.text_input("Server URL", value="http://localhost:8080")
|
126 |
+
DEMO_MODE = st.sidebar.checkbox("Demo Mode", value=True)
|
127 |
|
128 |
# Initialize session state
|
129 |
if 'client_simulator' not in st.session_state:
|
130 |
st.session_state.client_simulator = None
|
131 |
if 'training_history' not in st.session_state:
|
132 |
st.session_state.training_history = []
|
133 |
+
if 'debug_messages' not in st.session_state:
|
134 |
+
st.session_state.debug_messages = []
|
135 |
|
136 |
+
# Debug section in sidebar
|
137 |
+
with st.sidebar.expander("Debug Information"):
|
138 |
+
st.write("**Server Status:**")
|
139 |
+
if not DEMO_MODE:
|
140 |
+
is_healthy, health_info = check_server_health(SERVER_URL)
|
141 |
+
if is_healthy:
|
142 |
+
st.success("β
Server is healthy")
|
143 |
+
st.json(health_info)
|
144 |
+
else:
|
145 |
+
st.error(f"β Server error: {health_info}")
|
146 |
+
|
147 |
+
st.write("**Recent Logs:**")
|
148 |
+
if st.session_state.debug_messages:
|
149 |
+
for msg in st.session_state.debug_messages[-5:]: # Show last 5 messages
|
150 |
+
st.text(msg)
|
151 |
+
else:
|
152 |
+
st.text("No debug messages yet")
|
153 |
+
|
154 |
+
if st.button("Clear Debug Logs"):
|
155 |
+
st.session_state.debug_messages = []
|
156 |
+
|
157 |
+
# Sidebar educational content
|
158 |
+
with st.sidebar.expander("About Federated Learning"):
|
159 |
+
st.markdown("""
|
160 |
+
**Traditional ML:** Banks send data to central server β Privacy risk
|
161 |
+
|
162 |
+
**Federated Learning:**
|
163 |
+
- Banks keep data locally
|
164 |
+
- Only model updates are shared
|
165 |
+
- Collaborative learning without data sharing
|
166 |
+
""")
|
167 |
|
168 |
+
# Client Simulator in sidebar
|
169 |
st.sidebar.header("Client Simulator")
|
170 |
+
if st.sidebar.button("Start Client"):
|
171 |
if not DEMO_MODE:
|
172 |
+
try:
|
173 |
+
st.session_state.client_simulator = ClientSimulator(SERVER_URL)
|
174 |
+
st.session_state.client_simulator.start()
|
175 |
+
st.sidebar.success("Client started!")
|
176 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Client simulator started")
|
177 |
+
except Exception as e:
|
178 |
+
st.sidebar.error(f"Failed to start client: {e}")
|
179 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Failed to start client - {e}")
|
180 |
else:
|
181 |
+
st.sidebar.warning("Only works in Real Mode")
|
182 |
|
183 |
+
if st.sidebar.button("Stop Client"):
|
184 |
if st.session_state.client_simulator:
|
185 |
st.session_state.client_simulator.stop()
|
186 |
st.session_state.client_simulator = None
|
187 |
+
st.sidebar.success("Client stopped!")
|
188 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Client simulator stopped")
|
189 |
|
190 |
+
# Main content - focused on core functionality
|
191 |
st.header("Enter Customer Features")
|
192 |
with st.form("feature_form"):
|
193 |
features = []
|
|
|
198 |
features.append(val)
|
199 |
submitted = st.form_submit_button("Predict Credit Score")
|
200 |
|
201 |
+
# Prediction results
|
202 |
if submitted:
|
203 |
+
logger.info(f"Prediction requested with {len(features)} features")
|
204 |
if DEMO_MODE:
|
205 |
+
with st.spinner("Processing..."):
|
206 |
+
time.sleep(1)
|
207 |
+
demo_prediction = sum(features) / len(features) * 100 + 500
|
208 |
+
st.success(f"Predicted Credit Score: {demo_prediction:.2f}")
|
209 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Demo prediction: {demo_prediction:.2f}")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
210 |
else:
|
|
|
211 |
try:
|
212 |
+
logger.info(f"Sending prediction request to {SERVER_URL}/predict")
|
213 |
+
with st.spinner("Connecting to server..."):
|
214 |
resp = requests.post(f"{SERVER_URL}/predict", json={"features": features}, timeout=10)
|
215 |
|
216 |
if resp.status_code == 200:
|
217 |
prediction = resp.json().get("prediction")
|
218 |
st.success(f"Predicted Credit Score: {prediction:.2f}")
|
219 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Real prediction: {prediction:.2f}")
|
220 |
+
logger.info(f"Prediction successful: {prediction}")
|
221 |
else:
|
222 |
+
error_msg = f"Prediction failed: {resp.json().get('error', 'Unknown error')}"
|
223 |
+
st.error(error_msg)
|
224 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
225 |
+
logger.error(f"Prediction failed with status {resp.status_code}: {resp.text}")
|
226 |
+
except requests.exceptions.Timeout:
|
227 |
+
error_msg = "Timeout connecting to server"
|
228 |
+
st.error(error_msg)
|
229 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
230 |
+
logger.error("Prediction request timeout")
|
231 |
+
except requests.exceptions.ConnectionError as e:
|
232 |
+
error_msg = f"Connection error: {e}"
|
233 |
+
st.error(error_msg)
|
234 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
235 |
+
logger.error(f"Prediction connection error: {e}")
|
236 |
except Exception as e:
|
237 |
+
error_msg = f"Unexpected error: {e}"
|
238 |
+
st.error(error_msg)
|
239 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
240 |
+
logger.error(f"Prediction unexpected error: {e}")
|
|
|
241 |
|
242 |
+
# Training progress - simplified
|
243 |
+
st.header("Training Progress")
|
244 |
if DEMO_MODE:
|
|
|
245 |
col1, col2, col3, col4 = st.columns(4)
|
246 |
with col1:
|
247 |
+
st.metric("Round", "3/10")
|
248 |
with col2:
|
249 |
+
st.metric("Clients", "3")
|
250 |
with col3:
|
251 |
+
st.metric("Accuracy", "85.2%")
|
252 |
with col4:
|
253 |
+
st.metric("Status", "Active")
|
|
|
|
|
|
|
254 |
else:
|
|
|
255 |
try:
|
256 |
+
logger.debug(f"Fetching training status from {SERVER_URL}/training_status")
|
257 |
status = requests.get(f"{SERVER_URL}/training_status", timeout=5)
|
258 |
if status.status_code == 200:
|
259 |
data = status.json()
|
260 |
+
logger.debug(f"Training status received: {data}")
|
261 |
col1, col2, col3, col4 = st.columns(4)
|
262 |
with col1:
|
263 |
+
st.metric("Round", f"{data.get('current_round', 0)}/{data.get('total_rounds', 10)}")
|
264 |
with col2:
|
265 |
+
st.metric("Clients", data.get('active_clients', 0))
|
266 |
with col3:
|
267 |
+
st.metric("Ready", data.get('clients_ready', 0))
|
268 |
with col4:
|
269 |
+
st.metric("Status", "Active" if data.get('training_active', False) else "Inactive")
|
|
|
|
|
|
|
|
|
|
|
|
|
270 |
else:
|
271 |
+
error_msg = f"Training status failed: HTTP {status.status_code}"
|
272 |
+
st.warning(error_msg)
|
273 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
274 |
+
logger.warning(f"Training status returned {status.status_code}: {status.text}")
|
275 |
+
except requests.exceptions.Timeout:
|
276 |
+
error_msg = "Training status timeout"
|
277 |
+
st.warning(error_msg)
|
278 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
279 |
+
logger.warning("Training status request timeout")
|
280 |
+
except requests.exceptions.ConnectionError as e:
|
281 |
+
error_msg = f"Training status connection error: {e}"
|
282 |
+
st.warning(error_msg)
|
283 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
284 |
+
logger.error(f"Training status connection error: {e}")
|
285 |
except Exception as e:
|
286 |
+
error_msg = f"Training status unexpected error: {e}"
|
287 |
+
st.warning(error_msg)
|
288 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
289 |
+
logger.error(f"Training status unexpected error: {e}")
|
290 |
|
291 |
+
# Client status in sidebar
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
292 |
if st.session_state.client_simulator and not DEMO_MODE:
|
293 |
+
st.sidebar.header("Client Status")
|
294 |
if st.session_state.client_simulator.is_running:
|
295 |
+
st.sidebar.success("Connected")
|
296 |
+
st.sidebar.info(f"ID: {st.session_state.client_simulator.client_id}")
|
297 |
+
if st.session_state.client_simulator.last_error:
|
298 |
+
st.sidebar.error(f"Last Error: {st.session_state.client_simulator.last_error}")
|
299 |
else:
|
300 |
+
st.sidebar.warning("Disconnected")
|
|
|
|
|
|
|
|
|
|
webapp/streamlit_app.py
CHANGED
@@ -4,9 +4,14 @@ import numpy as np
|
|
4 |
import time
|
5 |
import threading
|
6 |
import json
|
|
|
7 |
from datetime import datetime
|
8 |
|
9 |
-
#
|
|
|
|
|
|
|
|
|
10 |
class ClientSimulator:
|
11 |
def __init__(self, server_url):
|
12 |
self.server_url = server_url
|
@@ -14,18 +19,21 @@ class ClientSimulator:
|
|
14 |
self.is_running = False
|
15 |
self.thread = None
|
16 |
self.last_update = "Never"
|
|
|
17 |
|
18 |
def start(self):
|
19 |
self.is_running = True
|
20 |
self.thread = threading.Thread(target=self._run_client, daemon=True)
|
21 |
self.thread.start()
|
|
|
22 |
|
23 |
def stop(self):
|
24 |
self.is_running = False
|
|
|
25 |
|
26 |
def _run_client(self):
|
27 |
try:
|
28 |
-
|
29 |
client_info = {
|
30 |
'dataset_size': 100,
|
31 |
'model_params': 10000,
|
@@ -33,9 +41,11 @@ class ClientSimulator:
|
|
33 |
}
|
34 |
|
35 |
resp = requests.post(f"{self.server_url}/register",
|
36 |
-
json={'client_id': self.client_id, 'client_info': client_info}
|
|
|
37 |
|
38 |
if resp.status_code == 200:
|
|
|
39 |
st.session_state.training_history.append({
|
40 |
'round': 0,
|
41 |
'active_clients': 1,
|
@@ -43,15 +53,13 @@ class ClientSimulator:
|
|
43 |
'timestamp': datetime.now()
|
44 |
})
|
45 |
|
46 |
-
# Simulate client participation
|
47 |
while self.is_running:
|
48 |
try:
|
49 |
-
|
50 |
-
status = requests.get(f"{self.server_url}/training_status")
|
51 |
if status.status_code == 200:
|
52 |
data = status.json()
|
53 |
-
|
54 |
-
# Update training history
|
55 |
st.session_state.training_history.append({
|
56 |
'round': data.get('current_round', 0),
|
57 |
'active_clients': data.get('active_clients', 0),
|
@@ -59,56 +67,127 @@ class ClientSimulator:
|
|
59 |
'timestamp': datetime.now()
|
60 |
})
|
61 |
|
62 |
-
# Keep only last 50 entries
|
63 |
if len(st.session_state.training_history) > 50:
|
64 |
st.session_state.training_history = st.session_state.training_history[-50:]
|
|
|
|
|
65 |
|
66 |
-
time.sleep(5)
|
67 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
68 |
except Exception as e:
|
69 |
-
|
|
|
70 |
time.sleep(10)
|
71 |
|
|
|
|
|
|
|
|
|
72 |
except Exception as e:
|
73 |
-
|
|
|
74 |
self.is_running = False
|
75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
76 |
st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
|
77 |
-
st.title("Federated Credit Scoring Demo
|
78 |
|
79 |
# Sidebar configuration
|
80 |
st.sidebar.header("Configuration")
|
81 |
SERVER_URL = st.sidebar.text_input("Server URL", value="http://localhost:8080")
|
82 |
-
DEMO_MODE = st.sidebar.checkbox("Demo Mode
|
83 |
|
84 |
# Initialize session state
|
85 |
if 'client_simulator' not in st.session_state:
|
86 |
st.session_state.client_simulator = None
|
87 |
if 'training_history' not in st.session_state:
|
88 |
st.session_state.training_history = []
|
|
|
|
|
89 |
|
90 |
-
|
91 |
-
|
92 |
-
|
93 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
94 |
|
95 |
-
#
|
96 |
st.sidebar.header("Client Simulator")
|
97 |
-
if st.sidebar.button("Start Client
|
98 |
if not DEMO_MODE:
|
99 |
-
|
100 |
-
|
101 |
-
|
|
|
|
|
|
|
|
|
|
|
102 |
else:
|
103 |
-
st.sidebar.warning("
|
104 |
|
105 |
-
if st.sidebar.button("Stop Client
|
106 |
if st.session_state.client_simulator:
|
107 |
st.session_state.client_simulator.stop()
|
108 |
st.session_state.client_simulator = None
|
109 |
-
st.sidebar.success("Client
|
|
|
110 |
|
111 |
-
#
|
112 |
st.header("Enter Customer Features")
|
113 |
with st.form("feature_form"):
|
114 |
features = []
|
@@ -119,124 +198,103 @@ with st.form("feature_form"):
|
|
119 |
features.append(val)
|
120 |
submitted = st.form_submit_button("Predict Credit Score")
|
121 |
|
122 |
-
#
|
123 |
if submitted:
|
|
|
124 |
if DEMO_MODE:
|
125 |
-
|
126 |
-
|
127 |
-
|
128 |
-
|
129 |
-
|
130 |
-
demo_prediction = sum(features) / len(features) * 100 + 500 # Scale to credit score range
|
131 |
-
st.success(f"Demo Prediction: Credit Score = {demo_prediction:.2f}")
|
132 |
-
st.info("π‘ This is a demo prediction. In a real federated system, this would come from the trained model.")
|
133 |
-
|
134 |
-
# Show what would happen in real mode
|
135 |
-
st.markdown("---")
|
136 |
-
st.markdown("**What happens in real federated learning:**")
|
137 |
-
st.markdown("1. Your features are sent to the federated server")
|
138 |
-
st.markdown("2. Server uses the global model (trained by multiple banks)")
|
139 |
-
st.markdown("3. Prediction is returned without exposing any bank's data")
|
140 |
-
|
141 |
else:
|
142 |
-
# Real mode - connect to server
|
143 |
try:
|
144 |
-
|
|
|
145 |
resp = requests.post(f"{SERVER_URL}/predict", json={"features": features}, timeout=10)
|
146 |
|
147 |
if resp.status_code == 200:
|
148 |
prediction = resp.json().get("prediction")
|
149 |
st.success(f"Predicted Credit Score: {prediction:.2f}")
|
150 |
-
st.
|
|
|
151 |
else:
|
152 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
153 |
except Exception as e:
|
154 |
-
|
155 |
-
st.
|
156 |
-
|
157 |
-
|
158 |
-
st.header("Federated Training Progress")
|
159 |
|
|
|
|
|
160 |
if DEMO_MODE:
|
161 |
-
# Demo training progress
|
162 |
col1, col2, col3, col4 = st.columns(4)
|
163 |
with col1:
|
164 |
-
st.metric("
|
165 |
with col2:
|
166 |
-
st.metric("
|
167 |
with col3:
|
168 |
-
st.metric("
|
169 |
with col4:
|
170 |
-
st.metric("
|
171 |
-
|
172 |
-
st.info("π‘ Demo mode showing simulated training progress. In real federated learning, multiple banks would be training collaboratively.")
|
173 |
-
|
174 |
else:
|
175 |
-
# Real training progress
|
176 |
try:
|
|
|
177 |
status = requests.get(f"{SERVER_URL}/training_status", timeout=5)
|
178 |
if status.status_code == 200:
|
179 |
data = status.json()
|
|
|
180 |
col1, col2, col3, col4 = st.columns(4)
|
181 |
with col1:
|
182 |
-
st.metric("
|
183 |
with col2:
|
184 |
-
st.metric("
|
185 |
with col3:
|
186 |
-
st.metric("
|
187 |
with col4:
|
188 |
-
st.metric("
|
189 |
-
|
190 |
-
# Show training history
|
191 |
-
if st.session_state.training_history:
|
192 |
-
st.subheader("Training History")
|
193 |
-
history_df = st.session_state.training_history
|
194 |
-
st.line_chart(history_df.set_index('round')[['active_clients', 'clients_ready']])
|
195 |
else:
|
196 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
197 |
except Exception as e:
|
198 |
-
|
|
|
|
|
|
|
199 |
|
200 |
-
#
|
201 |
-
if not DEMO_MODE:
|
202 |
-
st.header("Server Health")
|
203 |
-
try:
|
204 |
-
health = requests.get(f"{SERVER_URL}/health", timeout=5)
|
205 |
-
if health.status_code == 200:
|
206 |
-
health_data = health.json()
|
207 |
-
st.success(f"β
Server is healthy")
|
208 |
-
st.json(health_data)
|
209 |
-
else:
|
210 |
-
st.error("β Server health check failed")
|
211 |
-
except Exception as e:
|
212 |
-
st.error(f"β Cannot connect to server: {e}")
|
213 |
-
|
214 |
-
# --- How it works ---
|
215 |
-
st.header("How Federated Learning Works")
|
216 |
-
st.markdown("""
|
217 |
-
**Traditional ML:** All banks send their data to a central server β Privacy risk β
|
218 |
-
|
219 |
-
**Federated Learning:**
|
220 |
-
1. Each bank keeps their data locally β
|
221 |
-
2. Banks train models on their own data β
|
222 |
-
3. Only model updates (not data) are shared β
|
223 |
-
4. Server aggregates updates to create global model β
|
224 |
-
5. Global model is distributed back to all banks β
|
225 |
-
|
226 |
-
**Result:** Collaborative learning without data sharing! π―
|
227 |
-
""")
|
228 |
-
|
229 |
-
# --- Client Simulator Status ---
|
230 |
if st.session_state.client_simulator and not DEMO_MODE:
|
231 |
-
st.header("Client
|
232 |
if st.session_state.client_simulator.is_running:
|
233 |
-
st.success("
|
234 |
-
st.info(f"
|
235 |
-
|
|
|
236 |
else:
|
237 |
-
st.warning("
|
238 |
-
|
239 |
-
st.markdown("---")
|
240 |
-
st.markdown("""
|
241 |
-
*This is a demonstration of federated learning concepts. For full functionality, run the federated server and clients locally.*
|
242 |
-
""")
|
|
|
4 |
import time
|
5 |
import threading
|
6 |
import json
|
7 |
+
import logging
|
8 |
from datetime import datetime
|
9 |
|
10 |
+
# Configure logging
|
11 |
+
logging.basicConfig(level=logging.DEBUG)
|
12 |
+
logger = logging.getLogger(__name__)
|
13 |
+
|
14 |
+
# Client Simulator Class
|
15 |
class ClientSimulator:
|
16 |
def __init__(self, server_url):
|
17 |
self.server_url = server_url
|
|
|
19 |
self.is_running = False
|
20 |
self.thread = None
|
21 |
self.last_update = "Never"
|
22 |
+
self.last_error = None
|
23 |
|
24 |
def start(self):
|
25 |
self.is_running = True
|
26 |
self.thread = threading.Thread(target=self._run_client, daemon=True)
|
27 |
self.thread.start()
|
28 |
+
logger.info(f"Client simulator started for {self.server_url}")
|
29 |
|
30 |
def stop(self):
|
31 |
self.is_running = False
|
32 |
+
logger.info("Client simulator stopped")
|
33 |
|
34 |
def _run_client(self):
|
35 |
try:
|
36 |
+
logger.info(f"Attempting to register client {self.client_id} with server {self.server_url}")
|
37 |
client_info = {
|
38 |
'dataset_size': 100,
|
39 |
'model_params': 10000,
|
|
|
41 |
}
|
42 |
|
43 |
resp = requests.post(f"{self.server_url}/register",
|
44 |
+
json={'client_id': self.client_id, 'client_info': client_info},
|
45 |
+
timeout=10)
|
46 |
|
47 |
if resp.status_code == 200:
|
48 |
+
logger.info(f"Successfully registered client {self.client_id}")
|
49 |
st.session_state.training_history.append({
|
50 |
'round': 0,
|
51 |
'active_clients': 1,
|
|
|
53 |
'timestamp': datetime.now()
|
54 |
})
|
55 |
|
|
|
56 |
while self.is_running:
|
57 |
try:
|
58 |
+
logger.debug(f"Checking training status from {self.server_url}/training_status")
|
59 |
+
status = requests.get(f"{self.server_url}/training_status", timeout=5)
|
60 |
if status.status_code == 200:
|
61 |
data = status.json()
|
62 |
+
logger.debug(f"Training status: {data}")
|
|
|
63 |
st.session_state.training_history.append({
|
64 |
'round': data.get('current_round', 0),
|
65 |
'active_clients': data.get('active_clients', 0),
|
|
|
67 |
'timestamp': datetime.now()
|
68 |
})
|
69 |
|
|
|
70 |
if len(st.session_state.training_history) > 50:
|
71 |
st.session_state.training_history = st.session_state.training_history[-50:]
|
72 |
+
else:
|
73 |
+
logger.warning(f"Training status returned {status.status_code}: {status.text}")
|
74 |
|
75 |
+
time.sleep(5)
|
76 |
|
77 |
+
except requests.exceptions.Timeout:
|
78 |
+
logger.warning("Timeout while checking training status")
|
79 |
+
self.last_error = "Timeout connecting to server"
|
80 |
+
time.sleep(10)
|
81 |
+
except requests.exceptions.ConnectionError as e:
|
82 |
+
logger.error(f"Connection error while checking training status: {e}")
|
83 |
+
self.last_error = f"Connection error: {e}"
|
84 |
+
time.sleep(10)
|
85 |
except Exception as e:
|
86 |
+
logger.error(f"Unexpected error in client simulator: {e}")
|
87 |
+
self.last_error = f"Unexpected error: {e}"
|
88 |
time.sleep(10)
|
89 |
|
90 |
+
except requests.exceptions.ConnectionError as e:
|
91 |
+
logger.error(f"Failed to connect to server {self.server_url}: {e}")
|
92 |
+
self.last_error = f"Failed to connect to server: {e}"
|
93 |
+
self.is_running = False
|
94 |
except Exception as e:
|
95 |
+
logger.error(f"Failed to start client simulator: {e}")
|
96 |
+
self.last_error = f"Failed to start: {e}"
|
97 |
self.is_running = False
|
98 |
|
99 |
+
def check_server_health(server_url):
|
100 |
+
"""Check if server is reachable and healthy"""
|
101 |
+
try:
|
102 |
+
logger.debug(f"Checking server health at {server_url}/health")
|
103 |
+
resp = requests.get(f"{server_url}/health", timeout=5)
|
104 |
+
if resp.status_code == 200:
|
105 |
+
logger.info("Server is healthy")
|
106 |
+
return True, resp.json()
|
107 |
+
else:
|
108 |
+
logger.warning(f"Server health check returned {resp.status_code}")
|
109 |
+
return False, f"HTTP {resp.status_code}: {resp.text}"
|
110 |
+
except requests.exceptions.Timeout:
|
111 |
+
logger.error("Server health check timeout")
|
112 |
+
return False, "Timeout"
|
113 |
+
except requests.exceptions.ConnectionError as e:
|
114 |
+
logger.error(f"Server health check connection error: {e}")
|
115 |
+
return False, f"Connection refused: {e}"
|
116 |
+
except Exception as e:
|
117 |
+
logger.error(f"Server health check unexpected error: {e}")
|
118 |
+
return False, f"Unexpected error: {e}"
|
119 |
+
|
120 |
st.set_page_config(page_title="Federated Credit Scoring Demo", layout="centered")
|
121 |
+
st.title("Federated Credit Scoring Demo")
|
122 |
|
123 |
# Sidebar configuration
|
124 |
st.sidebar.header("Configuration")
|
125 |
SERVER_URL = st.sidebar.text_input("Server URL", value="http://localhost:8080")
|
126 |
+
DEMO_MODE = st.sidebar.checkbox("Demo Mode", value=True)
|
127 |
|
128 |
# Initialize session state
|
129 |
if 'client_simulator' not in st.session_state:
|
130 |
st.session_state.client_simulator = None
|
131 |
if 'training_history' not in st.session_state:
|
132 |
st.session_state.training_history = []
|
133 |
+
if 'debug_messages' not in st.session_state:
|
134 |
+
st.session_state.debug_messages = []
|
135 |
|
136 |
+
# Debug section in sidebar
|
137 |
+
with st.sidebar.expander("Debug Information"):
|
138 |
+
st.write("**Server Status:**")
|
139 |
+
if not DEMO_MODE:
|
140 |
+
is_healthy, health_info = check_server_health(SERVER_URL)
|
141 |
+
if is_healthy:
|
142 |
+
st.success("β
Server is healthy")
|
143 |
+
st.json(health_info)
|
144 |
+
else:
|
145 |
+
st.error(f"β Server error: {health_info}")
|
146 |
+
|
147 |
+
st.write("**Recent Logs:**")
|
148 |
+
if st.session_state.debug_messages:
|
149 |
+
for msg in st.session_state.debug_messages[-5:]: # Show last 5 messages
|
150 |
+
st.text(msg)
|
151 |
+
else:
|
152 |
+
st.text("No debug messages yet")
|
153 |
+
|
154 |
+
if st.button("Clear Debug Logs"):
|
155 |
+
st.session_state.debug_messages = []
|
156 |
+
|
157 |
+
# Sidebar educational content
|
158 |
+
with st.sidebar.expander("About Federated Learning"):
|
159 |
+
st.markdown("""
|
160 |
+
**Traditional ML:** Banks send data to central server β Privacy risk
|
161 |
+
|
162 |
+
**Federated Learning:**
|
163 |
+
- Banks keep data locally
|
164 |
+
- Only model updates are shared
|
165 |
+
- Collaborative learning without data sharing
|
166 |
+
""")
|
167 |
|
168 |
+
# Client Simulator in sidebar
|
169 |
st.sidebar.header("Client Simulator")
|
170 |
+
if st.sidebar.button("Start Client"):
|
171 |
if not DEMO_MODE:
|
172 |
+
try:
|
173 |
+
st.session_state.client_simulator = ClientSimulator(SERVER_URL)
|
174 |
+
st.session_state.client_simulator.start()
|
175 |
+
st.sidebar.success("Client started!")
|
176 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Client simulator started")
|
177 |
+
except Exception as e:
|
178 |
+
st.sidebar.error(f"Failed to start client: {e}")
|
179 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Failed to start client - {e}")
|
180 |
else:
|
181 |
+
st.sidebar.warning("Only works in Real Mode")
|
182 |
|
183 |
+
if st.sidebar.button("Stop Client"):
|
184 |
if st.session_state.client_simulator:
|
185 |
st.session_state.client_simulator.stop()
|
186 |
st.session_state.client_simulator = None
|
187 |
+
st.sidebar.success("Client stopped!")
|
188 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Client simulator stopped")
|
189 |
|
190 |
+
# Main content - focused on core functionality
|
191 |
st.header("Enter Customer Features")
|
192 |
with st.form("feature_form"):
|
193 |
features = []
|
|
|
198 |
features.append(val)
|
199 |
submitted = st.form_submit_button("Predict Credit Score")
|
200 |
|
201 |
+
# Prediction results
|
202 |
if submitted:
|
203 |
+
logger.info(f"Prediction requested with {len(features)} features")
|
204 |
if DEMO_MODE:
|
205 |
+
with st.spinner("Processing..."):
|
206 |
+
time.sleep(1)
|
207 |
+
demo_prediction = sum(features) / len(features) * 100 + 500
|
208 |
+
st.success(f"Predicted Credit Score: {demo_prediction:.2f}")
|
209 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Demo prediction: {demo_prediction:.2f}")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
210 |
else:
|
|
|
211 |
try:
|
212 |
+
logger.info(f"Sending prediction request to {SERVER_URL}/predict")
|
213 |
+
with st.spinner("Connecting to server..."):
|
214 |
resp = requests.post(f"{SERVER_URL}/predict", json={"features": features}, timeout=10)
|
215 |
|
216 |
if resp.status_code == 200:
|
217 |
prediction = resp.json().get("prediction")
|
218 |
st.success(f"Predicted Credit Score: {prediction:.2f}")
|
219 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: Real prediction: {prediction:.2f}")
|
220 |
+
logger.info(f"Prediction successful: {prediction}")
|
221 |
else:
|
222 |
+
error_msg = f"Prediction failed: {resp.json().get('error', 'Unknown error')}"
|
223 |
+
st.error(error_msg)
|
224 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
225 |
+
logger.error(f"Prediction failed with status {resp.status_code}: {resp.text}")
|
226 |
+
except requests.exceptions.Timeout:
|
227 |
+
error_msg = "Timeout connecting to server"
|
228 |
+
st.error(error_msg)
|
229 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
230 |
+
logger.error("Prediction request timeout")
|
231 |
+
except requests.exceptions.ConnectionError as e:
|
232 |
+
error_msg = f"Connection error: {e}"
|
233 |
+
st.error(error_msg)
|
234 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
235 |
+
logger.error(f"Prediction connection error: {e}")
|
236 |
except Exception as e:
|
237 |
+
error_msg = f"Unexpected error: {e}"
|
238 |
+
st.error(error_msg)
|
239 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
240 |
+
logger.error(f"Prediction unexpected error: {e}")
|
|
|
241 |
|
242 |
+
# Training progress - simplified
|
243 |
+
st.header("Training Progress")
|
244 |
if DEMO_MODE:
|
|
|
245 |
col1, col2, col3, col4 = st.columns(4)
|
246 |
with col1:
|
247 |
+
st.metric("Round", "3/10")
|
248 |
with col2:
|
249 |
+
st.metric("Clients", "3")
|
250 |
with col3:
|
251 |
+
st.metric("Accuracy", "85.2%")
|
252 |
with col4:
|
253 |
+
st.metric("Status", "Active")
|
|
|
|
|
|
|
254 |
else:
|
|
|
255 |
try:
|
256 |
+
logger.debug(f"Fetching training status from {SERVER_URL}/training_status")
|
257 |
status = requests.get(f"{SERVER_URL}/training_status", timeout=5)
|
258 |
if status.status_code == 200:
|
259 |
data = status.json()
|
260 |
+
logger.debug(f"Training status received: {data}")
|
261 |
col1, col2, col3, col4 = st.columns(4)
|
262 |
with col1:
|
263 |
+
st.metric("Round", f"{data.get('current_round', 0)}/{data.get('total_rounds', 10)}")
|
264 |
with col2:
|
265 |
+
st.metric("Clients", data.get('active_clients', 0))
|
266 |
with col3:
|
267 |
+
st.metric("Ready", data.get('clients_ready', 0))
|
268 |
with col4:
|
269 |
+
st.metric("Status", "Active" if data.get('training_active', False) else "Inactive")
|
|
|
|
|
|
|
|
|
|
|
|
|
270 |
else:
|
271 |
+
error_msg = f"Training status failed: HTTP {status.status_code}"
|
272 |
+
st.warning(error_msg)
|
273 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
274 |
+
logger.warning(f"Training status returned {status.status_code}: {status.text}")
|
275 |
+
except requests.exceptions.Timeout:
|
276 |
+
error_msg = "Training status timeout"
|
277 |
+
st.warning(error_msg)
|
278 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
279 |
+
logger.warning("Training status request timeout")
|
280 |
+
except requests.exceptions.ConnectionError as e:
|
281 |
+
error_msg = f"Training status connection error: {e}"
|
282 |
+
st.warning(error_msg)
|
283 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
284 |
+
logger.error(f"Training status connection error: {e}")
|
285 |
except Exception as e:
|
286 |
+
error_msg = f"Training status unexpected error: {e}"
|
287 |
+
st.warning(error_msg)
|
288 |
+
st.session_state.debug_messages.append(f"{datetime.now()}: {error_msg}")
|
289 |
+
logger.error(f"Training status unexpected error: {e}")
|
290 |
|
291 |
+
# Client status in sidebar
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
292 |
if st.session_state.client_simulator and not DEMO_MODE:
|
293 |
+
st.sidebar.header("Client Status")
|
294 |
if st.session_state.client_simulator.is_running:
|
295 |
+
st.sidebar.success("Connected")
|
296 |
+
st.sidebar.info(f"ID: {st.session_state.client_simulator.client_id}")
|
297 |
+
if st.session_state.client_simulator.last_error:
|
298 |
+
st.sidebar.error(f"Last Error: {st.session_state.client_simulator.last_error}")
|
299 |
else:
|
300 |
+
st.sidebar.warning("Disconnected")
|
|
|
|
|
|
|
|
|
|