Kye Gomez commited on
Commit
27a55e7
·
1 Parent(s): 8810a86
Files changed (2) hide show
  1. DOCS.md +421 -0
  2. README.md +5 -453
DOCS.md ADDED
@@ -0,0 +1,421 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Docs
2
+
3
+
4
+ ## ✨ Features
5
+
6
+ ### 🏥 Virtual Physician Panel
7
+ - **8 Specialized AI Agents**: Each with distinct medical expertise and decision-making roles
8
+ - **Iterative Deliberation**: Sequential consultation and consensus-building process
9
+ - **Bayesian Reasoning**: Probability-based differential diagnosis updates
10
+ - **Cognitive Bias Detection**: Built-in challenger agent to prevent diagnostic errors
11
+
12
+ ### 💰 Cost-Effectiveness Optimization
13
+ - **Comprehensive Cost Tracking**: Real-time budget monitoring with 25+ medical test costs
14
+ - **Resource Stewardship**: AI agent dedicated to cost-conscious care decisions
15
+ - **Budget Constraints**: Configurable spending limits with intelligent test prioritization
16
+ - **Value-Based Testing**: Information theory-driven test selection
17
+
18
+ ### 🎯 Multiple Operational Modes
19
+ - **Instant**: Immediate diagnosis from initial presentation
20
+ - **Question-Only**: History-taking without diagnostic tests
21
+ - **Budgeted**: Cost-constrained diagnostic workup
22
+ - **No-Budget**: Full diagnostic capability
23
+ - **Ensemble**: Multiple independent panels with consensus aggregation
24
+
25
+ ### 📊 Advanced Evaluation
26
+ - **Clinical Accuracy Scoring**: 5-point Likert scale with detailed rubric
27
+ - **Management Impact Assessment**: Evaluation based on treatment implications
28
+ - **Diagnostic Reasoning Tracking**: Complete conversation history and decision trails
29
+ - **Ensemble Methods**: Multi-run consensus for improved accuracy
30
+
31
+ ### 🔧 Technical Excellence
32
+ - **Model Agnostic**: Support for GPT, Gemini, Claude, and other LLMs
33
+ - **Robust Error Handling**: Comprehensive exception management and fallback mechanisms
34
+ - **Beautiful Logging**: Structured logging with Loguru for debugging and monitoring
35
+ - **Type Safety**: Full Pydantic models and type hints throughout
36
+
37
+ ## 🏗 Architecture
38
+
39
+ ### Virtual Panel Composition
40
+
41
+ The MAI-DxO system consists of 8 specialized AI agents that work together to provide comprehensive medical diagnosis:
42
+
43
+ #### Core Diagnostic Panel
44
+
45
+ **🧠 Dr. Hypothesis**
46
+ - Maintains probability-ranked differential diagnosis (top 3 conditions)
47
+ - Updates probabilities using Bayesian reasoning after each finding
48
+ - Tracks evidence supporting and contradicting each hypothesis
49
+
50
+ **🔬 Dr. Test-Chooser**
51
+ - Selects up to 3 diagnostic tests per round for maximum information value
52
+ - Optimizes for discriminatory power between competing hypotheses
53
+ - Balances diagnostic yield with patient burden
54
+
55
+ **🤔 Dr. Challenger**
56
+ - Acts as devil's advocate to prevent cognitive biases
57
+ - Identifies contradictory evidence and alternative explanations
58
+ - Proposes falsifying tests and guards against premature closure
59
+
60
+ **💰 Dr. Stewardship**
61
+ - Enforces cost-conscious, high-value care decisions
62
+ - Advocates for cheaper alternatives when diagnostically equivalent
63
+ - Evaluates test necessity and suggests cost-effective strategies
64
+
65
+ **✅ Dr. Checklist**
66
+ - Performs quality control on panel deliberations
67
+ - Validates test names and maintains logical consistency
68
+ - Flags errors and ensures proper diagnostic methodology
69
+
70
+ #### Coordination and Evaluation
71
+
72
+ **🤝 Consensus Coordinator**
73
+ - Synthesizes panel input into optimal next action
74
+ - Decides between asking questions, ordering tests, or diagnosing
75
+ - Balances accuracy, cost, efficiency, and thoroughness
76
+
77
+ **🔑 Gatekeeper**
78
+ - Serves as clinical information oracle with complete case access
79
+ - Provides objective findings and realistic synthetic results
80
+ - Maintains clinical realism while preventing information leakage
81
+
82
+ **⚖️ Judge**
83
+ - Evaluates final diagnoses against ground truth
84
+ - Uses rigorous 5-point clinical rubric
85
+ - Considers management implications and diagnostic completeness
86
+
87
+ ### Decision Process Flow
88
+
89
+ ```mermaid
90
+ graph TD
91
+ A[Initial Case Information] --> B[Panel Deliberation]
92
+ B --> C{Consensus Decision}
93
+ C -->|Ask| D[Question to Gatekeeper]
94
+ C -->|Test| E[Diagnostic Tests]
95
+ C -->|Diagnose| F[Final Diagnosis]
96
+ D --> G[Update Case Information]
97
+ E --> G
98
+ G --> H{Max Iterations or Budget?}
99
+ H -->|No| B
100
+ H -->|Yes| F
101
+ F --> I[Judge Evaluation]
102
+ I --> J[Diagnosis Result]
103
+ ```
104
+
105
+ ## 🎮 Usage
106
+
107
+ ### Basic Usage
108
+
109
+ ```python
110
+ from mai_dx import MaiDxOrchestrator
111
+
112
+ # Initialize orchestrator
113
+ orchestrator = MaiDxOrchestrator(
114
+ model_name="gemini/gemini-2.5-flash",
115
+ max_iterations=10,
116
+ initial_budget=10000
117
+ )
118
+
119
+ # Define case information
120
+ initial_info = "A 45-year-old male presents with chest pain..."
121
+ full_case = "Patient: 45-year-old male. History: Acute onset chest pain..."
122
+ ground_truth = "Myocardial infarction"
123
+
124
+ # Run diagnosis
125
+ result = orchestrator.run(initial_info, full_case, ground_truth)
126
+
127
+ # Access results
128
+ print(f"Diagnosis: {result.final_diagnosis}")
129
+ print(f"Accuracy Score: {result.accuracy_score}/5.0")
130
+ print(f"Total Cost: ${result.total_cost:,}")
131
+ print(f"Iterations: {result.iterations}")
132
+ ```
133
+
134
+ ### Advanced Configuration
135
+
136
+ ```python
137
+ # Custom orchestrator with specific settings
138
+ orchestrator = MaiDxOrchestrator(
139
+ model_name="gpt-4",
140
+ max_iterations=15,
141
+ initial_budget=5000,
142
+ mode="budgeted",
143
+ physician_visit_cost=250,
144
+ enable_budget_tracking=True
145
+ )
146
+
147
+ # Enable debug logging
148
+ import os
149
+ os.environ["MAIDX_DEBUG"] = "1"
150
+ ```
151
+
152
+ ## 📋 MAI-DxO Variants
153
+
154
+ The system supports five distinct operational variants, each optimized for different clinical scenarios:
155
+
156
+ ### 1. Instant Answer
157
+ ```python
158
+ orchestrator = MaiDxOrchestrator.create_variant("instant")
159
+ result = orchestrator.run(initial_info, full_case, ground_truth)
160
+ ```
161
+ - **Use Case**: Emergency triage, rapid screening
162
+ - **Behavior**: Immediate diagnosis from initial presentation only
163
+ - **Cost**: Single physician visit ($300)
164
+
165
+ ### 2. Question-Only
166
+ ```python
167
+ orchestrator = MaiDxOrchestrator.create_variant("question_only")
168
+ result = orchestrator.run(initial_info, full_case, ground_truth)
169
+ ```
170
+ - **Use Case**: Telemedicine, history-taking focused consultations
171
+ - **Behavior**: Detailed questioning without diagnostic tests
172
+ - **Cost**: Physician visit only
173
+
174
+ ### 3. Budgeted
175
+ ```python
176
+ orchestrator = MaiDxOrchestrator.create_variant("budgeted", budget=3000)
177
+ result = orchestrator.run(initial_info, full_case, ground_truth)
178
+ ```
179
+ - **Use Case**: Resource-constrained settings, cost-conscious care
180
+ - **Behavior**: Full panel with strict budget enforcement
181
+ - **Cost**: Limited by specified budget
182
+
183
+ ### 4. No-Budget
184
+ ```python
185
+ orchestrator = MaiDxOrchestrator.create_variant("no_budget")
186
+ result = orchestrator.run(initial_info, full_case, ground_truth)
187
+ ```
188
+ - **Use Case**: Academic medical centers, complex cases
189
+ - **Behavior**: Full diagnostic capability without cost constraints
190
+ - **Cost**: Unlimited (tracks for analysis)
191
+
192
+ ### 5. Ensemble
193
+ ```python
194
+ orchestrator = MaiDxOrchestrator.create_variant("ensemble")
195
+ result = orchestrator.run_ensemble(initial_info, full_case, ground_truth, num_runs=3)
196
+ ```
197
+ - **Use Case**: Critical diagnoses, second opinion simulation
198
+ - **Behavior**: Multiple independent panels with consensus aggregation
199
+ - **Cost**: Sum of all panel costs
200
+
201
+ ## ⚙️ Configuration
202
+
203
+ ### Model Configuration
204
+
205
+ ```python
206
+ # Supported models
207
+ models = [
208
+ "gemini/gemini-2.5-flash",
209
+ "gpt-4o",
210
+ "gpt-4o-mini",
211
+ "claude-3-5-sonnet-20241022",
212
+ "meta-llama/llama-3.1-8b-instruct"
213
+ ]
214
+
215
+ orchestrator = MaiDxOrchestrator(model_name="gpt-4o")
216
+ ```
217
+
218
+ ### Cost Database Customization
219
+
220
+ ```python
221
+ # Access and modify cost database
222
+ orchestrator = MaiDxOrchestrator()
223
+ orchestrator.test_cost_db.update({
224
+ "custom_test": 450,
225
+ "specialized_imaging": 2000
226
+ })
227
+ ```
228
+
229
+ ### Logging Configuration
230
+
231
+ ```python
232
+ # Enable detailed debug logging
233
+ import os
234
+ os.environ["MAIDX_DEBUG"] = "1"
235
+
236
+ # Custom log levels and formats available
237
+ ```
238
+
239
+ ## 📖 Examples
240
+
241
+ ### Example 1: Comprehensive Diagnostic Workup
242
+
243
+ ```python
244
+ from mai_dx import MaiDxOrchestrator
245
+
246
+ # Complex case requiring multiple tests
247
+ case_info = """
248
+ A 29-year-old woman was admitted to the hospital because of sore throat
249
+ and peritonsillar swelling and bleeding. Symptoms did not abate with
250
+ antimicrobial therapy.
251
+ """
252
+
253
+ case_details = """
254
+ Patient: 29-year-old female.
255
+ History: Onset of sore throat 7 weeks prior to admission. Worsening
256
+ right-sided pain and swelling. No fevers, headaches, or GI symptoms.
257
+ Physical Exam: Right peritonsillar mass, displacing the uvula.
258
+ Initial Labs: FBC, clotting studies normal.
259
+ """
260
+
261
+ ground_truth = "Embryonal rhabdomyosarcoma of the pharynx"
262
+
263
+ # Run with different variants
264
+ variants = ["question_only", "budgeted", "no_budget"]
265
+ results = {}
266
+
267
+ for variant in variants:
268
+ if variant == "budgeted":
269
+ orch = MaiDxOrchestrator.create_variant(variant, budget=3000)
270
+ else:
271
+ orch = MaiDxOrchestrator.create_variant(variant)
272
+
273
+ results[variant] = orch.run(case_info, case_details, ground_truth)
274
+
275
+ # Compare results
276
+ for variant, result in results.items():
277
+ print(f"{variant}: {result.final_diagnosis} (Score: {result.accuracy_score})")
278
+ ```
279
+
280
+ ### Example 2: Ensemble Diagnosis
281
+
282
+ ```python
283
+ # High-stakes diagnosis with ensemble approach
284
+ ensemble_orchestrator = MaiDxOrchestrator.create_variant("ensemble")
285
+
286
+ ensemble_result = ensemble_orchestrator.run_ensemble(
287
+ initial_case_info=case_info,
288
+ full_case_details=case_details,
289
+ ground_truth_diagnosis=ground_truth,
290
+ num_runs=5 # 5 independent diagnostic panels
291
+ )
292
+
293
+ print(f"Ensemble Diagnosis: {ensemble_result.final_diagnosis}")
294
+ print(f"Confidence Score: {ensemble_result.accuracy_score}/5.0")
295
+ print(f"Total Cost: ${ensemble_result.total_cost:,}")
296
+ ```
297
+
298
+ ### Example 3: Custom Cost Analysis
299
+
300
+ ```python
301
+ # Analyze cost-effectiveness across variants
302
+ import matplotlib.pyplot as plt
303
+
304
+ variants = ["instant", "question_only", "budgeted", "no_budget"]
305
+ costs = []
306
+ accuracies = []
307
+
308
+ for variant in variants:
309
+ orch = MaiDxOrchestrator.create_variant(variant)
310
+ result = orch.run(case_info, case_details, ground_truth)
311
+ costs.append(result.total_cost)
312
+ accuracies.append(result.accuracy_score)
313
+
314
+ # Plot cost vs accuracy
315
+ plt.scatter(costs, accuracies)
316
+ plt.xlabel('Total Cost ($)')
317
+ plt.ylabel('Accuracy Score')
318
+ plt.title('Cost vs Accuracy Trade-off')
319
+ for i, variant in enumerate(variants):
320
+ plt.annotate(variant, (costs[i], accuracies[i]))
321
+ plt.show()
322
+ ```
323
+
324
+ ## 🔍 API Reference
325
+
326
+ ### MaiDxOrchestrator Class
327
+
328
+ #### Constructor
329
+ ```python
330
+ MaiDxOrchestrator(
331
+ model_name: str = "gemini/gemini-2.5-flash",
332
+ max_iterations: int = 10,
333
+ initial_budget: int = 10000,
334
+ mode: str = "no_budget",
335
+ physician_visit_cost: int = 300,
336
+ enable_budget_tracking: bool = False
337
+ )
338
+ ```
339
+
340
+ #### Methods
341
+
342
+ **`run(initial_case_info, full_case_details, ground_truth_diagnosis)`**
343
+ - Executes the sequential diagnostic process
344
+ - Returns: `DiagnosisResult` object
345
+
346
+ **`run_ensemble(initial_case_info, full_case_details, ground_truth_diagnosis, num_runs=3)`**
347
+ - Runs multiple independent sessions with consensus aggregation
348
+ - Returns: `DiagnosisResult` object
349
+
350
+ **`create_variant(variant, **kwargs)` (Class Method)**
351
+ - Factory method for creating specialized variants
352
+ - Variants: "instant", "question_only", "budgeted", "no_budget", "ensemble"
353
+
354
+ ### DiagnosisResult Class
355
+
356
+ ```python
357
+ @dataclass
358
+ class DiagnosisResult:
359
+ final_diagnosis: str
360
+ ground_truth: str
361
+ accuracy_score: float
362
+ accuracy_reasoning: str
363
+ total_cost: int
364
+ iterations: int
365
+ conversation_history: str
366
+ ```
367
+
368
+ ### Utility Functions
369
+
370
+ **`run_mai_dxo_demo(case_info=None, case_details=None, ground_truth=None)`**
371
+ - Convenience function for quick demonstrations
372
+ - Returns: Dictionary of results from multiple variants
373
+
374
+ ## 🧪 Testing and Validation
375
+
376
+ ### Running Tests
377
+ ```bash
378
+ # Run the built-in demo
379
+ python -m mai_dx.main
380
+
381
+ # Run with custom cases
382
+ python -c "
383
+ from mai_dx import run_mai_dxo_demo
384
+ results = run_mai_dxo_demo()
385
+ print(results)
386
+ "
387
+ ```
388
+
389
+ ### Benchmarking
390
+ ```python
391
+ import time
392
+ from mai_dx import MaiDxOrchestrator
393
+
394
+ # Performance benchmarking
395
+ start_time = time.time()
396
+ orchestrator = MaiDxOrchestrator()
397
+ result = orchestrator.run(case_info, case_details, ground_truth)
398
+ elapsed = time.time() - start_time
399
+
400
+ print(f"Diagnosis completed in {elapsed:.2f} seconds")
401
+ print(f"Accuracy: {result.accuracy_score}/5.0")
402
+ print(f"Cost efficiency: ${result.total_cost/result.accuracy_score:.0f} per accuracy point")
403
+ ```
404
+
405
+ ## 🤝 Contributing
406
+
407
+ We welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details.
408
+
409
+ ### Development Setup
410
+ ```bash
411
+ git clone https://github.com/your-org/Open-MAI-Dx-Orchestrator.git
412
+ cd Open-MAI-Dx-Orchestrator
413
+ pip install -e ".[dev]"
414
+ pre-commit install
415
+ ```
416
+
417
+ ### Code Style
418
+ - Follow PEP 8 guidelines
419
+ - Use type hints throughout
420
+ - Maintain comprehensive docstrings
421
+ - Add tests for new features
README.md CHANGED
@@ -34,6 +34,8 @@ ANTHROPIC_API_KEY=""
34
 
35
  ----
36
 
 
 
37
  ```python
38
  from mai_dx import MaiDxOrchestrator
39
 
@@ -52,461 +54,11 @@ print(f"Accuracy: {result.accuracy_score}/5.0")
52
  print(f"Cost: ${result.total_cost:,}")
53
  ```
54
 
55
- ## 📚 Table of Contents
56
-
57
- - [Features](#-features)
58
- - [Installation](#-installation)
59
- - [Architecture](#-architecture)
60
- - [Usage](#-usage)
61
- - [MAI-DxO Variants](#-mai-dxo-variants)
62
- - [Configuration](#-configuration)
63
- - [Examples](#-examples)
64
- - [API Reference](#-api-reference)
65
- - [Contributing](#-contributing)
66
- - [Citation](#-citation)
67
-
68
- ## ✨ Features
69
-
70
- ### 🏥 Virtual Physician Panel
71
- - **8 Specialized AI Agents**: Each with distinct medical expertise and decision-making roles
72
- - **Iterative Deliberation**: Sequential consultation and consensus-building process
73
- - **Bayesian Reasoning**: Probability-based differential diagnosis updates
74
- - **Cognitive Bias Detection**: Built-in challenger agent to prevent diagnostic errors
75
-
76
- ### 💰 Cost-Effectiveness Optimization
77
- - **Comprehensive Cost Tracking**: Real-time budget monitoring with 25+ medical test costs
78
- - **Resource Stewardship**: AI agent dedicated to cost-conscious care decisions
79
- - **Budget Constraints**: Configurable spending limits with intelligent test prioritization
80
- - **Value-Based Testing**: Information theory-driven test selection
81
-
82
- ### 🎯 Multiple Operational Modes
83
- - **Instant**: Immediate diagnosis from initial presentation
84
- - **Question-Only**: History-taking without diagnostic tests
85
- - **Budgeted**: Cost-constrained diagnostic workup
86
- - **No-Budget**: Full diagnostic capability
87
- - **Ensemble**: Multiple independent panels with consensus aggregation
88
-
89
- ### 📊 Advanced Evaluation
90
- - **Clinical Accuracy Scoring**: 5-point Likert scale with detailed rubric
91
- - **Management Impact Assessment**: Evaluation based on treatment implications
92
- - **Diagnostic Reasoning Tracking**: Complete conversation history and decision trails
93
- - **Ensemble Methods**: Multi-run consensus for improved accuracy
94
-
95
- ### 🔧 Technical Excellence
96
- - **Model Agnostic**: Support for GPT, Gemini, Claude, and other LLMs
97
- - **Robust Error Handling**: Comprehensive exception management and fallback mechanisms
98
- - **Beautiful Logging**: Structured logging with Loguru for debugging and monitoring
99
- - **Type Safety**: Full Pydantic models and type hints throughout
100
-
101
- ## 🛠 Installation
102
-
103
- ### Prerequisites
104
- - Python 3.8 or higher
105
- - API keys for your chosen language model provider
106
-
107
- ### Standard Installation
108
- ```bash
109
- pip install mai-dx
110
- ```
111
-
112
- ### Development Installation
113
- ```bash
114
- git clone https://github.com/The-Swarm-Corporation/Open-MAI-Dx-Orchestrator.git
115
- cd Open-MAI-Dx-Orchestrator
116
- pip install -e .
117
- ```
118
-
119
- ### Dependencies
120
- The package automatically installs:
121
- - `swarms` - AI agent orchestration framework
122
- - `loguru` - Advanced logging
123
- - `pydantic` - Data validation and serialization
124
-
125
- ## 🏗 Architecture
126
-
127
- ### Virtual Panel Composition
128
-
129
- The MAI-DxO system consists of 8 specialized AI agents that work together to provide comprehensive medical diagnosis:
130
-
131
- #### Core Diagnostic Panel
132
-
133
- **🧠 Dr. Hypothesis**
134
- - Maintains probability-ranked differential diagnosis (top 3 conditions)
135
- - Updates probabilities using Bayesian reasoning after each finding
136
- - Tracks evidence supporting and contradicting each hypothesis
137
-
138
- **🔬 Dr. Test-Chooser**
139
- - Selects up to 3 diagnostic tests per round for maximum information value
140
- - Optimizes for discriminatory power between competing hypotheses
141
- - Balances diagnostic yield with patient burden
142
-
143
- **🤔 Dr. Challenger**
144
- - Acts as devil's advocate to prevent cognitive biases
145
- - Identifies contradictory evidence and alternative explanations
146
- - Proposes falsifying tests and guards against premature closure
147
-
148
- **💰 Dr. Stewardship**
149
- - Enforces cost-conscious, high-value care decisions
150
- - Advocates for cheaper alternatives when diagnostically equivalent
151
- - Evaluates test necessity and suggests cost-effective strategies
152
-
153
- **✅ Dr. Checklist**
154
- - Performs quality control on panel deliberations
155
- - Validates test names and maintains logical consistency
156
- - Flags errors and ensures proper diagnostic methodology
157
-
158
- #### Coordination and Evaluation
159
-
160
- **🤝 Consensus Coordinator**
161
- - Synthesizes panel input into optimal next action
162
- - Decides between asking questions, ordering tests, or diagnosing
163
- - Balances accuracy, cost, efficiency, and thoroughness
164
-
165
- **🔑 Gatekeeper**
166
- - Serves as clinical information oracle with complete case access
167
- - Provides objective findings and realistic synthetic results
168
- - Maintains clinical realism while preventing information leakage
169
-
170
- **⚖️ Judge**
171
- - Evaluates final diagnoses against ground truth
172
- - Uses rigorous 5-point clinical rubric
173
- - Considers management implications and diagnostic completeness
174
-
175
- ### Decision Process Flow
176
-
177
- ```mermaid
178
- graph TD
179
- A[Initial Case Information] --> B[Panel Deliberation]
180
- B --> C{Consensus Decision}
181
- C -->|Ask| D[Question to Gatekeeper]
182
- C -->|Test| E[Diagnostic Tests]
183
- C -->|Diagnose| F[Final Diagnosis]
184
- D --> G[Update Case Information]
185
- E --> G
186
- G --> H{Max Iterations or Budget?}
187
- H -->|No| B
188
- H -->|Yes| F
189
- F --> I[Judge Evaluation]
190
- I --> J[Diagnosis Result]
191
- ```
192
-
193
- ## 🎮 Usage
194
-
195
- ### Basic Usage
196
-
197
- ```python
198
- from mai_dx import MaiDxOrchestrator
199
-
200
- # Initialize orchestrator
201
- orchestrator = MaiDxOrchestrator(
202
- model_name="gemini/gemini-2.5-flash",
203
- max_iterations=10,
204
- initial_budget=10000
205
- )
206
-
207
- # Define case information
208
- initial_info = "A 45-year-old male presents with chest pain..."
209
- full_case = "Patient: 45-year-old male. History: Acute onset chest pain..."
210
- ground_truth = "Myocardial infarction"
211
-
212
- # Run diagnosis
213
- result = orchestrator.run(initial_info, full_case, ground_truth)
214
-
215
- # Access results
216
- print(f"Diagnosis: {result.final_diagnosis}")
217
- print(f"Accuracy Score: {result.accuracy_score}/5.0")
218
- print(f"Total Cost: ${result.total_cost:,}")
219
- print(f"Iterations: {result.iterations}")
220
- ```
221
-
222
- ### Advanced Configuration
223
-
224
- ```python
225
- # Custom orchestrator with specific settings
226
- orchestrator = MaiDxOrchestrator(
227
- model_name="gpt-4",
228
- max_iterations=15,
229
- initial_budget=5000,
230
- mode="budgeted",
231
- physician_visit_cost=250,
232
- enable_budget_tracking=True
233
- )
234
-
235
- # Enable debug logging
236
- import os
237
- os.environ["MAIDX_DEBUG"] = "1"
238
- ```
239
-
240
- ## 📋 MAI-DxO Variants
241
-
242
- The system supports five distinct operational variants, each optimized for different clinical scenarios:
243
-
244
- ### 1. Instant Answer
245
- ```python
246
- orchestrator = MaiDxOrchestrator.create_variant("instant")
247
- result = orchestrator.run(initial_info, full_case, ground_truth)
248
- ```
249
- - **Use Case**: Emergency triage, rapid screening
250
- - **Behavior**: Immediate diagnosis from initial presentation only
251
- - **Cost**: Single physician visit ($300)
252
-
253
- ### 2. Question-Only
254
- ```python
255
- orchestrator = MaiDxOrchestrator.create_variant("question_only")
256
- result = orchestrator.run(initial_info, full_case, ground_truth)
257
- ```
258
- - **Use Case**: Telemedicine, history-taking focused consultations
259
- - **Behavior**: Detailed questioning without diagnostic tests
260
- - **Cost**: Physician visit only
261
-
262
- ### 3. Budgeted
263
- ```python
264
- orchestrator = MaiDxOrchestrator.create_variant("budgeted", budget=3000)
265
- result = orchestrator.run(initial_info, full_case, ground_truth)
266
- ```
267
- - **Use Case**: Resource-constrained settings, cost-conscious care
268
- - **Behavior**: Full panel with strict budget enforcement
269
- - **Cost**: Limited by specified budget
270
-
271
- ### 4. No-Budget
272
- ```python
273
- orchestrator = MaiDxOrchestrator.create_variant("no_budget")
274
- result = orchestrator.run(initial_info, full_case, ground_truth)
275
- ```
276
- - **Use Case**: Academic medical centers, complex cases
277
- - **Behavior**: Full diagnostic capability without cost constraints
278
- - **Cost**: Unlimited (tracks for analysis)
279
-
280
- ### 5. Ensemble
281
- ```python
282
- orchestrator = MaiDxOrchestrator.create_variant("ensemble")
283
- result = orchestrator.run_ensemble(initial_info, full_case, ground_truth, num_runs=3)
284
- ```
285
- - **Use Case**: Critical diagnoses, second opinion simulation
286
- - **Behavior**: Multiple independent panels with consensus aggregation
287
- - **Cost**: Sum of all panel costs
288
-
289
- ## ⚙️ Configuration
290
 
291
- ### Model Configuration
292
-
293
- ```python
294
- # Supported models
295
- models = [
296
- "gemini/gemini-2.5-flash",
297
- "gpt-4o",
298
- "gpt-4o-mini",
299
- "claude-3-5-sonnet-20241022",
300
- "meta-llama/llama-3.1-8b-instruct"
301
- ]
302
-
303
- orchestrator = MaiDxOrchestrator(model_name="gpt-4o")
304
- ```
305
-
306
- ### Cost Database Customization
307
-
308
- ```python
309
- # Access and modify cost database
310
- orchestrator = MaiDxOrchestrator()
311
- orchestrator.test_cost_db.update({
312
- "custom_test": 450,
313
- "specialized_imaging": 2000
314
- })
315
- ```
316
-
317
- ### Logging Configuration
318
-
319
- ```python
320
- # Enable detailed debug logging
321
- import os
322
- os.environ["MAIDX_DEBUG"] = "1"
323
-
324
- # Custom log levels and formats available
325
- ```
326
-
327
- ## 📖 Examples
328
-
329
- ### Example 1: Comprehensive Diagnostic Workup
330
-
331
- ```python
332
- from mai_dx import MaiDxOrchestrator
333
-
334
- # Complex case requiring multiple tests
335
- case_info = """
336
- A 29-year-old woman was admitted to the hospital because of sore throat
337
- and peritonsillar swelling and bleeding. Symptoms did not abate with
338
- antimicrobial therapy.
339
- """
340
-
341
- case_details = """
342
- Patient: 29-year-old female.
343
- History: Onset of sore throat 7 weeks prior to admission. Worsening
344
- right-sided pain and swelling. No fevers, headaches, or GI symptoms.
345
- Physical Exam: Right peritonsillar mass, displacing the uvula.
346
- Initial Labs: FBC, clotting studies normal.
347
- """
348
-
349
- ground_truth = "Embryonal rhabdomyosarcoma of the pharynx"
350
-
351
- # Run with different variants
352
- variants = ["question_only", "budgeted", "no_budget"]
353
- results = {}
354
-
355
- for variant in variants:
356
- if variant == "budgeted":
357
- orch = MaiDxOrchestrator.create_variant(variant, budget=3000)
358
- else:
359
- orch = MaiDxOrchestrator.create_variant(variant)
360
-
361
- results[variant] = orch.run(case_info, case_details, ground_truth)
362
-
363
- # Compare results
364
- for variant, result in results.items():
365
- print(f"{variant}: {result.final_diagnosis} (Score: {result.accuracy_score})")
366
- ```
367
-
368
- ### Example 2: Ensemble Diagnosis
369
-
370
- ```python
371
- # High-stakes diagnosis with ensemble approach
372
- ensemble_orchestrator = MaiDxOrchestrator.create_variant("ensemble")
373
-
374
- ensemble_result = ensemble_orchestrator.run_ensemble(
375
- initial_case_info=case_info,
376
- full_case_details=case_details,
377
- ground_truth_diagnosis=ground_truth,
378
- num_runs=5 # 5 independent diagnostic panels
379
- )
380
-
381
- print(f"Ensemble Diagnosis: {ensemble_result.final_diagnosis}")
382
- print(f"Confidence Score: {ensemble_result.accuracy_score}/5.0")
383
- print(f"Total Cost: ${ensemble_result.total_cost:,}")
384
- ```
385
-
386
- ### Example 3: Custom Cost Analysis
387
-
388
- ```python
389
- # Analyze cost-effectiveness across variants
390
- import matplotlib.pyplot as plt
391
-
392
- variants = ["instant", "question_only", "budgeted", "no_budget"]
393
- costs = []
394
- accuracies = []
395
-
396
- for variant in variants:
397
- orch = MaiDxOrchestrator.create_variant(variant)
398
- result = orch.run(case_info, case_details, ground_truth)
399
- costs.append(result.total_cost)
400
- accuracies.append(result.accuracy_score)
401
-
402
- # Plot cost vs accuracy
403
- plt.scatter(costs, accuracies)
404
- plt.xlabel('Total Cost ($)')
405
- plt.ylabel('Accuracy Score')
406
- plt.title('Cost vs Accuracy Trade-off')
407
- for i, variant in enumerate(variants):
408
- plt.annotate(variant, (costs[i], accuracies[i]))
409
- plt.show()
410
- ```
411
-
412
- ## 🔍 API Reference
413
-
414
- ### MaiDxOrchestrator Class
415
-
416
- #### Constructor
417
- ```python
418
- MaiDxOrchestrator(
419
- model_name: str = "gemini/gemini-2.5-flash",
420
- max_iterations: int = 10,
421
- initial_budget: int = 10000,
422
- mode: str = "no_budget",
423
- physician_visit_cost: int = 300,
424
- enable_budget_tracking: bool = False
425
- )
426
- ```
427
-
428
- #### Methods
429
-
430
- **`run(initial_case_info, full_case_details, ground_truth_diagnosis)`**
431
- - Executes the sequential diagnostic process
432
- - Returns: `DiagnosisResult` object
433
-
434
- **`run_ensemble(initial_case_info, full_case_details, ground_truth_diagnosis, num_runs=3)`**
435
- - Runs multiple independent sessions with consensus aggregation
436
- - Returns: `DiagnosisResult` object
437
-
438
- **`create_variant(variant, **kwargs)` (Class Method)**
439
- - Factory method for creating specialized variants
440
- - Variants: "instant", "question_only", "budgeted", "no_budget", "ensemble"
441
-
442
- ### DiagnosisResult Class
443
-
444
- ```python
445
- @dataclass
446
- class DiagnosisResult:
447
- final_diagnosis: str
448
- ground_truth: str
449
- accuracy_score: float
450
- accuracy_reasoning: str
451
- total_cost: int
452
- iterations: int
453
- conversation_history: str
454
- ```
455
-
456
- ### Utility Functions
457
-
458
- **`run_mai_dxo_demo(case_info=None, case_details=None, ground_truth=None)`**
459
- - Convenience function for quick demonstrations
460
- - Returns: Dictionary of results from multiple variants
461
-
462
- ## 🧪 Testing and Validation
463
-
464
- ### Running Tests
465
- ```bash
466
- # Run the built-in demo
467
- python -m mai_dx.main
468
-
469
- # Run with custom cases
470
- python -c "
471
- from mai_dx import run_mai_dxo_demo
472
- results = run_mai_dxo_demo()
473
- print(results)
474
- "
475
- ```
476
-
477
- ### Benchmarking
478
- ```python
479
- import time
480
- from mai_dx import MaiDxOrchestrator
481
-
482
- # Performance benchmarking
483
- start_time = time.time()
484
- orchestrator = MaiDxOrchestrator()
485
- result = orchestrator.run(case_info, case_details, ground_truth)
486
- elapsed = time.time() - start_time
487
-
488
- print(f"Diagnosis completed in {elapsed:.2f} seconds")
489
- print(f"Accuracy: {result.accuracy_score}/5.0")
490
- print(f"Cost efficiency: ${result.total_cost/result.accuracy_score:.0f} per accuracy point")
491
- ```
492
-
493
- ## 🤝 Contributing
494
-
495
- We welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details.
496
-
497
- ### Development Setup
498
- ```bash
499
- git clone https://github.com/your-org/Open-MAI-Dx-Orchestrator.git
500
- cd Open-MAI-Dx-Orchestrator
501
- pip install -e ".[dev]"
502
- pre-commit install
503
- ```
504
 
505
- ### Code Style
506
- - Follow PEP 8 guidelines
507
- - Use type hints throughout
508
- - Maintain comprehensive docstrings
509
- - Add tests for new features
510
 
511
  ## 📄 License
512
 
 
34
 
35
  ----
36
 
37
+ ## Example
38
+
39
  ```python
40
  from mai_dx import MaiDxOrchestrator
41
 
 
54
  print(f"Cost: ${result.total_cost:,}")
55
  ```
56
 
57
+ ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
 
59
+ ## Documentation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
 
61
+ Learn more about this repository [with the docs](DOCS.md)
 
 
 
 
62
 
63
  ## 📄 License
64