ybchen928/oncall-guide-ai at b497e77c78027f0e7c76d7bdd57ead1299993593

oncall-guide-ai / evaluation

310 kB

6 contributors

YanBoChen

Add adaptive relevance thresholds for query complexity in PrecisionMRRAnalyzer; fix typo in condition mapping for postpartum hemorrhage

7620d26 about 1 month ago

old
Before Run the 1st Evalation: Add Precision & MRR Chart Generator and a sample test query about 1 month ago
direct_llm_evaluator.py
16.5 kB

Enhance Direct LLM Evaluator and Judge Evaluator: about 1 month ago
latency_evaluator.py
41.4 kB

Update threshold values in latency evaluator and coverage chart generator; enhance precision and MRR analysis with corrected thresholds and new chart generator for detailed metrics visualization. about 1 month ago
metric1_latency_chart_generator.py

13.6 kB

Before Run the 1st Evalation: Add Precision & MRR Chart Generator and a sample test query about 1 month ago
metric2_extraction_chart_generator.py

8.63 kB

Before Run the 1st Evalation: Add Precision & MRR Chart Generator and a sample test query about 1 month ago
metric3_relevance_chart_generator.py

9.93 kB

Update threshold values in latency evaluator and coverage chart generator; enhance precision and MRR analysis with corrected thresholds and new chart generator for detailed metrics visualization. about 1 month ago
metric4_coverage_chart_generator.py

9.32 kB

Update threshold values in latency evaluator and coverage chart generator; enhance precision and MRR analysis with corrected thresholds and new chart generator for detailed metrics visualization. about 1 month ago
metric5_6_judge_evaluator_manual.md

9.86 kB

Add multi-system evaluation support for clinical actionability and evidence quality metrics about 1 month ago
metric5_6_llm_judge_chart_generator.py
19.5 kB

Add multi-system evaluation support for clinical actionability and evidence quality metrics about 1 month ago
metric5_6_llm_judge_evaluator.py

30.3 kB

Enhance Direct LLM Evaluator and Judge Evaluator: about 1 month ago
metric7_8_precision_MRR.py
16.8 kB

Add adaptive relevance thresholds for query complexity in PrecisionMRRAnalyzer; fix typo in condition mapping for postpartum hemorrhage about 1 month ago
metric7_8_precision_mrr_chart_generator.py

23.8 kB

Update threshold values in latency evaluator and coverage chart generator; enhance precision and MRR analysis with corrected thresholds and new chart generator for detailed metrics visualization. about 1 month ago
pre_user_query_evaluate.txt
318 Bytes

Add latency and relevance evaluators for medical query analysis (evaluatoin) about 1 month ago
single_test_query.txt
122 Bytes

Before Run the 1st Evalation: Add Precision & MRR Chart Generator and a sample test query about 1 month ago
user_query.txt
2.06 kB

fix(mild bug): enhance user query prompts (more robust dealing process with .txt or .json) and add postpartum hemorrhage condition mapping about 1 month ago