jiagengwu commited on
Commit
51b43ef
Β·
verified Β·
1 Parent(s): 98dc0dd

Update docs.md

Browse files
Files changed (1) hide show
  1. docs.md +7 -8
docs.md CHANGED
@@ -22,12 +22,11 @@
22
 
23
  </div>
24
 
25
-
26
  <h2>πŸ“œ Background</h2>
27
  <p>Recent advances in <strong>Large Language Models (LLMs)</strong> have demonstrated transformative potential in <strong>healthcare</strong>, yet concerns remain around their reliability and clinical validity across diverse clinical tasks, specialties, and languages. To support timely and trustworthy evaluation, building upon our <a href="https://ai.nejm.org/doi/full/10.1056/AIra2400012">systematic review</a> of global clinical text resources, we introduce <a href="https://arxiv.org/abs/2504.19467">BRIDGE</a>, <strong>a multilingual benchmark that comprises 87 real-world clinical text tasks spanning nine languages and more than one million samples</strong>. Furthermore, we construct this leaderboard of LLM in clinical text understanding by systematically evaluating <strong>52 state-of-the-art LLMs</strong> (by 2024/04/29).</p>
28
 
29
  <div style="text-align: center;">
30
- <img src="figure/dataset.png" style="width: 100%;" alt="BRIDGE benchmark graphic">
31
  </div>
32
 
33
  <h2>πŸ† BRIDGE Leaderboard</h2>
@@ -40,7 +39,7 @@
40
  <p>In addition, BRIDGE offers multiple <strong>model filters</strong> and <strong>task filters</strong> to enable users to explore LLM performance across <strong>different clinical contexts</strong>, empowering researchers and clinicians to make informed decisions and track model advancements over time.</p>
41
 
42
  <div style="text-align: center;">
43
- <img src="figure/model.png" style="width: 100%;" alt="BRIDGE benchmark graphic">
44
  </div>
45
 
46
 
@@ -64,13 +63,13 @@ More Details can be found in our <a href="https://arxiv.org/abs/2504.19467">BRID
64
  <h2>πŸ› οΈ How to Evaluate Your Model on BRIDGE ?</h2>
65
  <h4>πŸ“‚ Dataset Access</h4>
66
  <p>All fully open-access datasets in BRIDGE are available in <a href="https://huggingface.co/datasets/YLab-Open/BRIDGE-Open">BRIDGE-Open</a>. To ensure fairness of this leaderboard, we publicly release the following data for each task:
67
- Five completed samples serving as few-shot examples, and all testing samples with instruction and input information.</p>
68
 
69
  <p>Due to privacy and security considerations of clinical data, regulated-access datasets can not be directly published. However, all detailed task descriptions and their corresponding data sources are available in our <a href="https://arxiv.org/abs/2504.19467">BRIDGE paper</a>.
70
  Importantly, all 87 datasets have been verified to be either fully open-access or publicly accessible via reasonable request.</p>
71
 
72
  <h4>πŸ”₯ Result Submission and Evaluation</h4>
73
- If you would like to submit your model results to BRIDGE and demonstrate its performance, please follow these steps:
74
  <ul>
75
  <li><strong>Run local inference:</strong> Download the <a href="https://huggingface.co/datasets/YLab-Open/BRIDGE-Open">BRIDGE-Open</a> dataset and perform inference locally. Save the generated output of each sample in its "pred" field for each dataset file.</li>
76
  <li><strong>Prepare your submission:</strong> Send your results to us (see the following Contact Information). Please clearly describe your model details and experimental settings, including the inference strategy used (zero-shot, few-shot, or CoT), and note any additional relevant configurations if applicable.</li>
@@ -92,7 +91,7 @@ If you have clinical text datasets that you would like to share for broader expl
92
  </ul>
93
 
94
  <h2>πŸ“¬ Contact Information</h2>
95
- If you have any questions about BRIDGE or the leaderboard, feel free to reach out!
96
  <ul>
97
  <li><strong>Leaderboard Managers</strong>: Jiageng Wu ([email protected]), Kevin Xie ([email protected])</li>
98
  <li><strong>Benchmark Managers</strong>: Bowen Gu ([email protected]), Jiageng Wu ([email protected])</li>
@@ -100,8 +99,8 @@ If you have any questions about BRIDGE or the leaderboard, feel free to reach ou
100
  </ul>
101
 
102
  <h2>πŸ“š Citation</h2>
103
- <pre><code>
104
- @article{BRIDGE-benchmark,
105
  title={BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text},
106
  author={Wu, Jiageng and Gu, Bowen and Zhou, Ren and Xie, Kevin and Snyder, Doug and Jiang, Yixing and Carducci, Valentina and Wyss, Richard and Desai, Rishi J and Alsentzer, Emily and Celi, Leo Anthony and Rodman, Adam and Schneeweiss, Sebastian and Chen, Jonathan H. and Romero-Brufau, Santiago and Lin, Kueiyu Joshua and Yang, Jie},
107
  year={2025},
 
22
 
23
  </div>
24
 
 
25
  <h2>πŸ“œ Background</h2>
26
  <p>Recent advances in <strong>Large Language Models (LLMs)</strong> have demonstrated transformative potential in <strong>healthcare</strong>, yet concerns remain around their reliability and clinical validity across diverse clinical tasks, specialties, and languages. To support timely and trustworthy evaluation, building upon our <a href="https://ai.nejm.org/doi/full/10.1056/AIra2400012">systematic review</a> of global clinical text resources, we introduce <a href="https://arxiv.org/abs/2504.19467">BRIDGE</a>, <strong>a multilingual benchmark that comprises 87 real-world clinical text tasks spanning nine languages and more than one million samples</strong>. Furthermore, we construct this leaderboard of LLM in clinical text understanding by systematically evaluating <strong>52 state-of-the-art LLMs</strong> (by 2024/04/29).</p>
27
 
28
  <div style="text-align: center;">
29
+ <img src="./dataset.png" style="width: 100%;" alt="BRIDGE benchmark graphic">
30
  </div>
31
 
32
  <h2>πŸ† BRIDGE Leaderboard</h2>
 
39
  <p>In addition, BRIDGE offers multiple <strong>model filters</strong> and <strong>task filters</strong> to enable users to explore LLM performance across <strong>different clinical contexts</strong>, empowering researchers and clinicians to make informed decisions and track model advancements over time.</p>
40
 
41
  <div style="text-align: center;">
42
+ <img src="./model.png" style="width: 100%;" alt="BRIDGE benchmark graphic">
43
  </div>
44
 
45
 
 
63
  <h2>πŸ› οΈ How to Evaluate Your Model on BRIDGE ?</h2>
64
  <h4>πŸ“‚ Dataset Access</h4>
65
  <p>All fully open-access datasets in BRIDGE are available in <a href="https://huggingface.co/datasets/YLab-Open/BRIDGE-Open">BRIDGE-Open</a>. To ensure fairness of this leaderboard, we publicly release the following data for each task:
66
+ Five completed samples serve as few-shot examples, and all testing samples with instruction and input information.</p>
67
 
68
  <p>Due to privacy and security considerations of clinical data, regulated-access datasets can not be directly published. However, all detailed task descriptions and their corresponding data sources are available in our <a href="https://arxiv.org/abs/2504.19467">BRIDGE paper</a>.
69
  Importantly, all 87 datasets have been verified to be either fully open-access or publicly accessible via reasonable request.</p>
70
 
71
  <h4>πŸ”₯ Result Submission and Evaluation</h4>
72
+ <p>If you would like to submit your model results to BRIDGE and demonstrate its performance, please follow these steps:</p>
73
  <ul>
74
  <li><strong>Run local inference:</strong> Download the <a href="https://huggingface.co/datasets/YLab-Open/BRIDGE-Open">BRIDGE-Open</a> dataset and perform inference locally. Save the generated output of each sample in its "pred" field for each dataset file.</li>
75
  <li><strong>Prepare your submission:</strong> Send your results to us (see the following Contact Information). Please clearly describe your model details and experimental settings, including the inference strategy used (zero-shot, few-shot, or CoT), and note any additional relevant configurations if applicable.</li>
 
91
  </ul>
92
 
93
  <h2>πŸ“¬ Contact Information</h2>
94
+ <p>If you have any questions about BRIDGE or the leaderboard, feel free to reach out!</p>
95
  <ul>
96
  <li><strong>Leaderboard Managers</strong>: Jiageng Wu ([email protected]), Kevin Xie ([email protected])</li>
97
  <li><strong>Benchmark Managers</strong>: Bowen Gu ([email protected]), Jiageng Wu ([email protected])</li>
 
99
  </ul>
100
 
101
  <h2>πŸ“š Citation</h2>
102
+ <p>If you find this leaderboard useful for your research and applications, please cite the following papers:</p>
103
+ <pre><code>@article{BRIDGE-benchmark,
104
  title={BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text},
105
  author={Wu, Jiageng and Gu, Bowen and Zhou, Ren and Xie, Kevin and Snyder, Doug and Jiang, Yixing and Carducci, Valentina and Wyss, Richard and Desai, Rishi J and Alsentzer, Emily and Celi, Leo Anthony and Rodman, Adam and Schneeweiss, Sebastian and Chen, Jonathan H. and Romero-Brufau, Santiago and Lin, Kueiyu Joshua and Yang, Jie},
106
  year={2025},