README.md

---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:46716
- loss:MultipleNegativesRankingLoss
base_model: sentence-transformers/all-MiniLM-L6-v2
widget:
- source_sentence: Electromagnetic radiation behaves like particles as well as what?
  sentences:
  - quantum metrology allows us to attain a measurement precision that surpasses the
    classically achievable limit by using quantum characters. the metrology precision
    is raised from the standard quantum limit ( sql ) to the heisenberg limit ( hl
    ) by using entanglement. however, it was reported that the hl returns to the sql
    in the presence of local dephasing environments under the long encoding - time
    condition. we evaluate here the exact impacts of local dissipative environments
    on quantum metrology, based on the ramsey interferometer. it is found that the
    hl is asymptotically recovered under the long encoding - time condition for a
    finite number of the probe atoms. our analysis reveals that this is essentially
    due to the formation of a bound state between each atom and its environment. this
    provides an avenue for experimentation to implement quantum metrology under practical
    conditions via engineering of the formation of the system - environment bound
    state.
  - plasmons in two - dimensional electron systems with nonparabolic bands, such as
    graphene, feature strong dependence on electron - electron interactions. we use
    a many - body approach to relate plasmon dispersion at long wavelengths to landau
    fermi - liquid interactions and quasiparticle velocity. an identical renormalization
    is shown to arise for the magnetoplasmon resonance. for a model with n > > 1 fermion
    species, this approach predicts a power - law dependence for plasmon frequency
    vs. carrier concentration, valid in a wide range of doping densities, both high
    and low. gate tunability of plasmons in graphene can be exploited to directly
    probe the effects of electron - electron interaction.
  - 'the study of earth - mass extrasolar planets via the radial - velocity technique
    and the measurement of the potential cosmological variability of fundamental constants
    call for very - high - precision spectroscopy at the level of $ \ updelta \ lambda
    / \ lambda < 10 ^ { - 9 } $. wavelength accuracy is obtained by providing two
    fundamental ingredients : 1 ) an absolute and information - rich wavelength source
    and 2 ) the ability of the spectrograph and its data reduction of transferring
    the reference scale ( wavelengths ) to a measurement scale ( detector pixels )
    in a repeatable manner. the goal of this work is to improve the wavelength calibration
    accuracy of the harps spectrograph by combining the absolute spectral reference
    provided by the emission lines of a thorium - argon hollow - cathode lamp ( hcl
    ) with the spectrally rich and precise spectral information of a fabry - p \ ''
    erot - based calibration source. on the basis of calibration frames acquired each
    night since the fabry - p \ '' erot etalon was installed on harps in 2011, we
    construct a combined wavelength solution which fits simultaneously the thorium
    emission lines and the fabry - p \ '' erot lines. the combined fit is anchored
    to the absolute thorium wavelengths, which provide the ` zero - point '' of the
    spectrograph, while the fabry - p \ '' erot lines are used to improve the ( spectrally
    ) local precision. the obtained wavelength solution is verified for auto - consistency
    and tested against a solution obtained using the harps laser - frequency comb
    ( lfc ). the combined thorium + fabry - p \ '' erot wavelength solution shows
    significantly better performances compared to the thorium - only calibration.
    the presented techniques will therefore be used in the new harps and harps - n
    pipeline, and will be exported to the espresso spectrograph.'
- source_sentence: There are several types of wetlands including marshes, swamps,
    bogs, mudflats, and salt marshes. the three shared characteristics among these
    types—what makes them wetlands—are their hydrology, hydrophytic vegetation, and
    this?
  sentences:
  - we report updated measurements of branching fractions ( $ \ mathcal { b } $ )
    and cp - violating charge asymmetries ( $ \ mathcal { a _ { \ rm cp } } $ ) for
    charmless $ b $ decays at belle ii, which operates on or near the $ \ upsilon
    $ ( 4s ) resonance at the superkekb asymmetric energy $ e ^ { + } e ^ { - } $
    collider. we use samples of 2019 and 2020 data corresponding to 62. 8 fb $ ^ {
    - 1 } $ of integrated luminosity. the samples are analysed using two - dimensional
    fits in $ \ delta e $ and $ m _ { \ it bc } $ to determine signal yields of approximately
    568, 103, and 115 decays for the channels $ b ^ 0 \ to k ^ + \ pi ^ - $, $ b ^
    + \ to k _ { \ rm s } ^ 0 \ pi ^ + $, and $ b ^ 0 \ to \ pi ^ + \ pi ^ - $, respectively.
    signal yields are corrected for efficiencies determined from simulation and control
    data samples to obtain branching fractions and cp - violating asymmetries for
    flavour - specific channels. the results are compatible with known determinations
    and contribute important information to an early assessment of belle ii detector
    performance.
  - ') – characterised by its brown colour. health and environmental concerns associated
    with electronics assembly have gained increased attention in recent years, especially
    for products destined to go to european markets. electrical components are generally
    mounted in the following ways : through - hole ( sometimes referred to as '' pin
    - through - hole '' ) surface mount chassis mount rack mount lga / bga / pga socket
    = = industry = = the electronics industry consists of various sectors. the central
    driving force behind the entire electronics industry is the semiconductor industry
    sector, which has annual sales of over $ 481 billion as of 2018. the largest industry
    sector is e - commerce, which generated over $ 29 trillion in 2017. the most widely
    manufactured electronic device is the metal - oxide - semiconductor field - effect
    transistor ( mosfet ), with an estimated 13 sextillion mosfets having been manufactured
    between 1960 and 2018. in the 1960s, u. s. manufacturers were unable to compete
    with japanese companies such as sony and hitachi who could produce high - quality
    goods at lower prices. by the 1980s, however, u. s. manufacturers became the world
    leaders in semiconductor development and assembly. however, during the 1990s and
    subsequently, the industry shifted overwhelmingly to east asia ( a process begun
    with the initial movement of microchip mass - production there in the 1970s ),
    as plentiful, cheap labor, and increasing technological sophistication, became
    widely available there. over three decades, the united states '' global share
    of semiconductor manufacturing capacity fell, from 37 % in 1990, to 12 % in 2022.
    america '' s pre - eminent semiconductor manufacturer, intel corporation, fell
    far behind its subcontractor taiwan semiconductor manufacturing company ( tsmc
    ) in manufacturing technology. by that time, taiwan had become the world '' s
    leading source of advanced semiconductors — followed by south korea, the united
    states, japan, singapore, and china. important semiconductor industry facilities
    ( which often are subsidiaries of a leading producer based elsewhere ) also exist
    in europe ( notably the netherlands ), southeast asia, south america, and israel.
    = = see also = = = = references = = = = further reading = = horowitz, paul ; hill,
    winfield ( 1980 ). the art of electronics. cambridge university press. isbn 978
    - 0521370950. mims, forrest m. ( 2003 ). getting started in electronics. master
    publishing, incorporated. isbn 978 - 0 - 945053 - 28 - 6. = = external links =
    = navy 1998 navy electricity and electronics'
  - 'we construct two - band topological semimetals in four dimensions using the unstable
    homotopy of maps from the three - torus $ t ^ 3 $ ( brillouin zone of a 3d crystal
    ) to the two - sphere $ s ^ 2 $. dubbed ` ` hopf semimetals '' '', these gapless
    phases generically host nodal lines, with a surface enclosing such a nodal line
    in the four - dimensional brillouin zone carrying a hopf flux. these semimetals
    show a unique class of surface states : while some three - dimensional surfaces
    host gapless fermi - arc states { \ em and } drumhead states, other surfaces have
    gapless fermi surfaces. gapless two - dimensional corner states are also present
    at the intersection of three - dimensional surfaces.'
- source_sentence: What play several important roles in the human body?
  sentences:
  - the problem of ranking is a multi - billion dollar problem. in this paper we present
    an overview of several production quality ranking systems. we show that due to
    conflicting goals of employing the most effective machine learning models and
    responding to users in real time, ranking systems have evolved into a system of
    systems, where each subsystem can be viewed as a component layer. we view these
    layers as being data processing, representation learning, candidate selection
    and online inference. each layer employs different algorithms and tools, with
    every end - to - end ranking system spanning multiple architectures. our goal
    is to familiarize the general audience with a working knowledge of ranking at
    scale, the tools and algorithms employed and the challenges introduced by adopting
    a layered approach.
  - this tutorial review provides a guiding reference to researchers who want to have
    an overview of the large body of literature about graph spanners. it reviews the
    current literature covering various research streams about graph spanners, such
    as different formulations, sparsity and lightness results, computational complexity,
    dynamic algorithms, and applications. as an additional contribution, we offer
    a list of open problems on graph spanners.
  - we present a perturbative correction within initiator full configuration interaction
    quantum monte carlo ( i - fciqmc ). in the existing i - fciqmc algorithm, a significant
    number of spawned walkers are discarded due to the initiator criteria. here we
    show that these discarded walkers have a form that allows calculation of a second
    - order epstein - nesbet correction, that may be accumulated in a trivial and
    inexpensive manner, yet substantially improves i - fciqmc results. the correction
    is applied to the hubbard model, the uniform electron gas and molecular systems.
- source_sentence: The cells in the follicle undergo physical changes and produce
    a structure called a what?
  sentences:
  - Following ovulation, the ovarian cycle enters its luteal phase, illustrated in
    Figure 43.15 and the menstrual cycle enters its secretory phase, both of which
    run from about day 15 to 28. The luteal and secretory phases refer to changes
    in the ruptured follicle. The cells in the follicle undergo physical changes and
    produce a structure called a corpus luteum. The corpus luteum produces estrogen
    and progesterone. The progesterone facilitates the regrowth of the uterine lining
    and inhibits the release of further FSH and LH. The uterus is being prepared to
    accept a fertilized egg, should it occur during this cycle. The inhibition of
    FSH and LH prevents any further eggs and follicles from developing, while the
    progesterone is elevated. The level of estrogen produced by the corpus luteum
    increases to a steady level for the next few days. If no fertilized egg is implanted
    into the uterus, the corpus luteum degenerates and the levels of estrogen and
    progesterone decrease. The endometrium begins to degenerate as the progesterone
    levels drop, initiating the next menstrual cycle. The decrease in progesterone
    also allows the hypothalamus to send GnRH to the anterior pituitary, releasing
    FSH and LH and starting the cycles again. Figure 43.17 visually compares the ovarian
    and uterine cycles as well as the commensurate hormone levels.
  - An ammeter measures the current traveling through the circuit. They are designed
    to be connected to the circuit in series, and have an extremely low resistance.
    If an ammeter were connected in parallel, all of the current would go through
    the ammeter and very little through any other resistor. As such, it is necessary
    for the ammeter to be connected in series with the resistors. This allows the
    ammeter to accurately measure the current flow without causing any disruptions.
    In the circuit sketched above, the ammeter is .
  - ', narasimha. later he had visions of scrolls of complex mathematical content
    unfolding before his eyes. he often said, " an equation for me has no meaning
    unless it expresses a thought of god. " hardy cites ramanujan as remarking that
    all religions seemed equally true to him. hardy further argued that ramanujan
    '' s religious belief had been romanticised by westerners and overstated — in
    reference to his belief, not practice — by indian biographers. at the same time,
    he remarked on ramanujan '' s strict vegetarianism. similarly, in an interview
    with frontline, berndt said, " many people falsely promulgate mystical powers
    to ramanujan '' s mathematical thinking. it is not true. he has meticulously recorded
    every result in his three notebooks, " further speculating that ramanujan worked
    out intermediate results on slate that he could not afford the paper to record
    more permanently. berndt reported that janaki said in 1984 that ramanujan spent
    so much of his time on mathematics that he did not go to the temple, that she
    and her mother often fed him because he had no time to eat, and that most of the
    religious stories attributed to him originated with others. however, his orthopraxy
    was not in doubt. = = mathematical achievements = = in mathematics, there is a
    distinction between insight and formulating or working through a proof. ramanujan
    proposed an abundance of formulae that could be investigated later in depth. g.
    h. hardy said that ramanujan '' s discoveries are unusually rich and that there
    is often more to them than initially meets the eye. as a byproduct of his work,
    new directions of research were opened up. examples of the most intriguing of
    these formulae include infinite series for π, one of which is given below : 1
    π = 2 2 9801 [UNK] k = 0 ∞ ( 4 k )! ( 1103 + 26390 k ) ( k! ) 4 396 4 k. { \ displaystyle
    { \ frac { 1 } { \ pi } } = { \ frac { 2 { \ sqrt { 2 } } } { 9801 } } \ sum _
    { k = 0 } ^ { \ infty } { \ frac { ( 4k )! ( 1103 + 26390k ) } { ( k! ) ^ { 4
    } 396 ^ { 4k } } }. } this result is based on the negative fundamental discriminant
    d'
- source_sentence: What type of electrons are electrons that are not confined to the
    bond between two atoms?
  sentences:
  - Gap genes themselves are under the effect of maternal effect genes, such as bicoid
    and nanos. Gap genes also regulate each other to achieve their precise striped
    expression patterns. The maternal effect is when the phenotype of offspring is
    partly determined by the phenotype of its mother, irrespective of genotype. This
    often occurs when the mother supplies mRNA or proteins to the egg, affecting early
    development. In developing Drosophila, maternal effects include axis determination.
  - the human capacity for working together and with tools builds on cognitive abilities
    that, while not unique to humans, are most developed in humans both in scale and
    plasticity. our capacity to engage with collaborators and with technology requires
    a continuous expenditure of attentive work that we show may be understood in terms
    of what is heuristically argued as ` trust ' in socio - economic fields. by adopting
    a ` social physics ' of information approach, we are able to bring dimensional
    analysis to bear on an anthropological - economic issue. the cognitive - economic
    trade - off between group size and rate of attention to detail is the connection
    between these. this allows humans to scale cooperative effort across groups, from
    teams to communities, with a trade - off between group size and attention. we
    show here that an accurate concept of trust follows a bipartite ` economy of work
    ' model, and that this leads to correct predictions about the statistical distribution
    of group sizes in society. trust is essentially a cognitive - economic issue that
    depends on the memory cost of past behaviour and on the frequency of attentive
    policing of intent. all this leads to the characteristic ` fractal ' structure
    for human communities. the balance between attraction to some alpha attractor
    and dispersion due to conflict fully explains data from all relevant sources.
    the implications of our method suggest a broad applicability beyond purely social
    groupings to general resource constrained interactions, e. g. in work, technology,
    cybernetics, and generalized socio - economic systems of all kinds.
  - we consider a long - term optimal investment problem where an investor tries to
    minimize the probability of falling below a target growth rate. from a mathematical
    viewpoint, this is a large deviation control problem. this problem will be shown
    to relate to a risk - sensitive stochastic control problem for a sufficiently
    large time horizon. indeed, in our theorem we state a duality in the relation
    between the above two problems. furthermore, under a multidimensional linear gaussian
    model we obtain explicit solutions for the primal problem.
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: sciq eval
      type: sciq-eval
    metrics:
    - type: cosine_accuracy@1
      value: 0.647
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.751
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.786
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.827
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.647
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2503333333333333
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.15719999999999998
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.08269999999999998
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.647
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.751
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.786
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.827
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.735176233512708
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7059130952380956
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7086971683832702
      name: Cosine Map@100
---

# SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 384 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'What type of electrons are electrons that are not confined to the bond between two atoms?',
    "the human capacity for working together and with tools builds on cognitive abilities that, while not unique to humans, are most developed in humans both in scale and plasticity. our capacity to engage with collaborators and with technology requires a continuous expenditure of attentive work that we show may be understood in terms of what is heuristically argued as ` trust ' in socio - economic fields. by adopting a ` social physics ' of information approach, we are able to bring dimensional analysis to bear on an anthropological - economic issue. the cognitive - economic trade - off between group size and rate of attention to detail is the connection between these. this allows humans to scale cooperative effort across groups, from teams to communities, with a trade - off between group size and attention. we show here that an accurate concept of trust follows a bipartite ` economy of work ' model, and that this leads to correct predictions about the statistical distribution of group sizes in society. trust is essentially a cognitive - economic issue that depends on the memory cost of past behaviour and on the frequency of attentive policing of intent. all this leads to the characteristic ` fractal ' structure for human communities. the balance between attraction to some alpha attractor and dispersion due to conflict fully explains data from all relevant sources. the implications of our method suggest a broad applicability beyond purely social groupings to general resource constrained interactions, e. g. in work, technology, cybernetics, and generalized socio - economic systems of all kinds.",
    'we consider a long - term optimal investment problem where an investor tries to minimize the probability of falling below a target growth rate. from a mathematical viewpoint, this is a large deviation control problem. this problem will be shown to relate to a risk - sensitive stochastic control problem for a sufficiently large time horizon. indeed, in our theorem we state a duality in the relation between the above two problems. furthermore, under a multidimensional linear gaussian model we obtain explicit solutions for the primal problem.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Dataset: `sciq-eval`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.647      |
| cosine_accuracy@3   | 0.751      |
| cosine_accuracy@5   | 0.786      |
| cosine_accuracy@10  | 0.827      |
| cosine_precision@1  | 0.647      |
| cosine_precision@3  | 0.2503     |
| cosine_precision@5  | 0.1572     |
| cosine_precision@10 | 0.0827     |
| cosine_recall@1     | 0.647      |
| cosine_recall@3     | 0.751      |
| cosine_recall@5     | 0.786      |
| cosine_recall@10    | 0.827      |
| **cosine_ndcg@10**  | **0.7352** |
| cosine_mrr@10       | 0.7059     |
| cosine_map@100      | 0.7087     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset

* Size: 46,716 training samples
* Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
* Approximate statistics based on the first 1000 samples:
  |         | sentence_0                                                                        | sentence_1                                                                          | label                                                          |
  |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:---------------------------------------------------------------|
  | type    | string                                                                            | string                                                                              | float                                                          |
  | details | <ul><li>min: 5 tokens</li><li>mean: 18.07 tokens</li><li>max: 75 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 175.71 tokens</li><li>max: 256 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.24</li><li>max: 1.0</li></ul> |
* Samples:
  | sentence_0                                                                                                                                                   | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | label            |
  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
  | <code>What occurs when a former inhabited area gets disturbed?</code>                                                                                        | <code>recent approaches to improving the extraction of text embeddings from autoregressive large language models ( llms ) have largely focused on improvements to data, backbone pretrained language models, or improving task - differentiation via instructions. in this work, we address an architectural limitation of autoregressive models : token embeddings cannot contain information from tokens that appear later in the input. to address this limitation, we propose a simple approach, " echo embeddings, " in which we repeat the input twice in context and extract embeddings from the second occurrence. we show that echo embeddings of early tokens can encode information about later tokens, allowing us to maximally leverage high - quality llms for embeddings. on the mteb leaderboard, echo embeddings improve over classical embeddings by over 9 % zero - shot and by around 0. 7 % when fine - tuned. echo embeddings with a mistral - 7b model achieve state - of - the - art compared to prior open source mod...</code> | <code>0.0</code> |
  | <code>Veins subdivide repeatedly and branch throughout what?</code>                                                                                          | <code>the notion of generalization has moved away from the classical one defined in statistical learning theory towards an emphasis on out - of - domain generalization ( oodg ). recently, there is a growing focus on inductive generalization, where a progression of difficulty implicitly governs the direction of domain shifts. in inductive generalization, it is often assumed that the training data lie in the easier side, while the testing data lie in the harder side. the challenge is that training data are always finite, but a learner is expected to infer an inductive principle that could be applied in an unbounded manner. this emerging regime has appeared in the literature under different names, such as length / logical / algorithmic extrapolation, but a formal definition is lacking. this work provides such a formalization that centers on the concept of model successors. then we outline directions to adapt well - established techniques towards the learning of model successors. this work calls...</code> | <code>0.0</code> |
  | <code>What is the term for physicians and scientists who research and develop vaccines and treat and study conditions ranging from allergies to aids?</code> | <code>we generalize the hierarchy construction to generic 2 + 1d topological orders ( which can be non - abelian ) by condensing abelian anyons in one topological order to construct a new one. we show that such construction is reversible and leads to a new equivalence relation between topological orders. we refer to the corresponding equivalent class ( the orbit of the hierarchy construction ) as " the non - abelian family ". each non - abelian family has one or a few root topological orders with the smallest number of anyon types. all the abelian topological orders belong to the trivial non - abelian family whose root is the trivial topological order. we show that abelian anyons in root topological orders must be bosons or fermions with trivial mutual statistics between them. the classification of topological orders is then greatly simplified, by focusing on the roots of each family : those roots are given by non - abelian modular extensions of representation categories of abelian groups.</code>      | <code>0.0</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `num_train_epochs`: 1
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 32
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `tp_size`: 0
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Training Logs
| Epoch  | Step | Training Loss | sciq-eval_cosine_ndcg@10 |
|:------:|:----:|:-------------:|:------------------------:|
| 0.0685 | 100  | -             | 0.6007                   |
| 0.1370 | 200  | -             | 0.7026                   |
| 0.2055 | 300  | -             | 0.7167                   |
| 0.2740 | 400  | -             | 0.7195                   |
| 0.3425 | 500  | 2.8082        | 0.7150                   |
| 0.4110 | 600  | -             | 0.7292                   |
| 0.4795 | 700  | -             | 0.7356                   |
| 0.5479 | 800  | -             | 0.7428                   |
| 0.6164 | 900  | -             | 0.7399                   |
| 0.6849 | 1000 | 2.6228        | 0.7339                   |
| 0.7534 | 1100 | -             | 0.7356                   |
| 0.8219 | 1200 | -             | 0.7375                   |
| 0.8904 | 1300 | -             | 0.7385                   |
| 0.9589 | 1400 | -             | 0.7351                   |
| 1.0    | 1460 | -             | 0.7352                   |


### Framework Versions
- Python: 3.12.8
- Sentence Transformers: 3.4.1
- Transformers: 4.51.3
- PyTorch: 2.5.1+cu124
- Accelerate: 1.3.0
- Datasets: 3.2.0
- Tokenizers: 0.21.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->