metadata

tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:46716
  - loss:MultipleNegativesRankingLoss
base_model: sentence-transformers/all-MiniLM-L6-v2
widget:
  - source_sentence: Electromagnetic radiation behaves like particles as well as what?
    sentences:
      - >-
        quantum metrology allows us to attain a measurement precision that
        surpasses the classically achievable limit by using quantum characters.
        the metrology precision is raised from the standard quantum limit ( sql
        ) to the heisenberg limit ( hl ) by using entanglement. however, it was
        reported that the hl returns to the sql in the presence of local
        dephasing environments under the long encoding - time condition. we
        evaluate here the exact impacts of local dissipative environments on
        quantum metrology, based on the ramsey interferometer. it is found that
        the hl is asymptotically recovered under the long encoding - time
        condition for a finite number of the probe atoms. our analysis reveals
        that this is essentially due to the formation of a bound state between
        each atom and its environment. this provides an avenue for
        experimentation to implement quantum metrology under practical
        conditions via engineering of the formation of the system - environment
        bound state.
      - >-
        plasmons in two - dimensional electron systems with nonparabolic bands,
        such as graphene, feature strong dependence on electron - electron
        interactions. we use a many - body approach to relate plasmon dispersion
        at long wavelengths to landau fermi - liquid interactions and
        quasiparticle velocity. an identical renormalization is shown to arise
        for the magnetoplasmon resonance. for a model with n > > 1 fermion
        species, this approach predicts a power - law dependence for plasmon
        frequency vs. carrier concentration, valid in a wide range of doping
        densities, both high and low. gate tunability of plasmons in graphene
        can be exploited to directly probe the effects of electron - electron
        interaction.
      - >-
        the study of earth - mass extrasolar planets via the radial - velocity
        technique and the measurement of the potential cosmological variability
        of fundamental constants call for very - high - precision spectroscopy
        at the level of $ \ updelta \ lambda / \ lambda < 10 ^ { - 9 } $.
        wavelength accuracy is obtained by providing two fundamental ingredients
        : 1 ) an absolute and information - rich wavelength source and 2 ) the
        ability of the spectrograph and its data reduction of transferring the
        reference scale ( wavelengths ) to a measurement scale ( detector pixels
        ) in a repeatable manner. the goal of this work is to improve the
        wavelength calibration accuracy of the harps spectrograph by combining
        the absolute spectral reference provided by the emission lines of a
        thorium - argon hollow - cathode lamp ( hcl ) with the spectrally rich
        and precise spectral information of a fabry - p \ ' erot - based
        calibration source. on the basis of calibration frames acquired each
        night since the fabry - p \ ' erot etalon was installed on harps in
        2011, we construct a combined wavelength solution which fits
        simultaneously the thorium emission lines and the fabry - p \ ' erot
        lines. the combined fit is anchored to the absolute thorium wavelengths,
        which provide the ` zero - point ' of the spectrograph, while the fabry
        - p \ ' erot lines are used to improve the ( spectrally ) local
        precision. the obtained wavelength solution is verified for auto -
        consistency and tested against a solution obtained using the harps laser
        - frequency comb ( lfc ). the combined thorium + fabry - p \ ' erot
        wavelength solution shows significantly better performances compared to
        the thorium - only calibration. the presented techniques will therefore
        be used in the new harps and harps - n pipeline, and will be exported to
        the espresso spectrograph.
  - source_sentence: >-
      There are several types of wetlands including marshes, swamps, bogs,
      mudflats, and salt marshes. the three shared characteristics among these
      types—what makes them wetlands—are their hydrology, hydrophytic
      vegetation, and this?
    sentences:
      - >-
        we report updated measurements of branching fractions ( $ \ mathcal { b
        } $ ) and cp - violating charge asymmetries ( $ \ mathcal { a _ { \ rm
        cp } } $ ) for charmless $ b $ decays at belle ii, which operates on or
        near the $ \ upsilon $ ( 4s ) resonance at the superkekb asymmetric
        energy $ e ^ { + } e ^ { - } $ collider. we use samples of 2019 and 2020
        data corresponding to 62. 8 fb $ ^ { - 1 } $ of integrated luminosity.
        the samples are analysed using two - dimensional fits in $ \ delta e $
        and $ m _ { \ it bc } $ to determine signal yields of approximately 568,
        103, and 115 decays for the channels $ b ^ 0 \ to k ^ + \ pi ^ - $, $ b
        ^ + \ to k _ { \ rm s } ^ 0 \ pi ^ + $, and $ b ^ 0 \ to \ pi ^ + \ pi ^
        - $, respectively. signal yields are corrected for efficiencies
        determined from simulation and control data samples to obtain branching
        fractions and cp - violating asymmetries for flavour - specific
        channels. the results are compatible with known determinations and
        contribute important information to an early assessment of belle ii
        detector performance.
      - >-
        ) – characterised by its brown colour. health and environmental concerns
        associated with electronics assembly have gained increased attention in
        recent years, especially for products destined to go to european
        markets. electrical components are generally mounted in the following
        ways : through - hole ( sometimes referred to as ' pin - through - hole
        ' ) surface mount chassis mount rack mount lga / bga / pga socket = =
        industry = = the electronics industry consists of various sectors. the
        central driving force behind the entire electronics industry is the
        semiconductor industry sector, which has annual sales of over $ 481
        billion as of 2018. the largest industry sector is e - commerce, which
        generated over $ 29 trillion in 2017. the most widely manufactured
        electronic device is the metal - oxide - semiconductor field - effect
        transistor ( mosfet ), with an estimated 13 sextillion mosfets having
        been manufactured between 1960 and 2018. in the 1960s, u. s.
        manufacturers were unable to compete with japanese companies such as
        sony and hitachi who could produce high - quality goods at lower prices.
        by the 1980s, however, u. s. manufacturers became the world leaders in
        semiconductor development and assembly. however, during the 1990s and
        subsequently, the industry shifted overwhelmingly to east asia ( a
        process begun with the initial movement of microchip mass - production
        there in the 1970s ), as plentiful, cheap labor, and increasing
        technological sophistication, became widely available there. over three
        decades, the united states ' global share of semiconductor manufacturing
        capacity fell, from 37 % in 1990, to 12 % in 2022. america ' s pre -
        eminent semiconductor manufacturer, intel corporation, fell far behind
        its subcontractor taiwan semiconductor manufacturing company ( tsmc ) in
        manufacturing technology. by that time, taiwan had become the world ' s
        leading source of advanced semiconductors — followed by south korea, the
        united states, japan, singapore, and china. important semiconductor
        industry facilities ( which often are subsidiaries of a leading producer
        based elsewhere ) also exist in europe ( notably the netherlands ),
        southeast asia, south america, and israel. = = see also = = = =
        references = = = = further reading = = horowitz, paul ; hill, winfield (
        1980 ). the art of electronics. cambridge university press. isbn 978 -
        0521370950. mims, forrest m. ( 2003 ). getting started in electronics.
        master publishing, incorporated. isbn 978 - 0 - 945053 - 28 - 6. = =
        external links = = navy 1998 navy electricity and electronics
      - >-
        we construct two - band topological semimetals in four dimensions using
        the unstable homotopy of maps from the three - torus $ t ^ 3 $ (
        brillouin zone of a 3d crystal ) to the two - sphere $ s ^ 2 $. dubbed `
        ` hopf semimetals ' ', these gapless phases generically host nodal
        lines, with a surface enclosing such a nodal line in the four -
        dimensional brillouin zone carrying a hopf flux. these semimetals show a
        unique class of surface states : while some three - dimensional surfaces
        host gapless fermi - arc states { \ em and } drumhead states, other
        surfaces have gapless fermi surfaces. gapless two - dimensional corner
        states are also present at the intersection of three - dimensional
        surfaces.
  - source_sentence: What play several important roles in the human body?
    sentences:
      - >-
        the problem of ranking is a multi - billion dollar problem. in this
        paper we present an overview of several production quality ranking
        systems. we show that due to conflicting goals of employing the most
        effective machine learning models and responding to users in real time,
        ranking systems have evolved into a system of systems, where each
        subsystem can be viewed as a component layer. we view these layers as
        being data processing, representation learning, candidate selection and
        online inference. each layer employs different algorithms and tools,
        with every end - to - end ranking system spanning multiple
        architectures. our goal is to familiarize the general audience with a
        working knowledge of ranking at scale, the tools and algorithms employed
        and the challenges introduced by adopting a layered approach.
      - >-
        this tutorial review provides a guiding reference to researchers who
        want to have an overview of the large body of literature about graph
        spanners. it reviews the current literature covering various research
        streams about graph spanners, such as different formulations, sparsity
        and lightness results, computational complexity, dynamic algorithms, and
        applications. as an additional contribution, we offer a list of open
        problems on graph spanners.
      - >-
        we present a perturbative correction within initiator full configuration
        interaction quantum monte carlo ( i - fciqmc ). in the existing i -
        fciqmc algorithm, a significant number of spawned walkers are discarded
        due to the initiator criteria. here we show that these discarded walkers
        have a form that allows calculation of a second - order epstein - nesbet
        correction, that may be accumulated in a trivial and inexpensive manner,
        yet substantially improves i - fciqmc results. the correction is applied
        to the hubbard model, the uniform electron gas and molecular systems.
  - source_sentence: >-
      The cells in the follicle undergo physical changes and produce a structure
      called a what?
    sentences:
      - >-
        Following ovulation, the ovarian cycle enters its luteal phase,
        illustrated in Figure 43.15 and the menstrual cycle enters its secretory
        phase, both of which run from about day 15 to 28. The luteal and
        secretory phases refer to changes in the ruptured follicle. The cells in
        the follicle undergo physical changes and produce a structure called a
        corpus luteum. The corpus luteum produces estrogen and progesterone. The
        progesterone facilitates the regrowth of the uterine lining and inhibits
        the release of further FSH and LH. The uterus is being prepared to
        accept a fertilized egg, should it occur during this cycle. The
        inhibition of FSH and LH prevents any further eggs and follicles from
        developing, while the progesterone is elevated. The level of estrogen
        produced by the corpus luteum increases to a steady level for the next
        few days. If no fertilized egg is implanted into the uterus, the corpus
        luteum degenerates and the levels of estrogen and progesterone decrease.
        The endometrium begins to degenerate as the progesterone levels drop,
        initiating the next menstrual cycle. The decrease in progesterone also
        allows the hypothalamus to send GnRH to the anterior pituitary,
        releasing FSH and LH and starting the cycles again. Figure 43.17
        visually compares the ovarian and uterine cycles as well as the
        commensurate hormone levels.
      - >-
        An ammeter measures the current traveling through the circuit. They are
        designed to be connected to the circuit in series, and have an extremely
        low resistance. If an ammeter were connected in parallel, all of the
        current would go through the ammeter and very little through any other
        resistor. As such, it is necessary for the ammeter to be connected in
        series with the resistors. This allows the ammeter to accurately measure
        the current flow without causing any disruptions. In the circuit
        sketched above, the ammeter is .
      - >-
        , narasimha. later he had visions of scrolls of complex mathematical
        content unfolding before his eyes. he often said, " an equation for me
        has no meaning unless it expresses a thought of god. " hardy cites
        ramanujan as remarking that all religions seemed equally true to him.
        hardy further argued that ramanujan ' s religious belief had been
        romanticised by westerners and overstated — in reference to his belief,
        not practice — by indian biographers. at the same time, he remarked on
        ramanujan ' s strict vegetarianism. similarly, in an interview with
        frontline, berndt said, " many people falsely promulgate mystical powers
        to ramanujan ' s mathematical thinking. it is not true. he has
        meticulously recorded every result in his three notebooks, " further
        speculating that ramanujan worked out intermediate results on slate that
        he could not afford the paper to record more permanently. berndt
        reported that janaki said in 1984 that ramanujan spent so much of his
        time on mathematics that he did not go to the temple, that she and her
        mother often fed him because he had no time to eat, and that most of the
        religious stories attributed to him originated with others. however, his
        orthopraxy was not in doubt. = = mathematical achievements = = in
        mathematics, there is a distinction between insight and formulating or
        working through a proof. ramanujan proposed an abundance of formulae
        that could be investigated later in depth. g. h. hardy said that
        ramanujan ' s discoveries are unusually rich and that there is often
        more to them than initially meets the eye. as a byproduct of his work,
        new directions of research were opened up. examples of the most
        intriguing of these formulae include infinite series for π, one of which
        is given below : 1 π = 2 2 9801 [UNK] k = 0 ∞ ( 4 k )! ( 1103 + 26390 k
        ) ( k! ) 4 396 4 k. { \ displaystyle { \ frac { 1 } { \ pi } } = { \
        frac { 2 { \ sqrt { 2 } } } { 9801 } } \ sum _ { k = 0 } ^ { \ infty } {
        \ frac { ( 4k )! ( 1103 + 26390k ) } { ( k! ) ^ { 4 } 396 ^ { 4k } } }.
        } this result is based on the negative fundamental discriminant d
  - source_sentence: >-
      What type of electrons are electrons that are not confined to the bond
      between two atoms?
    sentences:
      - >-
        Gap genes themselves are under the effect of maternal effect genes, such
        as bicoid and nanos. Gap genes also regulate each other to achieve their
        precise striped expression patterns. The maternal effect is when the
        phenotype of offspring is partly determined by the phenotype of its
        mother, irrespective of genotype. This often occurs when the mother
        supplies mRNA or proteins to the egg, affecting early development. In
        developing Drosophila, maternal effects include axis determination.
      - >-
        the human capacity for working together and with tools builds on
        cognitive abilities that, while not unique to humans, are most developed
        in humans both in scale and plasticity. our capacity to engage with
        collaborators and with technology requires a continuous expenditure of
        attentive work that we show may be understood in terms of what is
        heuristically argued as ` trust ' in socio - economic fields. by
        adopting a ` social physics ' of information approach, we are able to
        bring dimensional analysis to bear on an anthropological - economic
        issue. the cognitive - economic trade - off between group size and rate
        of attention to detail is the connection between these. this allows
        humans to scale cooperative effort across groups, from teams to
        communities, with a trade - off between group size and attention. we
        show here that an accurate concept of trust follows a bipartite `
        economy of work ' model, and that this leads to correct predictions
        about the statistical distribution of group sizes in society. trust is
        essentially a cognitive - economic issue that depends on the memory cost
        of past behaviour and on the frequency of attentive policing of intent.
        all this leads to the characteristic ` fractal ' structure for human
        communities. the balance between attraction to some alpha attractor and
        dispersion due to conflict fully explains data from all relevant
        sources. the implications of our method suggest a broad applicability
        beyond purely social groupings to general resource constrained
        interactions, e. g. in work, technology, cybernetics, and generalized
        socio - economic systems of all kinds.
      - >-
        we consider a long - term optimal investment problem where an investor
        tries to minimize the probability of falling below a target growth rate.
        from a mathematical viewpoint, this is a large deviation control
        problem. this problem will be shown to relate to a risk - sensitive
        stochastic control problem for a sufficiently large time horizon.
        indeed, in our theorem we state a duality in the relation between the
        above two problems. furthermore, under a multidimensional linear
        gaussian model we obtain explicit solutions for the primal problem.
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: sciq eval
          type: sciq-eval
        metrics:
          - type: cosine_accuracy@1
            value: 0.647
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.751
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.786
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.827
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.647
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2503333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.15719999999999998
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08269999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.647
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.751
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.786
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.827
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.735176233512708
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7059130952380956
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7086971683832702
            name: Cosine Map@100

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-MiniLM-L6-v2
Maximum Sequence Length: 256 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'What type of electrons are electrons that are not confined to the bond between two atoms?',
    "the human capacity for working together and with tools builds on cognitive abilities that, while not unique to humans, are most developed in humans both in scale and plasticity. our capacity to engage with collaborators and with technology requires a continuous expenditure of attentive work that we show may be understood in terms of what is heuristically argued as ` trust ' in socio - economic fields. by adopting a ` social physics ' of information approach, we are able to bring dimensional analysis to bear on an anthropological - economic issue. the cognitive - economic trade - off between group size and rate of attention to detail is the connection between these. this allows humans to scale cooperative effort across groups, from teams to communities, with a trade - off between group size and attention. we show here that an accurate concept of trust follows a bipartite ` economy of work ' model, and that this leads to correct predictions about the statistical distribution of group sizes in society. trust is essentially a cognitive - economic issue that depends on the memory cost of past behaviour and on the frequency of attentive policing of intent. all this leads to the characteristic ` fractal ' structure for human communities. the balance between attraction to some alpha attractor and dispersion due to conflict fully explains data from all relevant sources. the implications of our method suggest a broad applicability beyond purely social groupings to general resource constrained interactions, e. g. in work, technology, cybernetics, and generalized socio - economic systems of all kinds.",
    'we consider a long - term optimal investment problem where an investor tries to minimize the probability of falling below a target growth rate. from a mathematical viewpoint, this is a large deviation control problem. this problem will be shown to relate to a risk - sensitive stochastic control problem for a sufficiently large time horizon. indeed, in our theorem we state a duality in the relation between the above two problems. furthermore, under a multidimensional linear gaussian model we obtain explicit solutions for the primal problem.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: sciq-eval
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.647
cosine_accuracy@3	0.751
cosine_accuracy@5	0.786
cosine_accuracy@10	0.827
cosine_precision@1	0.647
cosine_precision@3	0.2503
cosine_precision@5	0.1572
cosine_precision@10	0.0827
cosine_recall@1	0.647
cosine_recall@3	0.751
cosine_recall@5	0.786
cosine_recall@10	0.827
cosine_ndcg@10	0.7352
cosine_mrr@10	0.7059
cosine_map@100	0.7087

Training Details

Training Dataset

Unnamed Dataset

Size: 46,716 training samples
Columns: sentence_0, sentence_1, and label
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1 label
type string string float
details
min: 5 tokens
mean: 18.07 tokens
max: 75 tokens

min: 2 tokens
mean: 175.71 tokens
max: 256 tokens

min: 0.0
mean: 0.24
max: 1.0

	sentence_0	sentence_1	label
type	string	string	float
details	min: 5 tokens mean: 18.07 tokens max: 75 tokens	min: 2 tokens mean: 175.71 tokens max: 256 tokens	min: 0.0 mean: 0.24 max: 1.0

Samples:

sentence_0	sentence_1	label
`What occurs when a former inhabited area gets disturbed?`	recent approaches to improving the extraction of text embeddings from autoregressive large language models ( llms ) have largely focused on improvements to data, backbone pretrained language models, or improving task - differentiation via instructions. in this work, we address an architectural limitation of autoregressive models : token embeddings cannot contain information from tokens that appear later in the input. to address this limitation, we propose a simple approach, " echo embeddings, " in which we repeat the input twice in context and extract embeddings from the second occurrence. we show that echo embeddings of early tokens can encode information about later tokens, allowing us to maximally leverage high - quality llms for embeddings. on the mteb leaderboard, echo embeddings improve over classical embeddings by over 9 % zero - shot and by around 0. 7 % when fine - tuned. echo embeddings with a mistral - 7b model achieve state - of - the - art compared to prior open source mod...	`0.0`
`Veins subdivide repeatedly and branch throughout what?`	the notion of generalization has moved away from the classical one defined in statistical learning theory towards an emphasis on out - of - domain generalization ( oodg ). recently, there is a growing focus on inductive generalization, where a progression of difficulty implicitly governs the direction of domain shifts. in inductive generalization, it is often assumed that the training data lie in the easier side, while the testing data lie in the harder side. the challenge is that training data are always finite, but a learner is expected to infer an inductive principle that could be applied in an unbounded manner. this emerging regime has appeared in the literature under different names, such as length / logical / algorithmic extrapolation, but a formal definition is lacking. this work provides such a formalization that centers on the concept of model successors. then we outline directions to adapt well - established techniques towards the learning of model successors. this work calls...	`0.0`
`What is the term for physicians and scientists who research and develop vaccines and treat and study conditions ranging from allergies to aids?`	we generalize the hierarchy construction to generic 2 + 1d topological orders ( which can be non - abelian ) by condensing abelian anyons in one topological order to construct a new one. we show that such construction is reversible and leads to a new equivalence relation between topological orders. we refer to the corresponding equivalent class ( the orbit of the hierarchy construction ) as " the non - abelian family ". each non - abelian family has one or a few root topological orders with the smallest number of anyon types. all the abelian topological orders belong to the trivial non - abelian family whose root is the trivial topological order. we show that abelian anyons in root topological orders must be bosons or fermions with trivial mutual statistics between them. the classification of topological orders is then greatly simplified, by focusing on the roots of each family : those roots are given by non - abelian modular extensions of representation categories of abelian groups.	`0.0`

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim"
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
num_train_epochs: 1
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss	sciq-eval_cosine_ndcg@10
0.0685	100	-	0.6007
0.1370	200	-	0.7026
0.2055	300	-	0.7167
0.2740	400	-	0.7195
0.3425	500	2.8082	0.7150
0.4110	600	-	0.7292
0.4795	700	-	0.7356
0.5479	800	-	0.7428
0.6164	900	-	0.7399
0.6849	1000	2.6228	0.7339
0.7534	1100	-	0.7356
0.8219	1200	-	0.7375
0.8904	1300	-	0.7385
0.9589	1400	-	0.7351
1.0	1460	-	0.7352

Framework Versions

Python: 3.12.8
Sentence Transformers: 3.4.1
Transformers: 4.51.3
PyTorch: 2.5.1+cu124
Accelerate: 1.3.0
Datasets: 3.2.0
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}