canho/lateon-ar-step1-wave1-ckpt30

PyLate

This is a PyLate model trained. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.

Model Details

Model Description

• Model Type: PyLate model
• Document Length: 300 tokens
• Query Length: 64 tokens
• Output Dimensionality: 128 tokens
• Similarity Function: MaxSim

Model Sources

• Documentation: PyLate Documentation
• Repository: PyLate on GitHub
• Hugging Face: PyLate models on Hugging Face

Full Model Architecture

ColBERT(
  (0): Transformer({'max_seq_length': 63, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Dense({'in_features': 768, 'out_features': 1536, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': True})
  (2): Dense({'in_features': 1536, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': True})
  (3): Dense({'in_features': 768, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': False})
)

Usage

First install the PyLate library:

pip install -U pylate

Retrieval

Use this model with PyLate to index and retrieve documents. The index uses FastPLAID for efficient similarity search.

Indexing documents

Load the ColBERT model and initialize the PLAID index, then encode and index your documents:

from pylate import indexes, models, retrieve

# Step 1: Load the ColBERT model
model = models.ColBERT(
    model_name_or_path="pylate_model_id",
)

# Step 2: Initialize the PLAID index
index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
    override=True,  # This overwrites the existing index if any
)

# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]

documents_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
    show_progress_bar=True,
)

# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=documents_embeddings,
)

Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:

# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
)

Retrieving top-k documents for queries

Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:

# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)

# Step 2: Encode the queries
queries_embeddings = model.encode(
    ["query for document 3", "query for document 1"],
    batch_size=32,
    is_query=True,  #  # Ensure that it is set to False to indicate that these are queries
    show_progress_bar=True,
)

# Step 3: Retrieve top-k documents
scores = retriever.retrieve(
    queries_embeddings=queries_embeddings,
    k=10,  # Retrieve the top 10 matches for each query
)

Reranking

If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:

from pylate import rank, models

queries = [
    "query A",
    "query B",
]

documents = [
    ["document A", "document B"],
    ["document 1", "document C", "document B"],
]

documents_ids = [
    [1, 2],
    [1, 3, 2],
]

model = models.ColBERT(
    model_name_or_path="pylate_model_id",
)

queries_embeddings = model.encode(
    queries,
    is_query=True,
)

documents_embeddings = model.encode(
    documents,
    is_query=False,
)

reranked_documents = rank.rerank(
    documents_ids=documents_ids,
    queries_embeddings=queries_embeddings,
    documents_embeddings=documents_embeddings,
)

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,389 training samples
• Columns: query, positive, negative_0, negative_1, negative_2, negative_3, negative_4, negative_5, negative_6, negative_7, and negative_8
• Approximate statistics based on the first 1000 samples:

  	query	positive	negative_0	negative_1	negative_2	negative_3	negative_4	negative_5	negative_6	negative_7	negative_8
  type	string	string	string	string	string	string	string	string	string	string	string
  details	min: 12 tokens mean: 27.28 tokens max: 55 tokens	min: 43 tokens mean: 63.94 tokens max: 64 tokens	min: 13 tokens mean: 62.71 tokens max: 64 tokens	min: 7 tokens mean: 63.03 tokens max: 64 tokens	min: 14 tokens mean: 63.13 tokens max: 64 tokens	min: 10 tokens mean: 63.2 tokens max: 64 tokens	min: 10 tokens mean: 62.87 tokens max: 64 tokens	min: 7 tokens mean: 62.75 tokens max: 64 tokens	min: 13 tokens mean: 63.17 tokens max: 64 tokens	min: 7 tokens mean: 63.06 tokens max: 64 tokens	min: 9 tokens mean: 62.93 tokens max: 64 tokens

• Samples:

  query	positive	negative_0	negative_1	negative_2	negative_3	negative_4	negative_5	negative_6	negative_7	negative_8
  How to determine whether reduced glutamatergic signaling in the central amygdala causally contributes to withdrawal-related anxiety rather than merely accompanying it	Amygdala circuitry mediating reversible and bidirectional control of anxiety. Anxiety—a sustained state of heightened apprehension in the absence of immediate threat—becomes severely debilitating in disease states. Anxiety disorders represent the most common of psychiatric diseases (28% lifetime prevalence) and contribute to the aetiology of major depression and substance abuse. Although it has been proposed that the amygdala, a brain region important for emotional processing, has a role in anxiety, the neural mechanisms that control anxiety remain unclear. Here we explore the neural circuits underlying anxiety-related behaviours by using optogenetics with two-photon microscopy, anxiety assays in freely moving mice, and electrophysiology. With the capability of optogenetics to control not only cell types but also specific connections between cells, we observed that temporally precise optogenetic stimulation of basolateral amygdala (BLA) terminals in the central nucleus of the amygdala ...	Thalamic Glutamatergic Afferents into the Rat Basolateral Amygdala Exhibit Increased Presynaptic Glutamate Function Following Withdrawal from Chronic Intermittent Ethanol. Amygdala glutamatergic neurotransmission regulates withdrawal induced anxiety-like behaviors following chronic ethanol exposure. The lateral/basolateral amygdala receives multiple glutamatergic projections that contribute to overall amygdala function. Our lab has previously shown that rat cortical (external capsule) afferents express postsynaptic alterations during chronic intermittent ethanol exposure and withdrawal. However, thalamic (internal capsule) afferents also provide crucial glutamatergic input during behavioral conditioning, and they have not been studied in the context of chronic drug exposure. We report here that these thalamic inputs express altered presynaptic function during withdrawal from chronic ethanol exposure. This is characterized by enhanced release probability, as exemplified by altered paire...	Pharmacology of 5-HT2 Modulation of Amygdala & Hypothalamus in Anxiety Disorders. “Anxiety disorders” is a blanket term covering several different forms of abnormal and pathological fear and anxiety, and is often comorbid with other mental disorders, particularly clinical depression. These conditions are often related to stressful life experiences, especially when chronic and traumatic. Stress appears to act as a predisposing and precipitating factor in these psychiatric conditions (Strohle and Holsboer, 2003). One particular, extreme case is post traumatic stress disorder (PTSD), a chronic anxiety disorder developed in the aftermath of traumatic stress exposure and persisting long after the removal of the participating stressors. Advances in cellular and molecular biology and imaging technology have opened several lines of inquiry into the pathogenesis and pharmacotherapy of the anxiety disorders. Dysregulation of neurotransmitter systems, alteration of signal transduction pathways, a...	Chronic Intermittent Ethanol and Withdrawal Differentially Modulate Basolateral Amygdala AMPA-type Glutamate Receptor Function and Trafficking. The amygdala plays a critical role in the generation and expression of anxiety-like behaviors including those expressed following withdrawal (WD) from chronic intermittent ethanol (CIE) exposure. In particular, the BLA glutamatergic system controls the expression of both innate and pathological anxiety. Recent data suggests that CIE and WD may functionally alter this system in a manner that closely parallels memory-related phenomena like long term potentiation (LTP). We therefore specifically dissected CIE/WD-induced changes in glutamatergic signaling using electrophysiological and biochemical approaches with a particular focus on the plasticity-related components of this neurotransmitter system. Our results indicate that cortical glutamatergic inputs arriving at BLA principal via the external capsule undergo predominantly post-synaptic alterat...	Adolescent Alcohol Exposure Persistently Impacts Adult Neurobiology and Behavior. Adolescence is a developmental period when physical and cognitive abilities are optimized, when social skills are consolidated, and when sexuality, adolescent behaviors, and frontal cortical functions mature to adult levels. Adolescents also have unique responses to alcohol compared with adults, being less sensitive to ethanol sedative–motor responses that most likely contribute to binge drinking and blackouts. Population studies find that an early age of drinking onset correlates with increased lifetime risks for the development of alcohol dependence, violence, and injuries. Brain synapses, myelination, and neural circuits mature in adolescence to adult levels in parallel with increased reflection on the consequence of actions and reduced impulsivity and thrill seeking. Alcohol binge drinking could alter human development, but variations in genetics, peer groups, family structure, early life experiences,...	Glutamate Plasticity in the Drunken Amygdala: the Making of an Anxious Synapse. Plasticity at glutamatergic synapses is believed to be the cellular correlate of learning and memory. Classic fear conditioning, for example, is dependent upon NMDA-type glutamate receptor activation in the lateral/basolateral amygdala followed by increased synaptic expression of AMPA-type glutamate receptors. This review provides an extensive comparison between the initiation and expression of glutamatergic plasticity during learning/memory and glutamatergic alterations associated with chronic ethanol exposure and withdrawal. The parallels between these neuro-adaptive processes suggest that long-term ethanol exposure might "chemically condition" amygdala-dependent fear/anxiety via the increased function of pre- and post-synaptic glutamate signaling.	Resolving the neural circuits of anxiety. Although anxiety disorders represent a major societal problem demanding new therapeutic targets, these efforts have languished in the absence of a mechanistic understanding of this subjective emotional state. While it is impossible to know with certainty the subjective experience of a rodent, rodent models hold promise in dissecting well-conserved limbic circuits. The application of modern approaches in neuroscience has already begun to unmask the neural circuit intricacies underlying anxiety by allowing direct examination of hypotheses drawn from existing psychological concepts. This information points toward an updated conceptual model for what neural circuit perturbations could give rise to pathological anxiety and thereby provides a roadmap for future therapeutic development.	The Central Amygdala as an Integrative Hub for Anxiety and Alcohol Use Disorders. The central amygdala (CeA) plays a central role in physiological and behavioral responses to fearful stimuli, stressful stimuli, and drug-related stimuli. The CeA receives dense inputs from cortical regions, is the major output region of the amygdala, is primarily GABAergic (inhibitory), and expresses high levels of pro- and anti-stress peptides. The CeA is also a constituent region of a conceptual macrostructure called the extended amygdala that is recruited during the transition to alcohol dependence. In this review, we discuss neurotransmission in the CeA as a potential integrative hub between anxiety disorders and Alcohol Use Disorder (AUD), which are commonly co-occurring in humans. Human imaging work and multi-disciplinary work in animals collectively suggest that CeA structure and function are altered in individuals with anxiety disorders and AUD, the end result of which may be disinhibition of dow...	Maturation of the adolescent brain. Adolescence is the developmental epoch during which children become adults – intellectually, physically, hormonally, and socially. Adolescence is a tumultuous time, full of changes and transformations. The pubertal transition to adulthood involves both gonadal and behavioral maturation. Magnetic resonance imaging studies have discovered that myelinogenesis, required for proper insulation and efficient neurocybernetics, continues from childhood and the brain’s region-specific neurocircuitry remains structurally and functionally vulnerable to impulsive sex, food, and sleep habits. The maturation of the adolescent brain is also influenced by heredity, environment, and sex hormones (estrogen, progesterone, and testosterone), which play a crucial role in myelination. Furthermore, glutamatergic neurotransmission predominates, whereas gamma-aminobutyric acid neurotransmission remains under construction, and this might be responsible for immature and impulsi...	Mechanisms and Mediators of the Relationship between Anxiety Disorders and Alcohol Use Disorders: Focus on Amygdalar NPY. High rates of co morbidity for Alcohol Use Disorders (AUDs) and anxiety disorders suggest a causative relationship between these disorders, as well as overlapping neurobiological mechanisms. While it is well established that alcohol withdrawal can precipitate and exacerbate the expression of anxiety, the extent to which pre-existing anxiety disorders contribute to the development of AUDs is less clear. Anxiety relief is commonly cited as a motivation to consume alcohol and recent preclinical studies focusing on the relationship between innate anxiety phenotypes and alcohol– related behaviors support the notion that elevated anxiety may contribute to increased alcohol consumption. However, the endogenous neural mechanisms that mediate this relationship have yet to be fully defined. This review focuses on the relationship between anxiety-related responses and acute al...
  How to explain lower oxygen isotope discrimination in one leaf group than another without reduced carbonyl sulfide uptake, when differences in shared enzyme activity alone are insufficient	The Role of Phosphoenolpyruvate Carboxylase during C4 Photosynthetic Isotope Exchange and Stomatal Conductance1[OA]. Phosphoenolpyruvate carboxylase (PEPC; EC 4.1.1.31) plays a key role during C4 photosynthesis and is involved in anaplerotic metabolism, pH regulation, and stomatal opening. Heterozygous (Pp) and homozygous (pp) forms of a PEPC-deficient mutant of the C4 dicot Amaranthus edulis were used to study the effect of reduced PEPC activity on CO2 assimilation rates, stomatal conductance, and 13CO2 (Δ13C) and C18OO (Δ18O) isotope discrimination during leaf gas exchange. PEPC activity was reduced to 42% and 3% and the rates of CO2 assimilation in air dropped to 78% and 10% of the wild-type values in the Pp and pp mutants, respectively. Stomatal conductance in air (531 μbar CO2) was similar in the wild-type and Pp mutant but the pp mutant had only 41% of the wild-type steady-state conductance under white light and the stomata opened more slowly in response to increased light or red...	Oxygen exchange in relation to carbon assimilation in water-stressed leaves during photosynthesis.. In a study on metabolic consumption of photosynthetic electrons and dissipation of excess light energy under water stress, O2 and CO2 gas exchange was measured by mass spectrometry in tomato plants using 18O2 and 13CO2. Under water stress, gross O2 evolution (E(O)), gross O2 uptake (U(O)), net CO2 uptake (PN), gross CO2 uptake (TPS), and gross CO2 evolution (Ec) declined. The ratio P(N)/E(O) fell during stress, while the ratios U(O)/E(O) and E(C)/TPS rose. Mitochondrial respiration in the light, which can be measured directly by 12CO2 evolution during 13CO2 uptake at 3000 microl l(-1) 13CO2, is small in relation to gross CO2 evolution and CO2 release from the glycolate pathway. It is concluded that PSII, the Calvin cycle and mitochondrial respiration are down-regulated under water stress. The percentages of photosynthetic electrons dissipated by CO2 assimilation, photorespiration and the...	Organic hydrogen-carbon isotope signatures of terrestrial higher plants during biosynthesis for distinctive photosynthetic pathways. Stable isotopic compositions of organic hydrogen (δD) have positive correlations (r2 > 0.95) with those of carbon (δ13C) among several compound fractions of terrestrial plant leaves possessing distinctive photosynthetic pathways (C3, C4 and CAM). The δD/δ13C slopes of C3 plants vary from ∼25 to 42, which are larger than those of C4 plants (11 to 12). CAM plants have intermediate δD/δ13C slopes (∼17 to 20) between C3 and C4 plants. Using a δD-δ13C diagram, photosynthetic metabolisms are clearly discriminated, even though they sometimes cannot be distinguished from each other only by carbon isotopes. Relative to bulk organic matter, hydrogen and carbon of lipid fraction are more depleted in 13C than those of pigment fraction, respectively. Furthermore, δD values of lipid and pigment fractions relative to bulk organic hydrogen have negative correlations with...	Factors affecting the relationship between carbon isotope discrimination and transpiration efficiency in blue oak (Quercus douglasii). In a greenhouse study, variation in plant and leaf transpiration efficiency(W, A/E)and carbon isotope discrimination (D) were assessed forQuercus douglasii Hook & Arn. (blue oak) from a‘wet’ (930 mm) and a ‘dry’ (500 mm) site. Plants weregrown at 75–100% (wet) and 50–75% (dry) of fieldcapacity. Family variation masked population level differences, and twocontrasting patterns emerged. The expected pattern was observed with drytreatment plants having lower leaf internal CO 2concentration (c i ) and D, andhigher W andA/E. For families with verylarge increases in plant size from dry to wet treatments, however, wettreatment plants had lower c i andD, and higher W andA/E, reflecting a greaterinfluence of the maximum assimilation rate (Amax ) compared with stomatal conductance(g s ) on ci . In addition, large within-population variation inplant size appears to...	Activities of the cyanide-resistant respiratory pathway in leaves of sun and shade species. Higher plant mitochondria have a cyanide-resistant alternative respiratory pathway of electron transport (AP) that is not coupled to proton translocation. To characterise the ecophysiology of this apparently ‘wasteful’ pathway, we constructed a system consisting of a gas-phase oxygen electrode and an air sampling line for measurement of stable oxygen isotope ratios. With this system, we were able to measure respiratory rates of a small amount of leaf segments of ca 0.6 g fresh weight, and collect about 100 L of the air from the oxygen electrode chamber several times. The 18 O/16 O ratio in the air samples was measured by mass spectrometry. The activity of AP was estimated based on the isotopic discrimination of 18 O. We used the leaves of Alocasia odora (Lodd.) Spach., a shade species, and Spinacia oleracea L. and Phaseolus vulgaris L., sun species. These plants were grown at two levels of photo...	Differences in Carbon Isotope Discrimination of Three Variants of D-Ribulose-1,5-bisphosphate Carboxylase/Oxygenase Reflect Differences in Their Catalytic Mechanisms*♦. The carboxylation kinetic (stable carbon) isotope effect was measured for purified d-ribulose-1,5-bisphosphate carboxylases/oxygenases (Rubiscos) with aqueous CO2 as substrate by monitoring Rayleigh fractionation using membrane inlet mass spectrometry. This resulted in discriminations (Δ) of 27.4 ± 0.9‰ for wild-type tobacco Rubisco, 22.2 ± 2.1‰ for Rhodospirillum rubrum Rubisco, and 11.2 ± 1.6‰ for a large subunit mutant of tobacco Rubisco in which Leu335 is mutated to valine (L335V). These Δ values are consistent with the photosynthetic discrimination determined for wild-type tobacco and transplastomic tobacco lines that exclusively produce R. rubrum or L335V Rubisco. The Δ values are indicative of the potential evolutionary variability of Δ values for a range of Rubiscos from different species: Form I Rubisco from hi...	Bio-synthetic origin of oxygen in the leaf xanthophylls.. Isotope enrichment was studied in the xanthophylls from Chlorella vulgaris grown in the dark and in the presence of either molecular oxygen or water enriched with O18. All three leaf xanthophylls—lutein, violaxanthin, and neoxanthin—incorporate oxygen from the atmosphere. Oxygen from water was incorporated by neoxanthin but not detectably by lutein. The pattern of incorporation suggests that O2 is utilized for the hydroxyl groups and H2O for the epoxides.	The relationship of glycolic acid to respiration and photosynthesis in tobacco leaves.. Studies with glycolic acid oxidase and glyoxylic acid reductase isolated from green leaves have led to the suggestion that these enzymes play a part in the oxygen uptake of leaf tissue (l-3). Glycolic acid oxidase is a flavoprotein which is capable of directly oxidizing glycolic acid, a substance rapidly produced during photosynthesis (4, 5). As has been demonstrated in model systems, the oxidase can also couple with glyoxylic acid reductase to oxidize reduced pyridine nucleotides	Mapping of QTLs Controlling Carbon Isotope Discrimination in the Photosynthetic System using Recombinant Inbred Lines Derived from a Cross between Two Different Rice(Oryza sativa L.) Cultivars. Abstract Carbon isotope discrimination (Ċ) occuring in the process of photosynthesis, shows variation among rice (Oryza sativa L.) cultivars. Elucidation of specific traits associated with the extent of this discrimination under irrigated conditions may be useful to improve photosynthetic ability in rice plants. We measured leaf photosynthesis and Ċ in Milyang 23 and Akihikari, and conducted quantitative trait loci (QTL) analysis on Ċ at heading stage using a population of 126 recombinant inbred lines (RILs), derived from a cross between the two cultivars. While the two parental cultivars showed a similar Ċ, the RILs showed a wide variation in Ċ including transgressive segregation. Seven QTLs were detected for Ċ; four on chromosomes 2 (two regions), 7, and 11 were those for Ċ that is increased b...	Difference in Single Leaf Photosynthesis between Old and New Rice Varieties : I. Single-leaf photosynthesis and its dependence on stomatal conductance. Old (resistered before 1945) and new (resistered after 1950) rice varieties were compared about single-leaf photosynthesis (CER) and stomatal conductance (gs). (1) At the heading stage, no or little difference in CER was observed between the old and new variety groups. However, at the ripening stage, CER was clearly higher in new verieties. The new/old ratio in CER was 124.4% in the average for three years. (2) gs was higher in the new variety groups than that in the old one at the ripening stage. (3) In most cases when CER and gs were measured, correlation coefficients more than +0.8 were obtained between the two, suggesting the varietal difference in CER was mainly caused by that in gs. However, the ΔCER/Δgs ratio in the ambient air humidifying treatment (ΔCER and Δgs are incremcnts in CER and gs respectively caused by the treatment) ...
  How to determine causal direction from changing distributions when multiple changing influences on observed variables have potentially dependent parameters	Quantifying causal influences. Many methods for causal inference generate directed acyclic graphs (DAGs) that formalize causal relations between $n$ variables. Given the joint distribution on all these variables, the DAG contains all information about how intervening on one variable changes the distribution of the other $n-1$ variables. However, quantifying the causal influence of one variable on another one remains a nontrivial question. Here we propose a set of natural, intuitive postulates that a measure of causal strength should satisfy. We then introduce a communication scenario, where edges in a DAG play the role of channels that can be locally corrupted by interventions. Causal strength is then the relative entropy distance between the old and the new distribution. Many other measures of causal strength have been proposed, including average causal effect, transfer entropy, directed information, and information flow. We explain how they fail to satisfy the postulates on simple DA...	Causal Discovery and Inference: Concepts and Recent Methodological Advances. This paper aims to give a broad coverage of central concepts and principles involved in automated causal inference and emerging approaches to causal discovery from i.i.d data and from time series. After reviewing concepts including manipulations, causal models, sample predictive modeling, causal predictive modeling, and structural equation models, we present the constraint-based approach to causal discovery, which relies on the conditional independence relationships in the data, and discuss the assumptions underlying its validity. We then focus on causal discovery based on structural equations models, in which a key issue is the identifiability of the causal structure implied by appropriately defined structural equation models: in the two-variable case, under what conditions (and why) is the causal direction between the two variables identifiable? We show that the independence between the error term and causes...	Inferring Causal Directions in Errors-in-Variables Models. Inferring the causal direction between two variables is a nontrivial problem in the subject of causal discovery from observed data. A method for errors-in-variables models where both the cause variable and the effect variable are observed with measurement errors is presented in this paper.	Causal Analysis With Panel Data. Introduction Modeling Change with Panel Data Models with Reciprocal Causation Measurement Error Models Models of Spurious Association Concluding Note on Causal Inference in Panel Analysis	Causality and Statistical Learning1. We review some approaches and philosophies of causal inference coming from sociology, economics, computer science, cognitive science, and statistics	[Strengthening causal inference in studies using non-experimental data: an application of propensity score and instrumental variable methods].. OBJECTIVES This study attempts to show how studies using non-experimental data can strengthen causal inferences by applying propensity score and instrumental variable methods based on the counterfactual framework. For illustrative purposes, we examine the effect of having private health insurance on the probability of experiencing at least one hospital admission in the previous year. METHODS Using data from the 4th wave of the Korea Labor and Income Panel Study, we compared the results obtained using propensity score and instrumental variable methods with those from conventional logistic and linear regression models, respectively. RESULTS While conventional multiple regression analyses fail to identify the effect, the results estimated using propensity score and instrumental variable methods suggest that having private health insurance has posi...	Mediation Analysis with Multiple Mediators. Abstract Recent advances in the causal inference literature on mediation have extended traditional approaches to direct and indirect effects to settings that allow for interactions and non-linearities. In this article, these approaches from causal inference are further extended to settings in which multiple mediators may be of interest. Two analytic approaches, one based on regression and one based on weighting are proposed to estimate the effect mediated through multiple mediators and the effects through other pathways. The approaches proposed here accommodate exposure–mediator interactions and, to a certain extent, mediator–mediator interactions as well. The methods handle binary or continuous mediators and binary, continuous or count outcomes. When the mediators affect one another, the strategy of trying to assess direct and indirect effects one mediator at a time will in general fail; the approach given in this article can still be used. ...	Dynamic modeling of neuronal responses in fMRI using cubature Kalman filtering. This paper presents a new approach to inverting (fitting) models of coupled dynamical systems based on state-of-the-art (cubature) Kalman filtering. Crucially, this inversion furnishes posterior estimates of both the hidden states and parameters of a system, including any unknown exogenous input. Because the underlying generative model is formulated in continuous time (with a discrete observation process) it can be applied to a wide variety of models specified with either ordinary or stochastic differential equations. These are an important class of models that are particularly appropriate for biological time-series, where the underlying system is specified in terms of kinetics or dynamics (i.e., dynamic causal models). We provide comparative evaluations with generalized Bayesian filtering (dynamic expectation maximization) and demonstrate marked improvements in accuracy and computational efficiency. We com...	Bayesian Online Changepoint Detection. Changepoints are abrupt variations in the generative parameters of a data sequence. Online detection of changepoints is useful in modelling and prediction of time series in application areas such as finance, biometrics, and robotics. While frequentist methods have yielded online filtering and prediction techniques, most Bayesian papers have focused on the retrospective segmentation problem. Here we examine the case where the model parameters before and after the changepoint are independent and we derive an online algorithm for exact inference of the most recent changepoint. We compute the probability distribution of the length of the current ``run,'' or time since the last changepoint, using a simple message-passing algorithm. Our implementation is highly modular so that the algorithm may be applied to a variety of types of data. We illustrate this modularity by demonstrating the algorithm on three different real-world data sets.	GRANGER CAUSALITY AND STRUCTURAL CAUSALITY IN CROSS-SECTION AND PANEL DATA. Granger noncausality in distribution is fundamentally a probabilistic conditional independence notion that can be applied not only to time series data but also to cross-section and panel data. In this paper, we provide a natural definition of structural causality in cross-section and panel data and forge a direct link between Granger (G–) causality and structural causality under a key conditional exogeneity assumption. To put it simply, when structural effects are well defined and identifiable, G–non-causality follows from structural noncausality, and with suitable conditions (e.g., separability or monotonicity), structural causality also implies G–causality. This justifies using tests of G–non-causality to test for structural noncausality under the key conditional exogeneity assumption for both cross-section and panel data. We pay special attention to heterogeneous populations, allowing both structural heterog...

• Loss: pylate.losses.contrastive.Contrastive

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 128
• learning_rate: 3e-06
• bf16: True
• eval_strategy: steps
• per_device_eval_batch_size: 128
• accelerator_config: {'split_batches': True, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• dataloader_num_workers: 8

All Hyperparameters

Click to expand

• per_device_train_batch_size: 128
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 3e-06
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: trackio
• eval_strategy: steps
• per_device_eval_batch_size: 128
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': True, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: True
• dataloader_num_workers: 8
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.1	1	7.9421
0.2	2	7.4669
0.3	3	7.7694
0.4	4	7.0725
0.5	5	7.5771
0.6	6	6.3384
0.7	7	8.0260
0.8	8	6.9247
0.9	9	6.6932
1.0	10	6.1268
1.1	11	6.3718
1.2	12	6.8566
1.3	13	6.5764
1.4	14	5.9388
1.5	15	5.6536
1.6	16	5.9714
1.7	17	6.3865
1.8	18	5.5237
1.9	19	6.4439
2.0	20	6.2620
2.1	21	7.4771
2.2	22	5.7841
2.3	23	5.9513
2.4	24	6.6038
2.5	25	6.0526
2.6	26	5.3472
2.7	27	5.6740
2.8	28	5.5965
2.9	29	5.4740
3.0	30	5.7496

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.3.0
• PyLate: 1.6.0
• Transformers: 5.3.0
• PyTorch: 2.11.0+cu128
• Accelerate: 1.15.0
• Datasets: 3.6.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084"
}

PyLate

@inproceedings{DBLP:conf/cikm/ChaffinS25,
  author       = {Antoine Chaffin and
                  Rapha{"{e}}l Sourty},
  editor       = {Meeyoung Cha and
                  Chanyoung Park and
                  Noseong Park and
                  Carl Yang and
                  Senjuti Basu Roy and
                  Jessie Li and
                  Jaap Kamps and
                  Kijung Shin and
                  Bryan Hooi and
                  Lifang He},
  title        = {PyLate: Flexible Training and Retrieval for Late Interaction Models},
  booktitle    = {Proceedings of the 34th {ACM} International Conference on Information
                  and Knowledge Management, {CIKM} 2025, Seoul, Republic of Korea, November
                  10-14, 2025},
  pages        = {6334--6339},
  publisher    = {{ACM}},
  year         = {2025},
  url          = {https://github.com/lightonai/pylate},
  doi          = {10.1145/3746252.3761608},
}