Pairwise learning for molecular property prediction and optimization

1 Introduction

Molecular optimization is inherently a comparative process, where decisions are guided by how structural changes alter critical molecular properties such as potency, selectivity, and pharmacokinetics. However, most machine learning approaches in cheminformatics are trained to predict absolute property values for individual molecules. Notwithstanding the impact these tools have had on modern drug discovery pipelines, an expanding body of literature suggests that methods explicitly designed to enable direct molecular comparisons can provide distinct advantages for guiding molecular optimization.

Several critical tools have been developed to more explicitly address molecular comparisons. For example, matched molecular pairs (MMP) analysis describes the systematic analysis of pairs of molecules with a small structural change to identify how chemical transformations impact molecular properties (Kenny and Sadowski, 2005). This highly interpretable approach can efficiently determine local structure–activity relationships by focusing on specific pairs of molecules with largely shared substructures (Griffen et al., 2011; Yang et al., 2023). However, this local focus limits its application for broader molecular comparisons. In molecular modeling, related ideas have been employed in the prediction of relative binding affinities (ΔΔG), where modeling differences between molecular states often yields more reliable predictions than absolute quantity modeling (Cournia et al., 2017). However, these approaches are most reliable within closely related congeneric ligand series and, because of high computational cost per comparison, are typically applied to a small, focused subset of candidates rather than broad chemical spaces. Therefore, these approaches collectively highlight the broader utility of learning from differences but employ a more restricted focus within these comparisons to the most chemically similar structures only.

Molecular pairwise learning addresses the limited scope of previous molecular comparison approaches by reformulating general molecular machine-learning tasks to operate on pairs of molecules and their property differences (Figure 1A) rather than on individual compounds and their absolute values (Figure 1B). In this framework, individual training datapoints are combined into tuples to form molecular pairs. The dependent variable associated with each molecular pair then becomes the difference in the original property values between the two molecules. Models are then trained to predict property differences between molecules, which can subsequently be aggregated to recover absolute property predictions or used directly for comparative tasks such as ranking or optimization. In contrast with related approaches such as Siamese Neural Networks and kernel methods relying on molecular comparisons to enable predictions (Jiménez-Luna et al., 2019; Fernández-Llaneza et al., 2021; McNutt and Koes, 2022; Zhang et al., 2023), pairwise learning does not assign constraints on input network weights or kernel functions or rely on distances, but instead directly learns to predict property differences.

In this review, we discuss recent developments in molecular pairwise learning, emphasizing how this framework improves predictive performance and enables uncertainty quantification by leveraging relative relationships between molecules (Figure 1C). We further highlight the ability of pairwise learning to directly model molecular differences for optimization tasks, augment data for deep learning in low-data regimes through combinatorial expansion of training data, and integrate diverse and heterogeneous data modalities that are difficult to incorporate using standard approaches. Taken together, we expect pairwise learning to become a cornerstone of molecular discovery and optimization, and hope that our review serves as a resource for the cheminformatics and drug discovery communities to better understand the advantages and future opportunities of this emerging approach.

2 Applications

2.1 Improving regression performance and uncertainty quantification

The model framework of pairwise learning was first proposed in the early 2020s (Tynes et al., 2021; Wetzel et al., 2022b). The pairwise difference regression (PADRE) approach reformulated the machine learning regression task by utilizing concatenated molecular feature vector pairs that concatenate the features of each individual molecule as well as a feature difference vector. These representations are then used as input to machine learning models to predict property differences between the paired molecules and subsequently convert these pairwise differences to absolute property values (Tynes et al., 2021). This conversion from property differences to absolute property values is achieved by averaging all property difference predictions into a single absolute property value. Similarly, the variance of all differences was implemented as a predictive uncertainty estimation. Across four datasets for metal-ligand complexes (consisting of computed binding energy, computed octanol-water partition coefficient, computed free energy of oxidation, and experimental binding affinity), this approach improved predictive performance of the employed random forest regressors compared to directly predicting property values in terms of reducing the mean absolute error (MAE) by a maximum of 33% and an average of 9%.

It was proposed this improvement was due to molecular differences being more representative of underlying chemical properties due to cancellation of systematic errors introduced by the theoretical or experimental method or underlying “true” signal (Tynes et al., 2021). Consistent with this hypothesis, pairing produced the largest improvement for the dataset derived using Density Functional Tight Binding (DFTB), a faster, but more approximate method with higher variability in accuracy. The experimental dataset, which is generally subject to higher measurement uncertainty than in silico datasets, showed greater improvement than all other computational datasets. Additionally, the uncertainty quantification arising from natural dispersion consistently improved over the base random forest’s uncertainty quantification as analyzed by Spearman’s ρ and confidence curves.

At the same time, Wetzel and colleagues developed the twin neural network approach that followed a similar approach but utilized neural network approaches (Wetzel et al., 2022b). Although so far only tested for standard, non-molecular benchmarking datasets, observed performance improvements were similar compared to the PADRE dataset and suggested that neural networks could be deployed successfully for molecular pairwise learning.

As an additional bonus, pairwise models can naturally encode invariants such as symmetry (A vs. B = –(B vs. A)), additivity (A vs. C = (A vs. B) + (B vs. C)), and identity (A vs. A = 0). Adherence to these invariants strongly correlates with model performance, providing an internal consistency check that allows researchers to assess reliability of a model without necessarily utilizing ground-truth labels (Fralish et al., 2023). This built-in mechanism for model reliability and domain-of-applicability estimation may increase transparency and strengthen confidence in model outputs to support decision making in medicinal chemistry. Additionally, invariants can be used for further model fine-tuning. For example, semi-supervised loops were added to pairwise model training to allow for ensembling of predicted differences between an unknown data point and all training data points (Wetzel et al., 2022a). These loops can incorporate unlabeled datapoints because the predicted differences should sum to zero irrespective of the actual datapoint label. The researchers showed that measuring adherence to this expected self-consistency can further improve predictive uncertainty estimates. Across nine tested datasets, this method showed improved regression performance compared to the non-looping implementation. It was suspected that these performance improvements are achieved by reducing the models’ bias-variance tradeoff through bagging-like internal ensembling.

2.2 Improving molecular property predictions for molecular differences

Beyond increasing predictive accuracy for predicting absolute molecular property values (Tynes et al., 2021; Wetzel et al., 2022b), pairwise learning can also be more directly used to predict molecular property differences (Fralish et al., 2023). This could potentially facilitate molecular comparisons and optimizations. To benchmark the application of pairwise learning for this task, 10 ADMET datasets, ranging from 253 to 1321 datapoints, were curated to assess performance on datasets that are often smaller, noisier, and derived from resource-intensive in vivo experimentation. Because in vivo datasets are often limited in size and experimentally expensive to generate, pairwise learning may be particularly well suited to these settings. By directly modeling relative in vivo differences between compounds and leveraging combinatorial expansion of training pairs, paired approaches can enhance data efficiency and improve predictive robustness in small-data regimes where conventional methods frequently struggle.

The paired version outcompeted the single-molecule implementation for 80% of datasets in terms of Pearson’s r and 70% for MAE. For the molecular pairs with the largest difference in property values, the paired approach exhibited the lowest error for all datasets for all models tested. Additionally, when capturing differences between molecules with divergent chemical scaffolds, the paired model outperformed all single-molecule models with a moderate but significant average improvement of 0.04 in terms of Pearson’s r and a small, but significant average improvement of 0.04 in terms of MAE. The paired models retained competitive performance for more fine-grained, within-scaffold optimizations. Based on these performance benefits, this work suggests that paired learning approaches might be particularly well suited for molecular optimization–although a prospective molecular optimization campaign has yet to be implemented using these tools to more directly validate this utility.

2.3 Pairwise learning in low-data regimes and active learning

One noteworthy capability of pairwise learning techniques is to combinatorially expand the training dataset, which makes them particularly suitable to enable training of machine learning models on smaller datasets of less than 1000 datapoints (Fralish and Reker, 2024; Lin et al., 2025; Nielsen et al., 2025). This makes them uniquely positioned for active learning workflows where models are built on very small datasets to support additional data acquisition. For example, the PADRE approach was applied within an active learning framework to augment a training dataset totaling 110 affinity evaluations for the transporter ABCG2 (Nielsen et al., 2025). Their models were initially trained on 10 random molecules and then selected 20 new molecules to be added for five iterations to explore 110 total molecules. The PADRE approach showed the greatest benefit for the multilayer perceptron, increasing the average hit rate from seven hits for the unpaired version to 30 for the paired version. Robustness was confirmed by noise tests and replication with multiple screening libraries, one containing drug-like structures spanning typical distributions of molecular properties (Enamine Discovery Diversity Set 10) and one containing synthetic and natural products identified as potential anticancer agents (Developmental Therapeutics Program repository).

Another example showed how the pairing approach can be used to significantly enhance the performance of the message passing neural network Chemprop during active learning (Fralish and Reker, 2024). Starting from a single training datapoint and adding one experiment at a time, this paired version of Chemprop started outperforming its single molecule counterpart at only 34 selected compounds on average across 99 K_i datasets with simulated time-splits. By 100 iterations, the paired version found 61% of the top ten percentile of most potent compounds compared to 45% by the unpaired version (+6.3 compounds on average). Additionally, the paired approach also favored chemically distinct scaffolds, retrospectively recovering a larger number of independent lead series for several projects (59% of all scaffolds for the paired implementation compared to 47% for the unpaired version). By enriching the scaffold richness of “negative” training data, it was hypothesized that this approach can also improve a model’s future selection ability by helping avoid premature convergence on narrow chemical spaces.

Pairwise learning was similarly applied for antibody design in low data regimes. The DyAb approach was developed to capture the nuances of protein sequence variation by learning on relative embeddings and property differences (Lin et al., 2025). This method transformed a starting affinity (pK_D) dataset of 500 datapoints into nearly 125,000 pairs of training data. Even when starting with only approximately 100 labeled training datapoints, prospective experimental validations showed most of the novel DyAb-generated sequences improved affinity beyond that of the most potent antibody in the training set. These impressive results, achieved with small datasets that are often considered insufficient for the training of complex machine learning models, highlight a particular opportunity for pairwise learning approaches to enhance model training in low-data regimes.

2.4 Integration of heterogeneous data

Beyond improving model performance within a single dataset by learning from property differences (Tynes et al., 2021), pairwise combination of data can also counteract systematic biases between datasets acquired from different sources (Nelen et al., 2025). For example, combining IC₅₀ or K_i measurements from different data sources often introduces substantial systematic biases making absolute values across datasets rarely comparable (Landrum and Riniker, 2024). However, potency differences between matched compound pairs within datasets can at least in-part normalize such systematic assay source differences (Nelen et al., 2025). Pairing may also improve model extrapolation to compounds with potencies beyond those observed in the training data (Wang and King, 2024). To test this hypothesis, support vector machines, random forests, and gradient boosting machines were trained to learn a bivariate function of drug activity differences and subsequently rank compounds. This paired approach showed improvements over standard regression for 1182/1436 (82%) of tested ChEMBL datasets when tasked with identifying new drug candidates with higher activity than the training data.

Pairwise learning approaches can also unlock valuable data that standard machine learning methods cannot use (Saar et al., 2023; Fralish et al., 2024), in addition to the unlabeled data as previously discussed (Wetzel et al., 2022a). For example, paired methods can be used to combine qualitative measurements with quantitative ones (Saar et al., 2023): when working to optimize protein-ligand affinity prediction during drug design, structural biology data is typically disconnected from predictive models for ligands. To combine these two types of information for model training, pairwise learning was used to extract physical descriptors of protein-ligand complexes and relate them to bioactivity. Specifically, a learning-to-rank approach was applied on physics-based energy descriptors from crystallographic characterizations of protein-ligand complexes to infer differences between binding modes. This approach was able to outperform ligand-based and structure-based methods on benchmarks of SARS-CoV-2 inhibition. During prospective deployment of the model to select 40 novel compounds, 30% had greater than 2-fold potency increases over the initial reference compound.

Another type of previously inaccessible data for regression models is bounded inhibition data (also known as censored data). These are measurements reported only as thresholds (e.g., “>10 µM”), which are common in early drug discovery screens performed at a single concentration but typically discarded when implementing regression models based on more rigorously characterized compounds with IC₅₀ or K_i values - leading to a substantial loss of information by excluding such thresholded datapoints. Through reframing the task as a ranking approach (Fralish et al., 2024), paired Chemprop and paired XGBoost models improved performance for identifying which of two compounds was more potent compared to three unpaired regression-based methods (random forest, XGBoost, and Chemprop). For example, the paired Chemprop was significantly better in at least 69% of 230 ChEMBL IC₅₀ datasets and was competitive in at least 96% of datasets compared to all unpaired methods. Given that pairing can partially normalize assay-specific differences (Nelen et al., 2025), this approach may have been particularly beneficial for the tested ChEMBL datasets compiled from heterogeneous experimental sources. The largest gains were seen with datasets containing substantial amounts of bounded data, highlighting the ability of paired models to learn from inexact measurements and their potential utility for targets with low druggability or novel target classes, where robustly characterized, potent compounds are often still scarce.

3 Challenges and future work

Although these pairwise learning techniques show promising performance, there is still much room for further work. The combinatorial nature of pairing inherently leads to exponential increases in computational expense. One strength and challenge of pairwise learning is the exhaustive generation of all pairs, which expands the training dataset but also leads to combinatorial dataset explosion which can make model training computationally expensive. Instead of pairing exhaustively, recent applications have been developed to optimize pairing strategies with the goal of improving computational efficiency as well as maintaining or even further improving model performance (Belaid et al., 2024; Zadorozhny et al., 2025).

For example, Similarity Quantized Relative Learning (SQRL) uses a similarity-thresholded framework to focus pairing only on the most chemically similar pairs of molecules (Zadorozhny et al., 2025). Several similarity metrics and thresholds were explored, and the final SQRL approach showed improvements across 30 activity prediction tasks from the MoleculeACE benchmark for every implemented neural network. However, none of the neural networks were able to outperform tree-based baseline models without SQRL pairing in the investigated benchmarks, signaling the need for further model tuning and data expansions to realize the benefits of neural networks for such tasks. Additionally, instead of changing the pairing during model training, the averaging of predicted differences for absolute property value inference has been adapted to a weighted average by learning the ideal contribution of each “anchor” training datapoint (Belaid et al., 2024). Various approaches to learning these weights were explored and deployed with various machine learning models across 231 OpenML datasets with <2000 datapoints with anchoring improving over baseline for the majority of datasets and tradeoffs arising between anchoring methods in terms of performance and speed.

Across an extensive set of 274 benchmark datasets including various molecular and non-molecular tasks aggregated from multiple publications (Tynes et al., 2021; Wetzel et al., 2022a; Belaid et al., 2024; Zadorozhny et al., 2025), paired approaches were shown to reduce error rates for the traditional regression task by up to 60% (Figure 2), although models ranked differently on different benchmarking datasets indicating the need for model fine-tuning for specific tasks. Combining the discussed subsampling strategies together could potentially leverage orthogonal signals and further reduce redundancy to further promote both efficient model training and broad chemical exploration.

Although pairwise learning approaches enable learning from all possible pairs of data, these approaches inherently lose some interpretability of established tools such as MMP and ΔΔG. Future efforts are needed to improve the explainability of the paired models to increase trust and encourage adoption by medicinal chemists. Additionally, user-friendly interfaces and designs that allow for direct incorporation of these models into existing platforms could facilitate model adoption. Finally, pairwise learning approaches have been primarily evaluated retrospectively so additional prospective tests will more clearly evaluate these models’ potential for real-world deployment in molecular discovery pipelines.

4 Discussion

Pairwise learning reformulates the traditional regression approach of training on individual molecules and their absolute property values into training on molecular pairs and the difference between their property values. Combinatorial data augmentation associated with pairing generates more training samples from small datasets, which can improve performance and accelerate convergence. By training on property differences, models can also directly learn to predict the effects of molecular changes (Fralish et al., 2023; Zadorozhny et al., 2025). This could directly support optimization tasks that involve choosing between alternatives allowing for more accurate compound rankings for prioritization and creating more interpretable predictions. The pairing of molecules during training also puts one molecule in the context of another molecule, allowing multiple types of data to be used during training (Saar et al., 2023; Fralish et al., 2024). Pairing also cancels systematic errors for datapoints derived from the same source (Tynes et al., 2021) and mitigates systematic biases when combining datasets from different sources (Nelen et al., 2025). As such, pairwise learning models may help scientists prioritize safer, more potent drugs with greater confidence.

We expect pairwise learning can be broadly deployed to improve molecular machine learning model prediction accuracy. Although pairing increases model complexity and computational cost, this can be addressed in part by smart pairing strategies that are currently being developed. That being said, if integrated in real world discovery pipelines, real-world experimentation often is the major bottleneck rather than computational cost during drug development. Therefore, the use of more complex computational approaches that show improved predictive power, such as these pairwise learning techniques, should typically be a better choice in place of faster, but less effective, approaches. Additionally, given its inherent data augmentation, pairwise learning holds particular promise for sparse and noisy datasets. We believe these easily deployable predictive tools and further extensions have the potential to supplement standard molecular machine learning approaches for intricate optimization and discovery tasks.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Alan Talevi, National University of La Plata, Argentina

Reviewed by: Alexander Kötter, Sanofi, Germany