Skip to content
Frontiers
Search

Perspective

Frontiers in Bioinformatics ·

Volume 6 - 2026 | https://doi.org/10.3389/fbinf.2026.1925740

Architectural good practices for reproducible benchmarking in protein machine learning

  • Julián García-Vinuesa1
  • Diego Fernández-Villegas1
  • Michelle Soto-García1
  • Xavier Cadet2
  • Mehdi D. Davari3
  • Frederic Cadet4
  • Juan A. Asenjo5
  • Roberto Uribe-Paredes1*corresponding author
  • David Medina-Ortiz1,3*corresponding author
  1. 1Departamento de Ingeniería En Computación, Universidad de Magallanes, Punta Arenas, Chile
  2. 2Dartmouth College, Hanover, NH, United States
  3. 3Department of Bioorganic Chemistry, Leibniz Institute of Plant Biochemistry, Halle, Germany
  4. 4AI for Biologics, PEACCEL, Paris, France
  5. 5Departamento de Ingeniería Química, Biotecnología y Materiales, Universidad de Chile, Santiago, Chile
Article metrics0 viewsView details
PDFCiteMetrics

Abstract

Reliable benchmarking in protein machine learning requires biological tasks, datasets, representations, execution conditions, and evaluation regimes to be explicitly defined and traceable. Protein-specific dependencies, including label semantics, negative-class construction, evolutionary relationships, similarity control, inferred structures, protein language model configurations, and potential pretraining contamination, can alter benchmark interpretation. This Perspective provides an operational, protein-specific synthesis of established reproducibility practices through an artefact-linked architecture. We define five core requirements for verifiable benchmarking, including deterministic curation, persistent and versioned artefacts, explicit interface contracts, reproducible execution, and transparent evaluation. Prediction provenance, uncertainty, calibration, and explainability are treated as an additional decision-readiness layer. These requirements are translated into minimum records, provenance links, and verification checks. An antimicrobial peptide classification case study demonstrates their application to alternative task formulations, negative-class definitions, dataset compositions, and partitioning strategies within a controlled workflow.

Keywords

1 Introduction

Machine learning is now central to protein science, supporting structure prediction, variant effect modelling, functional annotation, and de novo protein design (Kim et al., 2025; Sun and Shen, 2025; Jendrusch et al., 2025). Advances in biological data resources, representation learning, geometric modelling, and protein foundation models have expanded the range of computationally accessible tasks (Weissenow and Rost, 2025; García-Vinuesa et al., 2025; Lin et al., 2025; Consortium, 2025). [However, their reliability depends not only on predictive architectures, but also on how biological data and computational artefacts are curated, transformed, partitioned, executed, and evaluated (Kapoor and Narayanan, 2023). Studies addressing similar tasks may therefore differ in dataset construction, redundancy control, representation extraction, model selection, and evaluation protocols] (Martinez et al., 2025; Kucera et al., 2023; Kouba et al., 2023). When these [decisions are undocumented or distributed across scripts, configuration files, and software environments, results become difficult to reproduce, compare, audit, or trace to their provenance (Antunes and Hill, 2024; Gundersen et al., 2022; Leipzig et al., 2021; Pimentel et al., 2019; Kapoor and Narayanan, 2023).

These dependencies are particularly consequential in protein machine learning because protein sequences are evolutionarily related and many workflow inputs are computationally derived. Predicted structures, similarity control, functional annotations, and representation settings (Abramson et al., 2024; Ferrer Florensa et al., 2024; Char et al., 2025; Lin et al., 2023) can alter the effective prediction problem, the independence of evaluation data, and the biological interpretation of reported performance (Kapoor and Narayanan, 2023; Herrera-Rocha et al., 2025; Bernett et al., 2024).

The consequences are especially visible in benchmarking, where meaningful comparison requires more than common datasets and metrics (Zhang et al., 2026; Semmelrock et al., 2025). Resources such as EnzEngDB (L et al., 2026), CombinGym (Chen et al., 2026), ProteinGym (Notin et al., 2023), and FLIP (Dallago et al., 2021) provide curated tasks and evaluation settings, while EnzymeML (Lauterbach et al., 2023), Codabench (Xu et al., 2022), OpenML (Feurer et al., 2021), Snakemake (Köster and Rahmann, 2012), Nextflow (Di Tommaso et al., 2017), and related initiatives (Fursin et al., 2020; Swainston et al., 2018; Amstutz et al., 2016) support structured reporting, portable execution, and reproducible evaluation. Nevertheless, these resources generally address complementary parts of the workflow, while dependencies between task definition, curation, partitioning, representation generation, execution, and evaluation frequently remain fragmented (Herrera-Rocha et al., 2025; Cheng et al., 2026; Graber et al., 2025; Heil et al., 2021). The remaining gap is an integrated way to make these protein-specific dependencies explicit and verifiable across the workflow.

This Perspective provides an operational, protein-specific synthesis of established reproducibility practices (Wilkinson et al., 2016) through an artefact-linked architecture connecting data provenance (Schlegel and Sattler, 2025), similarity-aware generalisation (Ferrer Florensa et al., 2024), representation extraction, controlled execution, and benchmark interpretation (Herrera-Rocha et al., 2025; Semmelrock et al., 2025). We organise the recommendations around five core requirements, including deterministic curation, persistent and versioned artefacts, explicit interface contracts, reproducible execution, and transparent evaluation. Uncertainty, calibration, explainability, and prediction provenance are treated as an additional decision-readiness layer for workflows guiding biological or experimental actions. A demonstrative antimicrobial peptide classification case study illustrates how task definition, negative-class construction, dataset composition, and partitioning can reshape apparent performance under controlled representation, model, and evaluation settings. By treating dataset semantics and workflow assumptions as part of the scientific result, this Perspective aims to support more reproducible, comparable, and biologically interpretable protein machine learning benchmarks.

2 The infrastructure bottleneck in protein machine learning

The infrastructure bottleneck lies in preserving the biological meaning, configuration, and provenance of workflow objects as they move between stages. When these relationships are implicit, apparently comparable analyses may no longer represent the same task, inputs, or evaluation regime (Herrera-Rocha et al., 2025; Draizen et al., 2024; Kapoor and Narayanan, 2023; Bernett et al., 2024).

A protein dataset is not fully specified by sequences and labels alone. Database releases, retrieval queries, inclusion criteria, annotation rules, duplicate handling, label harmonisation, and ambiguous or missing annotations determine the effective prediction task (Consortium, 2025; Pop et al., 2025; Helmy et al., 2016; Youngs et al., 2014). Negative-class construction is particularly consequential because experimentally supported negatives, unannotated sequences, background proteins, and artificial decoys encode different biological questions and may introduce distinct compositional or physicochemical biases.

Evolutionary relationships add another constraint because protein sequences are not independent observations. Homologous or near-duplicate sequences shared across development and evaluation sets can inflate performance (Ferrer Florensa et al., 2024; Chen et al., 2021; Bushuiev et al., 2024), while similarity thresholds, clustering rules, and partitioning strategies determine the generalisation regime being tested (Chiu and Ong, 2022; Bonin et al., 2023; Petti and Eddy, 2022; Joeres et al., 2025). Random, redundancy-controlled, family-level, temporal, and structure-aware partitions probe distinct regimes and should therefore not be treated as interchangeable benchmark settings (Petti and Eddy, 2022; Ferrer Florensa et al., 2024; Zhou et al., 2019; Aguilera-Puga and Plisson, 2024).

Workflow inputs may also be computationally derived. Homology searches and alignments depend on database releases, software, thresholds, and parameters (Consortium, 2025; Chiu and Ong, 2022; Wei et al., 2023); structure-based workflows inherit the provenance and limitations of experimental or predicted structures (Abramson et al., 2024; Terwilliger et al., 2024); and protein language-model representations depend on checkpoint, layer selection, tokenisation, pooling, and preprocessing (Li et al., 2022; Dota et al., 2024; Hoang and Singh, 2025; Li et al., 2024). Pretraining contamination should also be considered because evaluation sequences or close homologues may have been encountered during foundation-model pretraining even when supervised partitions do not overlap (Hermann et al., 2024).

Even with a fixed biological task, workflow behaviour can change through dependency resolution, numerical precision, hardware, random seeds, nondeterministic operations, or incompatible artefact versions (Gundersen et al., 2022; Leipzig et al., 2021). Outputs from different tools are also not automatically interoperable: an artefact must preserve its meaning, configuration, version, and provenance, not merely use a readable format (Khan et al., 2019). Workflows assembled from scripts, notebooks, local files, and tool-specific configurations may therefore function within one laboratory while lacking the records required for independent inspection and reuse (Pimentel et al., 2019; Heil et al., 2021; Khan et al., 2019; Xiang et al., 2025; Fernández-Díaz et al., 2024).

Here, repeatability refers to rerunning an analysis under equivalent conditions, whereas reproducibility refers to obtaining consistent results through an independent execution using the reported materials and procedures. Auditability requires reconstructing the inputs, transformations, configurations, and executions that produced a result, while verifiability additionally requires explicit checks of artefact integrity, compatibility, provenance, partition constraints, and test isolation. Comparability depends on aligned task definitions, generalisation regimes, and evaluation protocols; interoperability requires interfaces that preserve artefact meaning and provenance; and benchmarkability requires evaluation under a documented and biologically interpretable generalisation regime.

These dependencies cannot be resolved through algorithmic innovation alone. They require explicit records, provenance links, compatibility rules, and validation procedures that remain stable across tools and environments (Graber et al., 2025; Heil et al., 2021; Kapoor and Narayanan, 2023; Bernett et al., 2024; Wilkinson et al., 2016). The following section translates these needs into operational requirements for verifiable protein machine-learning workflows.

3 Principles for verifiable protein machine learning

Building on established guidance for methodological reporting, FAIR data practices, controlled execution, and transparent benchmark design, we translate these practices into an operational architecture for protein machine learning] (Herrera-Rocha et al., 2025; Bernett et al., 2024; Heil et al., 2021; Wilkinson et al., 2016; Longjohn et al., 2024; Petti and Eddy, 2022). Within this architecture, verifiability requires each workflow object to retain sufficient records, provenance relationships, and checks to establish its integrity, compatibility, and declared methodological properties. We distinguish five core requirements for reproducible benchmarking and an additional decision-readiness layer for workflows whose outputs guide biological or experimental actions (Figure 1). Their operational implementation is summarised in Table 1, while Section S1 of the Supplementary Material provides detailed requirements and a minimum compliance checklist.

Architectural requirements for verifiable protein machine learning. Five core requirements form the reproducibility architecture: deterministic curation, persistent artefacts, explicit interface contracts, reproducible execution, and transparent evaluation. A decision-readiness layer extends this architecture when workflow outputs guide biological or experimental actions, incorporating prediction provenance, uncertainty, calibration, and, where appropriate, explainability. Connections between components indicate that configuration, compatibility, and provenance must remain traceable across workflow stages.
Figure 1. Architectural requirements for verifiable protein machine learning. Five core requirements form the reproducibility architecture: deterministic curation, persistent artefacts, explicit interface contracts, reproducible execution, and transparent evaluation. A decision-readiness layer extends this architecture when workflow outputs guide biological or experimental actions, incorporating prediction provenance, uncertainty, calibration, and, where appropriate, explainability. Connections between components indicate that configuration, compatibility, and provenance must remain traceable across workflow stages.

TABLE 1. Operational requirements for verifiable protein machine learning workflows.

RequirementProtein-specific riskMinimum required recordVerification checkImplementation and extension
Deterministic curationDatabase or annotation drift; ambiguous labels; inappropriate negative classes; redundancy leakageSource and release; access date; retrieval and filtering rules; sequence and label definitions; inclusion and exclusion criteria; duplicate and redundancy procedures; negative-class constructionReconstruct the dataset; validate identifiers and sequence alphabets; detect duplicates; confirm class composition and provenanceMachine-readable manifests, dataset cards, FAIR metadata, and versioned curation scripts. Extensions include ontology-supported labels, source comparison, and continuous curation validation
Persistent artefactsStale or incompatible datasets, representations, partitions, models, predictions, or resultsPersistent identifier; artefact type and version; checksum; generating configuration; execution identifier; upstream and downstream provenance linksValidate checksum integrity, version and schema compatibility, and completeness of artefact lineageVersioned repositories, persistent identifiers, provenance records, and model or dataset cards. Extensions include immutable archival, retention policies, and automated invalidation of dependent artefacts
Explicit interface contractsIncompatible identifiers, schemas, dimensions, tokenisation settings, or input–output assumptionsRequired fields; data types; dimensions; identifier conventions; allowed values; compatibility constraints; expected input and output behaviourValidate schemas, dimensions, identifiers, allowed values, and contracts at workflow boundariesJSON schema, typed interfaces, workflow specifications, and the common workflow language. Extensions include automated contract testing, schema migration rules, and cross-implementation compatibility checks
Reproducible executionDependency, hardware, numerical-precision, seed, or runtime driftSoftware and dependency versions; environment or container; hardware context; numerical precision; random seeds; runtime parameters; relevant nondeterministic operationsExecute from a clean environment; validate configuration; compare output checksums; test deterministic behaviour where feasibleWorkflow engines, containers, lockfiles, environment manifests, and versioned configuration files. Extensions include continuous integration, multi-platform reproduction, and automated output comparison
Transparent evaluationLeakage; incompatible generalisation regimes; prevalence-dependent metric interpretation; test-set contaminationExact split assignments; similarity thresholds or grouping rules; split sizes and prevalence; metrics and baselines; model-selection protocol; test-isolation procedureCheck duplicates and homology across partitions; validate prevalence and test isolation; independently recompute reported metricsVersioned split files, benchmark cards, evaluation manifests, and reusable metric implementations. Extensions include external, temporal, family-level, structure-aware, and prospective evaluation
Decision readiness (extension)Overconfident predictions or unsupported biological and experimental prioritisationSample-level prediction; model, representation, partition, configuration, and execution identifiers; score or probability; uncertainty and calibration records when relevantConfirm prediction traceability and output completeness; assess calibration and applicability using task-appropriate proceduresPrediction records, model cards, calibration reports, and provenance-aware result tables. Extensions include explanation artefacts, decision logs, and linkage with prospective experimental outcomes

Verifiability begins with deterministic curation, which establishes the biological and computational meaning of the dataset before model development. Dataset construction should be captured through versioned, machine-readable records that preserve sources, transformations, annotation rules, and similarity-control procedures (AlQuraishi, 2019; González-Cebrián et al., 2024; Ferrer Florensa et al., 2024; Youngs et al., 2014).

Datasets and downstream outputs should also remain identifiable beyond the execution in which they were produced. Representations, partitions, trained models, predictions, and evaluation results should therefore be treated as persistent artefacts linked to their versions, generating configurations, execution records, and upstream dependencies (Hartley and Olsson, 2020; Bayarri et al., 2024; Gundersen et al., 2022; Khan et al., 2019). Persistence must be complemented by explicit interface contracts defining identifiers, required fields, data types, dimensions, compatibility conditions, and validation rules before artefacts are consumed downstream (Kanwal et al., 2017; Amstutz et al., 2016; Meyer, 2002).

Stable artefacts and interfaces must be supported by reproducible execution. Software and dependency versions, computational environments, hardware context, numerical precision, random seeds, runtime parameters, and relevant sources of nondeterminism should be recorded with each run (Leipzig et al., 2021). These records should enable clean-environment reruns and associate each result with its generating conditions. Workflow engines such as Nextflow (Di Tommaso et al., 2017), Snakemake (Köster and Rahmann, 2012), and the Common Workflow Language (Amstutz et al., 2016), together with containers, lockfiles, dependency managers, and deterministic seeding, provide practical mechanisms for limiting environmental drift (Kurtzer et al., 2017; Merkel , 2014).

Transparent evaluation closes the core architecture by linking reported performance to a precisely defined benchmark setting. Partition assignments, similarity or grouping rules, class prevalence, metrics, baselines, model-selection procedures, and test-isolation conditions should remain versioned with the evaluation record. This allows results to be interpreted according to the biological generalisation regime actually assessed (Kucera et al., 2023; Ferrer Florensa et al., 2024; Bernett et al., 2024; Pineau et al., 2021; Leipzig et al., 2021).

For decision-facing applications, the architecture can be extended beyond benchmark reproducibility. When predictions are used to prioritise variants, select candidates, or guide experiments (Hie et al., 2020), sample-level outputs should preserve their model, representation, configuration, partition, and execution provenance. Calibration, uncertainty, probabilities, and explanation artefacts should be retained when relevant to the intended decision and supported by the analytical design (Leipzig et al., 2021; Doshi-Velez and Kim, 2017; Hie et al., 2020; Biswas et al., 2021). This extension is not required for every benchmark, but becomes important when computational outputs directly influence biological or experimental actions.

The first five requirements define a minimum basis for reproducible benchmarking, while decision-readiness and other advanced practices can be introduced according to application scale, risk, and reuse expectations. Large or regenerable objects need not always be preserved when they can be reconstructed from stable identifiers, configurations, provenance, and integrity records.

4 Failure modes and good-practice recommendations

The operational requirements described above help identify failures that arise when biological meaning, configuration, provenance, or compatibility is lost across task definition, dataset construction, representation generation, model development, execution, and evaluation.

Task-definition and label-semantic drift occur when nominally similar datasets encode different prediction problems. Changes in source data, annotation rules, ambiguous evidence, or negative-class construction (Joshi et al., 2023; Sidorczuk et al., 2022; Pop et al., 2025) can alter class meaning, prevalence, composition, and physicochemical biases, making comparisons misleading when a shared task label conceals distinct biological questions.

A recurrent source of overoptimistic performance in protein machine learning is unintended similarity or dependence between development and evaluation data. Following Kaufman et al. (Kaufman et al., 2012) and its discussion in biological machine learning by Bernett et al. (Bernett et al., 2024), leakage occurs when information unavailable during model development influences training, validation, or performance estimation. In protein machine learning, homologous, near-duplicate, or otherwise non-independent sequences may cross training and test sets when redundancy control, clustering thresholds, metadata filters, or partitioning rules are insufficiently specified (Bushuiev et al., 2024; Bonin et al., 2023). This can inflate performance and overstate biological generalisation. Preventing such leakage, however, does not by itself establish distant biological generalisation. Random, redundancy-controlled, family-level, temporal, and structure-aware partitions preserve different relationships between development and evaluation data and therefore support different performance claims (Ferrer Florensa et al., 2024; Petti and Eddy, 2022; Zhou et al., 2019; Aguilera-Puga and Plisson, 2024).

Even when the underlying protein sequence is unchanged, its computational representation may vary with how it is generated. Protein language-model embeddings can differ with the checkpoint, tokenisation, padding or truncation, layer selection, pooling strategy, implementation, batching, and numerical precision (Li et al., 2024; Vieir et al., 2025), whereas alignments, derived features, and predicted structures depend on database releases, tool or model versions, parameters, and preprocessing procedures (Consortium, 2025; Abramson et al., 2024; Chen et al., 2022; Zheng et al., 2024). If these conditions are not preserved, identical sequences may produce non-equivalent downstream inputs. A related provenance risk is pretraining contamination, because evaluation sequences or close homologues may have been included in a foundation model’s pretraining corpus even when supervised partitions do not overlap (Hermann et al., 2024). Studies should therefore report checkpoint identifiers, available pretraining-corpus provenance, relevant dataset dates, and known sequence-overlap or homology analyses whenever possible.

Execution drift occurs when changes in software dependencies, hardware, random seeds, runtime parameters, numerical precision, or nondeterministic operations alter results across runs, whereas lineage drift arises when outputs can no longer be traced unambiguously to their generating artefacts and configurations (Martinez et al., 2025; Antunes and Hill, 2024; Schlegel and Sattler, 2025). If datasets, representations, partitions, models, or result tables are updated independently, downstream analyses may combine incompatible objects and obscure whether variation originates from the data, modelling choices, or execution conditions (Leipzig et al., 2021; Schlegel and Sattler, 2025). Good practice therefore requires recording the computational state of each run and the provenance relationships connecting its inputs, intermediate artefacts, models, and outputs.

An additional failure arises when benchmark results and workflow artefacts are not maintained as linked and versioned objects. Apparent improvements may reflect changes in preprocessing, partitions, leakage controls, class prevalence, metric implementations, baseline definitions, model-selection procedures, run aggregation, or test-set access rather than substantive methodological progress (Orioli et al., 2020; Xu et al., 2022; Bushuiev et al., 2024). Comparisons should therefore remain traceable to the exact artefact versions, partitions, metric definitions, validation procedures, and test-isolation conditions used. Uncertainty summaries should describe the number and structure of repeated runs, and conclusions should remain limited to the datasets, representations, models, and generalisation regimes actually evaluated.

Decision-readiness failures form a separate category. Reproducible benchmarks do not universally require explanation artefacts, but predictions used to prioritise candidates or guide experiments require stronger traceability. Prediction provenance, calibration, uncertainty, and applicability should be recorded when relevant, while explanations should be included only when supported by the model, analytical design, and intended decision (Karim et al., 2023; Hie et al., 2020; Medina-Ortiz et al., 2025).

These failures are interdependent. Task semantics determine class composition, partitioning defines the generalisation regime, representation and execution drift affect downstream models, and inconsistent evaluation can obscure the origin of performance differences. The following case study examines these dependencies in antimicrobial peptide classification through alternative task formulations, dataset compositions, and partitioning strategies under a controlled representation and modelling setup.

5 Demonstrative case study: workflow decisions reshape apparent model performance

To illustrate why benchmark-defining decisions should be treated as part of the scientific result, we examined how alternative task and evaluation choices changed apparent performance in antimicrobial peptide classification. The objective was not to develop a new AMP predictor or rank classifiers, but to show how commonly implicit workflow decisions can alter reported performance and its interpretation.

Two binary classification tasks were constructed from a sequence-level activity matrix derived from Peptipedia v2.0 (Cabas-Mora et al., 2024), containing 86,477 unique canonical peptide sequences of 2–150 residues. Both used the same 34,410 antimicrobial-positive peptides but differed in the negative class, with 10,211 toxic peptides without antimicrobial evidence in the first task and all 52,067 peptides without antimicrobial evidence in the second. These formulations differed in size, prevalence, composition, and biological meaning and therefore represent alternative benchmark constructions rather than simple relabelling of the same observations.

Each task was evaluated under a stratified-random reference and four sequence-similarity-aware regimes, H90, H70, H50, and H30, with complete MMseqs2 (Steinegger and Söding, 2017) groups retained within partitions. A single frozen 320-dimensional ESM2 (Lin et al., 2023) representation was used throughout. Four conventional classifier families were included as analytical strata; one task-specific variant per family was selected under RANDOM validation and then frozen across all partitioning regimes. The analysis comprised 4,000 final model executions from two tasks, five regimes, four model families, and 100 partition seeds. Complete methodological details are provided in SI Section S2, with model-selection outcomes and execution-integrity checks in SI Section S3.3.

Figure 2A first isolates the observed difference between the two benchmark formulations within each partitioning regime. Replacing the toxic background with the broader candidate-negative background consistently reduced average precision, showing that two[the choice of negative-class definition and the resulting benchmark construction can substantially change the apparent difficulty of an AMP-classification benchmark.] Because the tasks also differed in prevalence, size, sequence composition, and physicochemical properties, this pattern should not be interpreted as the isolated causal effect of relabelling. Instead, it shows that studies described under the same AMP-classification label may encode meaningfully different prediction problems. Additional task-characterisation analyses are provided in SI Sections S3.1 and S3.4.

Workflow decisions reshape apparent performance in antimicrobial peptide classification. **(A)** Performance difference between the broad-background and toxic-background task across RANDOM and four similarity-aware regimes. Negative values indicate lower performance under the broader negative-class construction. **(B)** Model-aggregated test average precision and MCC across partitioning regimes for both tasks. **(C)** Predefined contrasts for negative-class construction, partition strategy, and the H30–H90 similarity-threshold comparison. Points show medians, thick intervals show interquartile ranges, and thin intervals show bootstrap 95% confidence intervals. **(D)** Absolute methodological contrasts compared with variation among the four model families. Results are descriptive and limited to the evaluated tasks, representation, classifiers, and partitioning regimes.
Figure 2. Workflow decisions reshape apparent performance in antimicrobial peptide classification. (A) Performance difference between the broad-background and toxic-background task across RANDOM and four similarity-aware regimes. Negative values indicate lower performance under the broader negative-class construction. (B) Model-aggregated test average precision and MCC across partitioning regimes for both tasks. (C) Predefined contrasts for negative-class construction, partition strategy, and the H30–H90 similarity-threshold comparison. Points show medians, thick intervals show interquartile ranges, and thin intervals show bootstrap 95% confidence intervals. (D) Absolute methodological contrasts compared with variation among the four model families. Results are descriptive and limited to the evaluated tasks, representation, classifiers, and partitioning regimes.

Figure 2B shows how test performance changed across the RANDOM and similarity-aware regimes for both task formulations. The relationship was not monotonic with increasingly stringent similarity control. In particular, H90 could outperform RANDOM, whereas more restrictive regimes reduced performance, demonstrating that similarity-aware evaluation is not inherently more difficult and cannot be interpreted as a single benchmark condition. Its effect depends on the clustering procedure, similarity threshold, resulting group structure, and exact partition assignments. Supporting analyses of similarity structure and partition feasibility are provided in SI Sections S3.2 and S3.5.

The predefined contrasts in Figure 2C quantify these patterns across the full set of repeated runs. Changing from the toxic to the broader negative-class construction produced a median contrast of (−0.090) in average precision and (−0.052) in MCC. Relative to RANDOM, H90 increased median average precision by 0.018 and MCC by 0.052, whereas moving from H90 to H30 reduced average precision by 0.029 and MCC by 0.150. The associated interquartile ranges and bootstrap 95% confidence intervals summarise the variation across the predefined analytical comparisons rather than a single train–test split.

Figure 2D places the magnitude of these methodological effects alongside variation among the four model families. Two[Within each task, regime, seed, and metric, model-family variation was calculated as the median absolute pairwise performance difference among the four classifier families and was subsequently aggregated across task–regime conditions at the seed level. For average precision, the median absolute change associated with negative-class construction was 0.090, compared with a model-family twovariation of 0.021. For MCC, the H30–H90 contrast reached 0.150, compared with a model-family twovariation of 0.089. twoBecause only four conventional classifier families were evaluated, this comparison is intended as an illustrative comparison of effect magnitudes rather than as a general estimate of algorithmic variability. These results do not establish a universal hierarchy between workflow and model effects, but show that benchmark construction and evaluation choices can produce differences comparable to or larger than those observed among the evaluated predictive algorithms. Additional model-family comparisons are reported in SI Sections S3.6–S3.8.

The case study therefore reinforces the central argument of this Perspective. Task semantics, negative-class construction, similarity control, and partition definitions are part of the benchmark itself. They determine what is being predicted, which form of generalisation is evaluated, and how performance should be interpreted. Without explicit records of these decisions, differences between studies may be incorrectly attributed to model quality or dataset difficulty. Scope and limitations are discussed in SI Section S3.9.

6 Outlook

The future reliability of protein machine learning will depend not only on model advances, but also on whether benchmark-defining decisions become explicit, verifiable, and comparable across studies. As protein language models, multimodal architectures, and generative systems expand the range of prediction and design tasks (Xiao et al., 2025; Valentini et al., 2023; Trinquier et al., 2021; Hsieh et al., 2025), differences in task semantics, similarity control, representation provenance, execution conditions, and evaluation regimes will become increasingly consequential. Our demonstrative case study shows that such choices can materially alter reported performance and should therefore be treated as part of the scientific result.

Adoption can proceed incrementally. A minimal implementation may include versioned task definitions, dataset and partition manifests, immutable configurations, checksums, environment records, and predictions linked to their evaluation outputs. More complex or decision-facing workflows can extend this foundation with formal schemas, workflow engines, persistent archives, automated validation, calibration, uncertainty analysis, and prospective experimental records. Implementations may differ across laboratories, but the biological meaning, provenance, compatibility, and verification status of workflow artefacts should remain recoverable.

Further work should test these practices across additional protein tasks, representations, data modalities, and generalisation settings, including temporal, family-level, species-aware, structure-aware, and prospective evaluations. Particular attention is needed for representation provenance and potential pretraining contamination, which cannot be addressed through supervised train–test separation alone. Community agreement on minimum records and verification checks could improve reproducibility and comparability without prescribing a specific software stack, helping protein machine learning move toward workflows whose results can be independently reconstructed, verified, and interpreted in their biological context (Wilkinson et al., 2016; Chen et al., 2025; Boeckhout et al., 2018).

Generative AI statement

The author(s) declared that generative AI was used in the creation of this manuscript. As non-native English speakers, the authors used ChatGPT (OpenAI) to assist with language polishing, clarity, and grammar during the preparation of this manuscript. All content was subsequently reviewed and edited by the authors, who take full responsibility for the final version of the manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fbinf.2026.1925740/full#supplementary-material

Supplementary material Supplementary material at the source

Footnotes

Edited by: Bruno César Feltes, Federal University of Rio Grande do Sul, Brazil

Reviewed by: Petr Simecek, Central European Institute of Technology (CEITEC), Czechia

Éderson Pinto, Federal University of Rio Grande do Sul, Brazil

Statements

Conflict of interest

The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

The author(s) declared that financial support was received for this work and/or its publication. DM-O and JG-V acknowledge funding from FONDECYT Iniciación 11250295. JA, DM-O, and RU-P gratefully acknowledge support from the Centre for Biotechnology and Bioengineering - CeBiB (PIA project FB0001 and AFB240001, ANID, Chile). MDD acknowledges EU COST Action CA21160 (ML4NGP). PEACCEL was supported through a research program funded by the European Union (EU) and Region Reunion (FEDER).

Data availability

All code, configurations, partition assignments, sample-level predictions, analysis outputs, and instructions required to reproduce the demonstrative case study are openly available at https://github.com/kren-ai-lab/demonstrative_examples_architectural_good_practices. The starting peptide activity matrix was derived from Peptipedia v2.0 (Cabas-Mora et al., 2024), as described in the Supplementary Information. The workflow was verified in a clean environment and can be executed using the archived environment specification without undocumented local dependencies. The version associated with this article is release v0.1.0 at Git commit 86c4cc857259ef0ead56efcf87a27e963b2541b4 and is archived in Zenodo at https://zenodo.org/records/21810167. Source code is released under the MIT License, while documentation, results, and metadata are available under the Creative Commons Attribution 4.0 International License.

Author contributions

JG-V: Conceptualization, Investigation, Methodology, Visualization, Writing – original draft, Writing – review and editing. DF-V: Investigation, Visualization, Writing – original draft, Writing – review and editing. MS-G: Investigation, Visualization, Writing – original draft, Writing – review and editing. XC: Writing – original draft, Writing – review and editing. MD: Writing – original draft, Writing – review and editing. FC: Writing – original draft, Writing – review and editing. JA: Project administration, Validation, Writing – original draft, Writing – review and editing. RU-P: Funding acquisition, Resources, Validation, Writing – original draft, Writing – review and editing. DM-O: Conceptualization, Data curation, Formal Analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review and editing.

References

  1. Abramson J., Adler J., Dunger J., Evans R., Green T., Pritzel A. (2024). Accurate structure prediction of biomolecular interactions with alphafold 3. Nature 630, 493 – 500. doi:10.1038/s41586-024-07487-w
  2. Aguilera-Puga M. D., Plisson F. (2024). Structure-aware machine learning strategies for antimicrobial peptide discovery. Sci. Reports 14, 11995. doi:10.1038/s41598-024-62419-y
  3. AlQuraishi M. (2019). Proteinnet: a standardized data set for machine learning of protein structure. BMC Bioinformatics 20, 311. doi:10.1186/s12859-019-2932-0
  4. Amstutz P., Crusoe M. R., Tijanić N., Chapman B., Chilton J., Heuer M. (2016). Common Workflow Language, v1. 0.
  5. Antunes B., Hill D. R. (2024). Reproducibility, replicability and repeatability: a survey of reproducible research with a focus on high performance computing. Comput. Sci. Rev. 53, 100655. doi:10.1016/j.cosrev.2024.100655
  6. Bayarri G., Andrio P., Gelpí J. L., Hospital A., Orozco M. (2024). Using interactive jupyter notebooks and bioconda for fair and reproducible biomolecular simulation workflows. PLOS Comput. Biol. 20, e1012173. doi:10.1371/journal.pcbi.1012173
  7. Bernett J., Blumenthal D. B., Grimm D. G., Haselbeck F., Joeres R., Kalinina O. V. (2024). Guiding questions to avoid data leakage in biological machine learning applications. Nat. Methods 21, 1444 – 1453. doi:10.1038/s41592-024-02362-y
  8. Biswas S., Khimulya G., Alley E. C., Esvelt K. M., Church G. M. (2021). Low-n protein engineering with data-efficient deep learning. Nat. Methods 18, 389 – 396. doi:10.1038/s41592-021-01100-y
  9. Boeckhout M., Zielhuis G. A., Bredenoord A. L. (2018). The fair guiding principles for data stewardship: fair enough? Eur. Journal Human Genetics 26, 931 – 936. doi:10.1038/s41431-018-0160-0
  10. Bonin A., Guerrieri A., Ficetola G. F. (2023). Optimal sequence similarity thresholds for clustering of molecular operational taxonomic units in dna metabarcoding studies. Mol. Ecol. Resour. 23, 368 – 381. doi:10.1111/1755-0998.13709
  11. Bushuiev A., Bushuiev R., Sedlar J., Pluskal T., Damborsky J., Mazurenko S. (2024). Revealing Data Leakage in Protein Interaction Benchmarks, 10457. doi:10.48550/arXiv.2404.10457
  12. Cabas-Mora G., Daza A., Soto-García N., Garrido V., Alvarez D., Navarrete M. (2024). Peptipedia v2. 0: a peptide sequence database and user-friendly web platform. a major update. Database 2024, baae113. doi:10.1093/database/baae113
  13. Char S., Corley N., Alamdari S., Yang K. K., Amini A. P. (2025). Protnote: a multimodal method for protein–function annotation. Bioinformatics 41, btaf170. doi:10.1093/bioinformatics/btaf170
  14. Chen T. R., Lo C. H., Juan S. H., Lo W. C. (2021). The influence of dataset homology and a rigorous evaluation strategy on protein secondary structure prediction. PLoS One 16, e0254555. doi:10.1371/journal.pone.0254555
  15. Chen Z., Liu X., Zhao P., Li C., Wang Y., Li F. (2022). Ifeatureomega: an integrative platform for engineering, visualization and analysis of features from molecular sequences, structural and ligand data sets. Nucleic Acids Research 50, W434 – W447. doi:10.1093/nar/gkac351
  16. Chen Q., Hu Y., Peng X., Xie Q., Jin Q., Gilson A. (2025). Benchmarking large language models for biomedical natural language processing applications and recommendations. Nat. Communications 16, 3280. doi:10.1038/s41467-025-56989-2
  17. Chen Y., Fu L., Lu X., Li W., Gao Y., Wang Y. (2026). Combingym: a benchmark platform for machine learning-assisted design of combinatorial protein variants. doi:10.64898/2026.03.24.714074
  18. Cheng R., Liu T., Liao C., Wu X., Zhu L., Zhang S. (2026). Integrating protein language models with multimodal embeddings to accelerate function prediction of uncharacterized proteins. Int. J. Mol. Sci. 27, 3891. doi:10.3390/ijms27093891
  19. Chiu J. K. H., Ong R. T. H. (2022). Clustering biological sequences with dynamic sequence similarity threshold. BMC Bioinformatics 23, 108. doi:10.1186/s12859-022-04643-9
  20. Consortium T. U. (2025). Uniprot: the universal protein knowledgebase in 2025. Nucleic Acids Res. 53, D609 – D617. doi:10.1093/nar/gkae1010
  21. Dallago C., Mou J., Johnston K. E., Wittmann B. J., Bhattacharya N., Goldman S. (2021). Flip: benchmark tasks in fitness landscape inference for proteins. doi:10.1101/2021.11.09.467890
  22. Di Tommaso P., Chatzou M., Floden E. W., Barja P. P., Palumbo E., Notredame C. (2017). Nextflow enables reproducible computational workflows. Nat. Biotechnology 35, 316 – 319. doi:10.1038/nbt.3820
  23. Doshi-Velez F., Kim B. (2017). Towards a Rigorous Science of Interpretable Machine Learning. doi:10.48550/arXiv.1702.08608
  24. Dotan E., Jaschek G., Pupko T., Belinkov Y. (2024). Effect of tokenization on transformers for biological sequences. Bioinformatics 40, btae196. doi:10.1093/bioinformatics/btae196
  25. Draizen E. J., Readey J., Mura C., Bourne P. E. (2024). Prop3d: a flexible, python-based platform for machine learning with protein structural properties and biophysical data. BMC Bioinformatics 25, 11. doi:10.1186/s12859-023-05586-5
  26. Fernández-Díaz R., Cossio-Pérez R., Agoni C., Lam H. T., Lopez V., Shields D. C. (2024). Autopeptideml: a study on how to build more trustworthy peptide bioactivity predictors. Bioinformatics 40, btae555. doi:10.1093/bioinformatics/btae555
  27. Ferrer Florensa A., Almagro Armenteros J. J., Nielsen H., Aarestrup F. M., Clausen PTLC (2024). Spanseq: similarity-based sequence data splitting method for improved development and assessment of deep learning projects. NAR Genomics Bioinformatics 6, lqae106. doi:10.1093/nargab/lqae106
  28. Feurer M., Van Rijn J. N., Kadra A., Gijsbers P., Mallik N., Ravi S. (2021). Openml-python: an extensible python Api for openml. J. Mach. Learn. Res. 22, 1 – 5. Available online at:.
  29. Fursin G., Guillou H., Essayan N. (2020). Codereef: An Open Platform for Portable Mlops, Reusable Automation Actions and Reproducible Benchmarking. doi:10.48550/arXiv.2001.07935
  30. García-Vinuesa J., Rojas J., Soto-García N., Martínez N., Alvarez-Saravia D., Uribe-Paredes R. (2025). Geometric Deep Learning Assists Protein Engineering. Opportunities and Challenges. doi:10.1016/j.biotechadv.2025.108790
  31. González-Cebrián A., Bradford M., Chis A. E., González-Vélez H. (2024). Standardised versioning of datasets: a fair–compliant proposal. Sci. Data 11, 358. doi:10.1038/s41597-024-03153-y
  32. Graber D., Stockinger P., Meyer F., Mishra S., Horn C., Buller R. (2025). Resolving data bias improves generalization in binding affinity prediction. Nat. Mach. Intell. 7, 1 – 13. doi:10.1038/s42256-025-01124-5
  33. Gundersen O. E., Shamsaliei S., Isdahl R. J. (2022). Do machine learning platforms provide out-of-the-box reproducibility? Future Gener. Comput. Syst. 126, 34 – 47. doi:10.1016/j.future.2021.06.014
  34. Hartley M., Olsson T. S. (2020). Dtoolai: reproducibility for deep learning. Patterns 1, 100073. doi:10.1016/j.patter.2020.100073
  35. Heil B. J., Hoffman M. M., Markowetz F., Lee S. I., Greene C. S., Hicks S. C. (2021). Reproducibility standards for machine learning in the life sciences. Nat. Methods 18, 1132 – 1135. doi:10.1038/s41592-021-01256-7
  36. Helmy M., Crits-Christoph A., Bader G. D. (2016). Ten Simple Rules for Developing Public Biological Databases.
  37. Hermann L., Fiedler T., Nguyen H. A., Nowicka M., Bartoszewicz J. M. (2024). Beware of data leakage from protein llm pretraining. doi:10.1101/2024.07.23.604678
  38. Herrera-Rocha F., Medina-Ortiz D., Mauz F., Pleiss J., Davari M. D. (2025). Best practices for machine learning-assisted protein engineering. J. Chem. Inf. Model. doi:10.1021/acs.jcim.5c01983
  39. Hie B., Bryson B. D., Berger B. (2020). Leveraging uncertainty in machine learning accelerates biological discovery and design. Cell Systems 11, 461 – 477. doi:10.1016/j.cels.2020.09.007
  40. Hoang M., Singh M. (2025). Locality-aware pooling enhances protein language model performance across varied applications. Bioinformatics 41, i217 – i226. doi:10.1093/bioinformatics/btaf178
  41. Hsieh C. Y., Wang X., Zhang D., Xue D., Ye F., Huang S. (2025). Elucidating the Design Space of Multimodal Protein Language Models, 11454. doi:10.48550/arXiv.2504.11454
  42. Jendrusch M. A., Yang A. L., Cacace E., Bobonis J., Voogdt C. G., Kaspar S. (2025). Alphadesign: a de novo protein design framework based on alphafold. Mol. Syst. Biol. 21, 1 – 24. doi:10.1038/s44320-025-00119-z
  43. Joeres R., Blumenthal D. B., Kalinina O. V. (2025). Data splitting to avoid information leakage with datasail. Nat. Commun. 16, 3337. doi:10.1038/s41467-025-58606-8
  44. Joshi P., Banerjee S., Hu X., Khade P. M., Friedberg I. (2023). Gothresher: a program to remove annotation biases from protein function annotation datasets. Bioinformatics 39, btad048. doi:10.1093/bioinformatics/btad048
  45. Kanwal S., Khan F. Z., Lonie A., Sinnott R. O. (2017). Investigating reproducibility and tracking provenance–a genomic workflow case study. BMC Bioinformatics 18, 337. doi:10.1186/s12859-017-1747-0
  46. Kapoor S., Narayanan A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns 4, 100804. doi:10.1016/j.patter.2023.100804
  47. Karim M. R., Islam T., Shajalal M., Beyan O., Lange C., Cochez M. (2023). Explainable ai for bioinformatics: methods, tools and applications. Briefings Bioinformatics 24, bbad236. doi:10.1093/bib/bbad236
  48. Kaufman S., Rosset S., Perlich C., Stitelman O. (2012). Leakage in data mining: formulation, detection, and avoidance. ACM Trans. Knowl. Discov. Data (TKDD) 6, 1 – 21. doi:10.1145/2382577.2382579
  49. Khan F. Z., Soiland-Reyes S., Sinnott R. O., Lonie A., Goble C., Crusoe M. R. (2019). Sharing interoperable workflow provenance: a review of best practices and their practical application in cwlprov. GigaScience 8, giz095. doi:10.1093/gigascience/giz095
  50. Kim G., Lee S., Levy K. E., Kim H., Moriwaki Y., Ovchinnikov S. (2025). Easy and accurate protein structure prediction using colabfold. Nat. Protoc. 20, 620 – 642. doi:10.1038/s41596-024-01060-5
  51. Köster J., Rahmann S. (2012). Snakemake—A scalable bioinformatics workflow engine. Bioinformatics 28, 2520 – 2522. doi:10.1093/bioinformatics/bts480
  52. Kouba P., Kohout P., Haddadi F., Bushuiev A., Samusevich R., Sedlar J. (2023). Machine learning-guided protein engineering. ACS Catalysis 13, 13863 – 13895. doi:10.1021/acscatal.3c02743
  53. Kucera T., Oliver C., Chen D., Borgwardt K. (2023). Proteinshake: building datasets and benchmarks for deep learning on protein structures. Adv. Neural Inf. Process. Syst. 36, 58277 – 58289. doi:10.52202/075280-2539
  54. Kurtzer G. M., Sochat V., Bauer M. W. (2017). Singularity: scientific containers for mobility of compute. PloS One 12, e0177459. doi:10.1371/journal.pone.0177459
  55. Long Y., Abbasinejad F., Li F. Z., Reinprecht P., Wittmann B., Kennemur J. L. (2026). Enzyme engineering database (enzengdb): a platform for sharing and interpreting sequence–function relationships across protein engineering campaigns. Nucleic Acids Res. 54, D564 – D571. doi:10.1093/nar/gkaf1142
  56. Lauterbach S., Dienhart H., Range J., Malzacher S., Spoering J. D., Rother D. (2023). Enzymeml: seamless data flow and modeling of enzymatic data. Nat. Methods 20, 400 – 402. doi:10.1038/s41592-022-01763-1
  57. Leipzig J., Nüst D., Hoyt C. T., Ram K., Greenberg J. (2021). The role of metadata in reproducible computational research. Patterns 2, 100322. doi:10.1016/j.patter.2021.100322
  58. Li F. Z., Amini A. P., Yang K. K., Lu A. X. (2022). Pretrained protein language model transfer learning: is the final layer representation what we want. Proc. Mach. Learn. Struct. Biol. Work. NeurIPS 2022. Available online at:.
  59. Li F. Z., Amini A. P., Yue Y., Yang K. K., Lu A. X. (2024). Feature reuse and scaling: understanding transfer learning with protein language models. doi:10.1101/2024.02.05.578959
  60. Lin Z., Akin H., Rao R., Hie B., Zhu Z., Lu W. (2023). Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 379, 1123 – 1130. doi:10.1126/science.ade2574
  61. Lin H., Huang Y., Zhang O., Ma S., Liu M., Li X. (2025). Diffbp: generative diffusion of 3d molecules for target protein binding. Chem. Sci. 16, 1417 – 1431. doi:10.1039/d4sc05894a
  62. Longjohn R., Kelly M., Singh S., Smyth P. (2024). Benchmark data repositories for better benchmarking. Adv. Neural Inf. Process. Syst. 37, 86435 – 86457. doi:10.52202/079017-2744
  63. Martinez K. M., Wilding K., Llewellyn T. R., Jacobsen D. E., Montoya M. M., Kubicek-Sutherland J. Z. (2025). Evaluating the factors influencing accuracy, interpretability, and reproducibility in the use of machine learning classifiers in biology to enable standardization. Sci. Rep. 15, 16651. doi:10.1038/s41598-025-00245-6
  64. Medina-Ortiz D., Khalifeh A., Anvari-Kazemabad H., Davari M. D. (2025). Interpretable and explainable predictive machine learning models for data-driven protein engineering. Biotechnol. Adv. 79, 108495. doi:10.1016/j.biotechadv.2024.108495
  65. Merkel D. (2014). Docker: lightweight linux containers for consistent development and deployment. Linux J. 239, 2. Available online at:.
  66. Meyer B. (2002). Applying’design by contract. Computer 25, 40 – 51. doi:10.1109/2.161279
  67. Notin P., Kollasch A., Ritter D., Van Niekerk L., Paul S., Spinner H. (2023). Proteingym: Large-scale benchmarks for protein fitness prediction and design. Adv. Neural Information Processing Systems 36, 64331 – 64379. doi:10.52202/075280-2810
  68. Orioli S., Larsen A., Bottaro S., Lindorff-Larsen K. (2020). How to Learn from Inconsistencies: Integrating Molecular Simulations with Experimental Data in Computational Approaches for Understanding Dynamical Systems: Protein Folding and Assembly.
  69. Petti S., Eddy S. R. (2022). Constructing benchmark test sets for biological sequence analysis using independent set algorithms. PLOS Comput. Biol. 18, e1009492. doi:10.1371/journal.pcbi.1009492
  70. Pimentel J. F., Murta L., Braganholo V., Freire J. (2019). “ A large-scale study about quality and reproducibility of jupyter notebooks,” in 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR) (IEEE), 507 – 517.
  71. Pineau J., Vincent-Lamarre P., Sinha K., Larivière V., Beygelzimer A., d’Alché Buc F. (2021). Improving reproducibility in machine learning research (a report from the neurips 2019 reproducibility program). J. Machine Learning Research 22, 1 – 20. Available online at:.
  72. Pop M., Attwood T. K., Blake J. A., Bourne P. E., Conesa A., Gaasterland T. (2025). Biological databases in the age of generative artificial intelligence. Bioinforma. Adv. 5, vbaf044. doi:10.1093/bioadv/vbaf044
  73. Schlegel M., Sattler K. U. (2025). Capturing end-to-end provenance for machine learning pipelines. Inf. Syst. 132, 102495. doi:10.1016/j.is.2024.102495
  74. Semmelrock H., Ross-Hellauer T., Kopeinik S., Theiler D., Haberl A., Thalmann S. (2025). Reproducibility in machine-learning-based research: overview, barriers, and drivers. AI Mag. 46, e70002. doi:10.1002/aaai.70002
  75. Sidorczuk K., Gagat P., Pietluch F., Kała J., Rafacz D., Bąkała L. (2022). Benchmarks in antimicrobial peptide prediction are biased due to the selection of negative data. Briefings Bioinforma. 23, bbac343. doi:10.1093/bib/bbac343
  76. Steinegger M., Söding J. (2017). Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nat. Biotechnology 35, 1026 – 1028. doi:10.1038/nbt.3988
  77. Sun Y., Shen Y. (2025). Structure-informed protein language models are robust predictors for variant effects. Hum. Genet. 144, 209 – 225. doi:10.1007/s00439-024-02695-w
  78. Swainston N., Baici A., Bakker B. M., Cornish-Bowden A., Fitzpatrick P. F., Halling P. (2018). Strenda db: enabling the validation and sharing of enzyme kinetics data. FEBS Journal 285, 2193 – 2204. doi:10.1111/febs.14427
  79. Terwilliger T. C., Liebschner D., Croll T. I., Williams C. J., McCoy A. J., Poon B. K. (2024). Alphafold predictions are valuable hypotheses and accelerate but do not replace experimental structure determination. Nat. Methods 21, 110 – 116. doi:10.1038/s41592-023-02087-4
  80. Trinquier J., Uguzzoni G., Pagnani A., Zamponi F., Weigt M. (2021). Efficient generative modeling of protein sequences using simple autoregressive models. Nat. Communications 12, 5800. doi:10.1038/s41467-021-25756-4
  81. Valentini G., Malchiodi D., Gliozzo J., Mesiti M., Soto-Gomez M., Cabri A. (2023). The promises of large language models for protein design and modeling. Front. Bioinforma. 3, 1304099. doi:10.3389/fbinf.2023.1304099
  82. Vieira L. C., Handojo M. L., Wilke C. O. (2025). Medium-sized protein language models perform well at transfer learning on realistic datasets: lc vieira et al. Sci. Rep. 15, 21400. doi:10.1038/s41598-025-05674-x
  83. Wei Z. G., Chen X., Zhang X. D., Zhang H., Fan X. G., Gao H. Y. (2023). Comparison of methods for biological sequence clustering. IEEE/ACM Trans. Comput. Biol. Bioinforma. 20, 2874 – 2888. doi:10.1109/tcbb.2023.3253138
  84. Weissenow K., Rost B. (2025). Are protein language models the new universal key? Curr. Opin. Struct. Biol. 91, 102997. doi:10.1016/j.sbi.2025.102997
  85. Wilkinson M. D., Dumontier M., Aalbersberg I. J., Appleton G., Axton M., Baak A. (2016). The fair guiding principles for scientific data management and stewardship. Sci. Data 3, 1 – 9. doi:10.1038/sdata.2016.18
  86. Xiang R., Domínguez-Dalmases C., Cañellas-Solé A., Guallar V. (2025). Amlprot: an automated machine learning library for protein applications. Bioinformatics 41, btaf543. doi:10.1093/bioinformatics/btaf543
  87. Xiao Y., Zhao W., Zhang J., Jin Y., Zhang H., Ren Z. (2025). Protein Large Language Models: A Comprehensive Survey. arXiv preprint arXiv:2502.17504.
  88. Xu Z., Escalera S., Pavão A., Richard M., Tu W. W., Yao Q. (2022). Codabench: flexible, easy-to-use, and reproducible meta-benchmark platform. Patterns 3, 100543. doi:10.1016/j.patter.2022.100543
  89. Youngs N., Penfold-Brown D., Bonneau R., Shasha D. (2014). Negative example selection for protein function prediction: the nogo database. PLoS Computational Biology 10, e1003644. doi:10.1371/journal.pcbi.1003644
  90. Zhang J., Wang R., Zhou K., Xiao T., Zhu L., Min Y. (2026). Pepbenchmark: A Standardized Benchmark for Peptide Machine Learning. arXiv preprint arXiv:2604.10531.
  91. Zheng W., Wuyun Q., Li Y., Zhang C., Freddolino L., Zhang Y. (2024). Improving deep learning protein monomer and complex structure prediction using deepmsa2 with huge metagenomics data. Nat. Methods 21, 279 – 289. doi:10.1038/s41592-023-02130-4
  92. Zhou N., Jiang Y., Bergquist T. R., Lee A. J., Kacsoh B. Z., Crocker A. W. (2019). The cafa challenge reports improved protein function prediction and new functional annotations for hundreds of genes through experimental screens. Genome Biology 20, 244. doi:10.1186/s13059-019-1835-8

Citation

García-Vinuesa, J., et al. (2026). Architectural good practices for reproducible benchmarking in protein machine learning. Frontiers in Bioinformatics, Volume 6. https://doi.org/
Received
Accepted
Published
Volume
6 - 2026

Copyright

Copyright © 2026 García-Vinuesa, Fernández-Villegas, Soto-García, Cadet, Davari, Cadet, Asenjo, Uribe-Paredes and Medina-Ortiz.

License: https://creativecommons.org/licenses/by/4.0/

*Correspondence: Roberto Uribe-Paredes; David Medina-Ortiz

Article metrics

Views
0
Downloads
0
Citations
0
Shares
0

Related articles

Original ResearchIdentification and validation of biomarkers related to oxidative stress in pediatric sepsis utilizing WGCNA and multiple machine learning algorithmsYumeng Liu, Gang LuoReviewStrategic roadmap for FAIR-compliant transcriptomic workflows: from technical challenges to bioinformatics democratizationVerónica Jiménez Jacinto, Leticia Vega-AlvaradoOriginal ResearchMachine learning-based identification of CXCL13 and CXCL10 as key biomarkers in rheumatoid arthritisSubash M, Sheik Asraf S, Santhanamari M, Nagaraj P, Ayodeji Olalekan Salau, Abdullahi Garba Usman, Mouaz Yousfan