From sensor to solution: grounded LLM interfaces for equitable, smartphone-based pathogen surveillance

1 Introduction

Timely pathogen detection is crucial for maintaining health and saving lives (Xu et al., 2025). To identify these infectious “attackers,” a wide range of diagnostic tools has been proposed (Lazcka et al., 2007; Aladhadh, 2023; Lee G. et al., 2023; Bagi et al., 2025; Jamalzadegan et al., 2025; Xu et al., 2025). In recent years, smartphone-based pathogen detection tools have been increasingly introduced (Lazcka et al., 2007; Wei et al., 2014; Zhou et al., 2022; Aladhadh, 2023; Lee G. et al., 2023; Bagi et al., 2025; Hossain et al., 2025; Jamalzadegan et al., 2025; Xu et al., 2025; Zhang et al., 2025) because they bring several practical advantages for point-of-care (POC) testing, including portability, low cost, fast readout, and easy deployment outside a centralized lab (Xu et al., 2025) (Figure 1a). The smartphone is a “ready-to-use platform” that already includes a camera, computational unit, data connectivity, and a user-friendly interface, and when combined with low-cost biosensor assays, it can generate diagnostic signals directly in the field (Xu et al., 2025). However, the main bottleneck is not the hardware but the signal-to-action transition, how we translate raw sensor output into a trustworthy action for the end user.

More specifically, the input data from smartphone-based pathogen detection tools is not a single “clean” data type. It mainly comes in two main formats (Ding et al., 2018): (1) vision/image-based data, and (2) numerical/intensity-based data. The issue is that both data types can easily fail in real-world settings if they do not undergo strict quality checks before interpretation. For example, image-based readouts can be distorted by motion blur, glare, uneven lighting, and framing errors, while intensity-based measurements can be affected by sensor drift, contact instability, and electronic noise. Therefore, if we do not filter and validate these inputs with modality-specific QC steps, the final output may appear “confident,” but it will not be reliable.

This becomes even more critical for pathogen detection compared to many chronic disease monitoring workflows, because a faster response is usually needed in infectious disease settings. A small interpretation error (or even a delay caused by confusion) can lead to missed triage, delayed isolation, delayed referral, or weak reporting, increasing transmission risk and outbreak escalation. Data quality is not just about improving accuracy on a plot; it is directly tied to safe, timely action in the field. That is why having a strong, multi-step QC pipeline (and a strict “no-call” option when the input is low-quality or uncertain) is essential before making any final decision.

On the other hand, generative AI, a subset of artificial intelligence, can generate responses from a variety of inputs, including text, images, and other data, depending on the user’s objectives. Large language models (LLMs), a major branch of generative AI, have advanced rapidly in recent years and expanded across many application domains due to their unique capabilities in reasoning, multimodal understanding, and natural-language interaction. Specifically for healthcare applications, several recent studies have shown that GenAI and LLMs have been extensively used for various purposes in the healthcare field (Figure 1b), which could provide us a pathway to extend the applications of these AI subsets for smartphone pathogen detection assays to check the quality control of smartphone raw data before responding to end users.

In this Perspective, we first summarize the current status of using conventional AI and generative AI in smartphone-based pathogen detection (Figure 1c), then introduce the main data modalities used in smartphone-based pathogen detection assays and summarize common assay platforms and processing methods for each modality. We then highlight the key challenges that arise from these modalities. Next, we propose Grounded Multimodal LLM Interfaces (GM-LLMs) as an alternative “interpretation layer” that can serve as a safety-governed gate for these pipelines by enforcing evidence-first quality control (QC), retrieving version-controlled protocols, and generating protocol-constrained next steps (retest, refer, report). Finally, we discuss how GM-LLMs can also support real-world pathogen monitoring assays using smartphones to improve trust and equity in practical deployment.

2 Current landscape: conventional AI and GenAI in smartphone pathogen detection

A significant amount of research has used traditional AI methods, particularly Deep Learning (DL) architectures such as Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), to improve smartphone-based sensing in areas ranging from infectious disease surveillance to food safety. Studies have shown that ViTs can achieve high diagnostic accuracy for conditions such as polio and lumpy skin disease, yet these models often operate as “black boxes” that lack semantic reasoning and fail to guide users when image quality degrades (Ayana et al., 2025; Ullah et al., 2025). Similarly, in the food industry, these classical models achieved precise quantitative detection of meat and avocado quality; however, their systems proved fragile to environmental variations, such as lighting changes, and provided limited explainability for why a specific classification was made (Gao et al., 2024; Lee et al., 2025). These conventional approaches excel at specific detection tasks but consistently face an “interpretation gap,” in which the model generates a probability score without the contextual awareness or protocol-based guidance needed to support actionable decision-making in real-world scenarios.

In response to these limitations, recent research has shifted toward Generative AI and Multimodal Large Language Models (MLLMs) to introduce reasoning, interactivity, and comprehensive data interpretation into the sensing pipeline. Cutting-edge systems like SkinGPT-4 and MOSAIC have successfully leveraged LLMs to conduct interactive patient history taking and automate image quality control, effectively functioning as intelligent “copilots” that guide users through the diagnostic process (Zhou et al., 2024; Lee et al., 2025; Li et al., 2025; Ma et al., 2025). Further applications have used Visual Question Answering (VQA) and Retrieval-Augmented Generation (RAG) to provide farmers and consumers with context-aware, actionable advice rather than simple labels (Nanavaty et al., 2024; Soekamto et al., 2025). However, this transition is not without challenges; generalist models like GPT-4 Omni, while highly sensitive, suffer from low specificity and “hallucination,” leading to a high rate of false positives compared to specialized neural networks (Deng et al., 2025). Consequently, while GenAI offers the reasoning layer missing from conventional models, its deployment requires rigorous grounding mechanisms to ensure safety and accuracy, paving the way for the hybrid Grounded Multimodal LLM (GM-LLM) approach proposed in this work. These studies demonstrate the promise of generative AI and large language models (LLMs) for application in smartphone-based pathogen detection assays. In the sections that follow, we focus primarily on the scientific potential of these AI subsets to serve as quality control (QC) units for raw data extracted from smartphone assays.

3 Data modalities and bottlenecks in smartphone-based pathogen detection

Generally, smartphone-based pathogen detection tools generate data through two main streams: Vision/Image-Based Data and Numerical/Intensity-Based Data, both leading to diagnostic results (Pawar et al., 2023). The vision/image-based stream encompasses colorimetric imaging (Nelis et al., 2020), lateral-flow assays (LFAs) (Rong et al., 2019), microfluidic paper-based analytical devices (μPADs) (Cho et al., 2015), and colorimetric nucleic acid amplification tests, such as loop-mediated isothermal amplification (LAMP) (Nguyen et al., 2020; Garcia-Bernalt Diego et al., 2022), microscopic and morphological imaging (Banik et al., 2021) in brightfield (BF), fluorescence (FL), or darkfield (DF) modes, lensless holographic/interferometric imaging (Wu and Ozcan, 2018), digital assay quantification (e.g., digital PCR (dPCR) or droplet digital PCR (ddPCR), and related digital LAMP formats) (Ding et al., 2018), fluorescence/bioluminescence intensity measurements (Wei et al., 2013), and spectral methods including Raman/surface-enhanced Raman scattering (SERS) spectroscopy (Mu et al., 2018; Jiang et al., 2025).

On the other hand, the numerical or intensity-based data stream is mainly generated from electrochemical transducers (specifically voltammetry, amperometry/chronoamperometry, and electrochemical impedance spectroscopy (EIS)) (Sun et al., 2014) and magnetic sensors (such as giant magnetoresistance (GMR) resistance changes) linked with smartphone readers (Choi et al., 2016). This stream can also include acoustic/resonant measurements (Liu et al., 2023) (e.g., surface acoustic wave (SAW) (Turbé et al., 2017) or quartz crystal microbalance (QCM) readouts expressed as frequency or phase shifts (Lim et al., 2020)) and other electrical modalities such as resistive, capacitive, or photoelectrochemical (PEC) (Scott et al., 2022) signals. Finally, the phone provides critical contextual metadata (time, geolocation, user/workflow inputs) and connectivity (e.g., Bluetooth and cloud integration) that enable automated analysis, longitudinal tracking, and outbreak mapping.

Beyond the data modalities themselves, the challenges associated with these data types can be categorized into several primary failure modes. First, measurement and quality-control (QC) failures include blur or glare, bubbles, clogging, contact noise, dust, invalid tests, poor fill, and drift or saturation. Second, domain shift and calibration issues arise from phone-to-phone variance, lighting and exposure differences, reagent-lot effects, environmental humidity or temperature, and site-specific priors. Third, interpretation-to-action failures occur when a “positive” readout does not translate into a safe or locally feasible next step, such as deciding between retesting, referral, or reporting. Many current pipelines rely on deterministic, threshold-based algorithms, such as line detection or pixel-intensity cutoffs, which are susceptible to failure under field variability and distribution shift. Additional limitations in field deployment include data scarcity and dataset bias, such as small, curated datasets, limited hard cases, and label noise; evaluation and reporting gaps, including reliance on accuracy-only metrics without abstention, robustness stress tests, calibration, or safety-violation reporting; privacy, security, and abuse risks, such as metadata leakage, spoofing or replay, and adversarial misuse; and regulatory and governance barriers, including protocol versioning and updates, auditability, and accountability.

Collectively, these challenges can significantly affect end users’ final interpretation of biosensor readouts, reducing not only analytical performance but also practical trust and adoption in real-world settings. In real-world field deployments, even small QC failures or distribution shifts can propagate into incorrect triage decisions, while unclear “next-step” guidance can lead to alarm fatigue and cause users to abandon the tool entirely. To address this essential problem, a multi-step, evidence-first data quality validation pipeline is required before any interpretation-to-action step.

4 Proposed framework: grounded and safe AI interfaces (GM-LLMs)

Given the data-quality challenges in smartphone-based pathogen detection, grounded multimodal LLMs could be an outstanding candidate to serve as a “middle layer” in the pipeline, acting as a safety-governed quality-control (QC) gate for raw inputs before any downstream interpretation or results are delivered to end users. To date, only a limited number of studies have demonstrated how LLMs can be integrated into smartphone-based medical decision support tools, and even fewer have focused specifically on pathogen detection workflows. As an adjacent example, a recent study developed a multimodal LLM-based system to screen input image quality, detect ocular surface diseases, grade severity, and generate human-readable explanations without model fine-tuning, relying mainly on structured prompts and few-shot examples (Li et al., 2025). These early demonstrations motivate exploring similar grounded, QC-first interfaces for pathogen-focused smartphone diagnostics, where input validity and safe action are critical, especially since many current smartphone pathogen readers still depend on brittle image/signal pipelines and can fail under real-world noise and ambiguity (Lee S. et al., 2023).

Following this aim, the main idea is that, depending on the data modality generated by the smartphone platform, image/visual inputs or numerical/signal inputs (typically time-series or continuous streams from wearable sensors), we define a dedicated quality-control (QC) layer as a first-class gate in the pipeline to evaluate input validity before any downstream interpretation. For image-based inputs, this QC unit can check issues such as blur, glare, dust, poor framing, or missing control features. For numerical or time-series signals, QC can focus on noise, drift, saturation, contact artifacts, unstable baselines, or abnormal patterns that indicate an invalid measurement. By enforcing this evidence-first QC gate, the system can prevent low-quality or compromised inputs from propagating into confident but unreliable results, and it can trigger re-testing or escalation when input quality is insufficient. The next key resource for this QC unit is leveraging trusted, verified protocols, such as guidance from the World Health Organization (WHO), national public-health agencies, device Instructions for Use (IFUs), and well-established clinical workflows reported in the literature, to determine whether an input is interpretable and what constraints should apply before any action is suggested. To achieve this, retrieval-augmented generation (RAG) (Lewis et al., 2020; Gao et al., 2023) can be a great solution, acting as an “LLM accessory” that builds a curated protocol library from authoritative sources (e.g., WHO, FDA, NIH, and local ministry guidelines) and retrieves the most relevant, versioned content for a specific assay and context (Zarfati et al., 2025). In this way, the QC-first pipeline not only checks whether the data is error-free but also whether the evidence and recommended next steps are consistent with the retrieved protocol rules, rather than relying on the LLM to guess or hallucinate policies.

Concrete evidence from adjacent clinical domains supports this approach. In radiology guideline consultations, a RAG system that integrated clinical protocols into a vector retrieval pipeline outperformed a standalone LLM baseline in factual accuracy (0.89 vs. 0.68) and clinical usefulness (Komenda et al., 2026), while a separate study found that RAG completely eliminated hallucinations (0% vs. 8% for the ungrounded model) and improved response rankings (Wada et al., 2025). These results demonstrate that anchoring LLM outputs to versioned, retrieved documents effectively reduces hallucination and protocol drift. Applying the same logic to the GM-LLM QC layer proposed here, a RAG module would retrieve the relevant guideline for the specific assay being run (e.g., an LFA interpretation protocol for a particular pathogen and device), and the QC gate would proceed to generate a re-test, refer, or report recommendation only if explicitly supported by the retrieved protocol text, extending the radiology paradigm to a field-deployed environment where protocol traceability is vital.

By building this QC unit that considers both input data quality and compares the input evidence against verified protocols, this QC layer can serve as a robust gate in the “middle spot,” bridging raw inputs from smartphone-based pathogen monitoring to the final reported results for end users. To ensure safety concerns are addressed, the first action of this QC gate is to prevent the GM-LLM pipeline from continuing if the input does not meet minimum quality validation; in other words, the RAG-based protocol verification step will not be activated for low-quality or invalid inputs. However, if the first QC gate confirms that the data quality is acceptable, then the RAG component inside the QC unit will be activated to retrieve the most relevant protocol resources from verified references, so that the final output to end users becomes more robust, safer, and consistent with versioned guidance, and is also explicitly cited based on the verified reference protocols (Figure 2).

To build this GM-LLM as the QC gate, several LLM-related considerations are essential to increase the accuracy and trustworthiness of the final output delivered to end users. One important aspect is prompt design (Wang et al., 2025): writing accurate, structured prompts that minimize hallucination and force the model to follow the QC-first logic (e.g., “check validity first, then retrieve protocols, then answer”). Beyond prompt design, model selection is crucial because commercial and open-source multimodal LLMs vary in reasoning ability, robustness to noisy retrieved context, and consistency of responses (Şakar and Emekci, 2025).

While multimodal LLMs provide strong reasoning and interpretability, they also carry risks such as hallucinated outputs, overconfidence under high uncertainty, and reduced reliability under distribution shift (Freyer et al., 2024). In smartphone-based pathogen detection, these risks could manifest as incorrect protocol guidance, unsupported result explanations, or confident interpretation of poor-quality sensor data, any of which could influence triage, reporting, or treatment decisions. The GM-LLM framework mitigates these risks through the QC-first gate and RAG grounding described above, supplemented by additional safeguards: a “no-call” output when confidence is low, mandatory evidence citations, version-controlled protocol libraries, and human-in-the-loop review for borderline cases (Nishisako et al., 2025). Importantly, GM-LLMs are designed to support, not replace, clinical decisions. This design aligns with Software as a Medical Device (SaMD) regulatory frameworks, which emphasize transparency, auditability, and clear allocation of responsibility among developers, deployers, and clinical users (IMDRF, 2013).

5 Discussion

Accuracy is not just a technical goal; it is essential for fairness. In low-resource food systems, false alarms waste limited testing supplies, while false reassurance allows contaminated food to stay on the market, worsening health risks and economic inequality (Free et al., 2013). However, field performance is repeatedly constrained by user-dependent variability (lighting, framing, and sample handling), matrix interference and background color in complex foods, instability of enzymatic and nucleic-acid reagents in non-laboratory settings, cross-reactivity of recognition elements, and the limited dynamic range of portable detectors compared with benchtop instruments, factors that together increase false positives and negatives and thereby undermine equitable outcomes (Wang et al., 2020).

As an example of the potential economic impact of a QC-gated pipeline, published public health data show that lateral flow assays have false-positive rates of about 5%–8% in field conditions due to cross-reactivity and environmental interference (Centers for Disease Control and Prevention, 2008; Varshochi et al., 2011; Kinyua et al., 2025). Based on published cost data from SARS-CoV-2 self-testing programs (Hansen et al., 2024), if 100,000 samples were screened using self-tests costing approximately $1.00–$4.50 each, then 5,000–8,000 false positives could trigger substantial confirmatory testing costs. Assuming confirmatory laboratory testing costs $3–$50 per sample, the resulting avoidable follow-up cost would be approximately $15,000–$400,000. If our proposed QC gate intercepts even half of these, the savings could offset the marginal computational cost of the GM-LLM layer while reducing unnecessary burden on reference laboratories.

Finally, to get the most benefit from AI-powered smartphone testing, the system must be easy for everyone to use. This includes support for people who speak different languages or have limited reading skills, using voice guidance, clear icons, and simple wording, so advanced tests do not increase existing inequalities. From a cost standpoint, smartphone and AI systems that include quality checks can help avoid unnecessary follow-up tests, incorrect referrals, and wasted products. By guiding users to the right decisions, these systems make better use of limited resources and allow laboratories and clinics to focus on the cases that truly need attention. Embedding smartphone systems with straightforward QC measures and pairing them with inclusive interfaces can markedly reduce operator error and make outputs both scientifically robust and socially accessible; crucially, a quality-control layer between raw smartphone readout and AI-based classification enables error flagging, uncertainty reporting, and optional remote expert review, creating a telemedicine-style oversight loop that limits inappropriate escalation, reduces wasted tests, and strengthens trust and participation in large-scale pathogen surveillance networks (Free et al., 2013).

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Younseong Song, Center for NanoImaging, Brigham and Women’s Hospital, United States

Reviewed by: Alessandro Perrella, Hospital of the Hills, Italy