Efficient high-resolution microscopic ghost imaging via sequenced speckle illumination and deep learning from a single noisy image

1 Introduction

Microscopy has transformed scientific research by enabling visualization of micro- and nanoscale structures, which is essential for biological studies and disease research. However, traditional optical microscopy is fundamentally limited by the Abbe diffraction limit, restricting achievable resolution (Abbe, 1873). While reducing the illumination wavelength and increasing the numerical aperture can improve resolution, imaging biological samples at the nanoscale requires XUV wavelengths (10–100 nm), which can damage samples and suffer from high absorption losses and limited optical components.

To overcome these challenges, ghost imaging (GI) has emerged as a promising technique. GI reconstructs images using correlated measurements from spatially separated detectors, making it attractive for lensless imaging and applications requiring low radiation doses. Originally developed with quantum entangled photons (Pittman et al., 1995), GI has since been demonstrated with classical light sources, broadening its applicability (Abouraddy et al., 2001; Bennink et al., 2002).

Advancements in computational techniques and single-pixel detectors have further enhanced GI. Computational ghost imaging (CGI) uses compressed sensing (CS) and reconstruction algorithms to recover images from fewer measurements (Shapiro, 2008; Katz et al., 2009), enabling lensless imaging in environments where traditional optics are impractical, such as with XUV sources. Despite using only, a single-pixel detector, CGI has enabled rapid computational imaging (Welsh et al., 2013). Simplified holography processes have also expanded the potential for 3D imaging and microscopy (Clemente et al., 2013).

Recent progress has extended GI into the x-ray and XUV regimes, including demonstrations with conventional x-ray sources (Yu et al., 2016; Pelliccia et al., 2016; Zhang A-X. et al., 2018), XUV free-electron lasers (Kim et al., 2020), and x-ray phase-contrast GI (Olbinado et al., 2021). Our recent work has enabled microscopic ghost imaging with a tabletop XUV source (Oh et al., 2025), and other studies have shown high-resolution GI of microscopic objects (Sun et al., 2019; Vinu et al., 2020; Dou et al., 2020). Techniques such as imaging at megahertz switching rates with cyclic Hadamard masks (Hahamovich et al., 2021) and novel phase microscopy approaches have significantly improved imaging speed and reduced system complexity (Zhao et al., 2023).

Nevertheless, many CGI methods rely on traditional pattern-based approaches, such as Hadamard and Fourier transforms, which require a large number of illumination patterns to achieve high resolution. For example, capturing Extended Graphics Array (XGA) resolution images (1,024 × 768 pixels) requires over 1.5 million patterns with the differential Hadamard method and over 3 million with the 4-step Fourier method (Duarte et al., 2008; Zhang et al., 2017; Gibson et al., 2020), resulting in long acquisition times and heavy computational loads. In contrast, our previous work with sequenced speckle illumination demonstrated that GI reconstruction is possible with significantly fewer patterns (Oh et al., 2023). In this study, we adopted sequenced speckle as the illumination strategy for our experiments to validate its effectiveness within the proposed framework.

Despite recent advances in deep learning for ghost imaging, existing approaches have notable limitations. Traditional deep learning methods typically require extensive training datasets, including paired low-resolution and reference images, which are often difficult to obtain in practical microscopic settings (Lyu et al., 2017; Wang et al., 2019; Hu et al., 2020). More recent methods, such as ghost imaging with deep neural network constraints (Wang et al., 2022), employ untrained networks and compressed sensing to reconstruct images directly from measurement data. However, these approaches still face significant challenges: they cannot fully exploit the spatial information inherent in speckle patterns, and they demand large volumes of measurement data and considerable computational resources. These limitations hinder the practical application of high-resolution, high-quality ghost imaging, especially when rapid or resource-efficient imaging is required.

To address these limitations, we introduce a denoising algorithm based on Noise2Void (N2V) (Krull et al., 2019), a deep learning method that requires only a single noisy image for training, eliminating the need for reference images. By combining ghost images obtained from random speckle patterns with the N2V model, we achieve high-quality, high-resolution images with fewer patterns and minimal computational resources. The N2V framework is built on the U-Net architecture (Ronneberger et al., 2015; Wu et al., 2024; Komatsu and Gonsalves, 2020; Jia et al., 2021), which is well-suited for image restoration and denoising, especially with limited training data.

We evaluated the quality of denoised images using Rayleigh, Abbe, and Sparrow criteria, and investigated how speckle pattern size affects microscopic ghost image quality. Our results reveal a clear relationship between speckle size and image resolution, demonstrating the effectiveness of our approach for efficient, high-quality microscopic ghost imaging.

2 Experimental setup and measurement principle

2.1 Experimental setup

Figure 1 illustrates the experimental setup. The experiments were conducted using a DMD-based CGI setup—specifically the Vialux^TM DLP V-Module V4100 board—which is capable of projecting sequenced speckle patterns at a rate of 50 frames per second. To enhance the randomness and diversity of the illumination, we employed sequenced speckle patterns with varying sizes. The DMD features a 0.7-inch diagonal array comprising 1,024 × 768 micromirrors, each with a pitch of 13.6 μm. A beam expander was used to illuminate the entire area of the DMD. The reflected light was directed through a positive lens (focal length = 125 mm) and then focused onto the object via an objective lens. For magnification, we used ZEISS A-Plan objectives with ×40 magnification (NA = 0.65, focal length = 4.1 mm, infinity-corrected). According to the Rayleigh criterion, the spatial resolution of the objective is 0.61 λ / N A, which yields approximately 0.6 µm for a 632.8 nm He–Ne laser wavelength. The light transmitted through the object was collected by a focusing lens and measured by a bucket detector (Thorlabs, PDA36A-EC), which recorded the total transmitted intensity.

During the experimental procedure, sequential patterns were displayed on the DMD. The reconstructed ghost image was obtained by correlating the intensity signals detected by the bucket detector with the random speckle patterns displayed on the DMD. To validate the feasibility and performance of the CMGI technique, a USAF 1951 resolution target was used, featuring the smallest line width of 2.2 μm.

2.2 Principle of measurement

We utilize a DMD to generate pre-designed speckle patterns of specific sizes, which are then projected onto the target as speckle patterns denoted by In(x, y). It is important to note that the bucket detector records the total intensity, Sn, of the light transmitted by the object. The object image, G (x, y), can be reconstructed by calculating the second-order correlation function (Abouraddy et al., 2001; Oh et al., 2023) between the intensity distribution, In, illuminated onto the target, and the total intensity, Sn, recorded by the bucket detector, as shown in Equation 1. Here, N represents the total number of patterns used in each experiment.

G x , y = 1 N ∑ n = 1 N I n x , y − I S n x , y (1)

2.3 Noise2Void Ghost Imaging (N2VGI) algorithm

In this study, we propose a Noise2Void-based Ghost Imaging (N2VGI) framework to denoise reconstructed ghost images using a self-supervised U-Net architecture. Although U-Net was originally developed for biomedical image segmentation (Ronneberger et al., 2015), its encoder–decoder design with skip connections is highly effective in preserving spatial information. In ghost imaging, the reconstruction can reflect the illumination pattern; since we employ sequenced speckle illumination, the resulting images contain locally uncorrelated speckle noise—the kind of random variation that N2V can learn to remove.

While more advanced architectures such as U-Net++, ResUNet, Attention U-Net, Pix2Pix, MSGU-Net, and GA-UNet (Zhou et al., 2018; Zhang Z. et al., 2018; Oktay et al., 2018; Isola et al., 2017; Cheng et al., 2024; Kaur et al., 2021) have demonstrated superior performance in supervised or data-rich environments, they generally rely on clean ground truth data, complex training pipelines, and greater computational resources-requirements incompatible with our self-supervised setting, which operates on single noisy ghost images without access to clean references. In contrast, U-Net serves as the fundamental baseline architecture for image denoising and restoration tasks. Its simple yet powerful encoder–decoder structure with skip connections has been widely validated across diverse applications, making it the standard reference point for both performance and efficiency. U-Net is computationally efficient, converges quickly, and performs reliably even in unsupervised or self-supervised noise modeling with limited data (Wu et al., 2024; Komatsu and Gonsalves, 2020). Therefore, U-Net was selected as the most practical and effective choice, providing an optimal balance between simplicity, computational cost, and robust denoising performance under our experimental constraints.

The U-Net architecture used in our model is configured with a kernel size of 3, a network depth of 3, and an initial filter size of 32. This configuration facilitates multi-scale feature extraction, allowing the network to effectively learn complex noise distributions and improve image fidelity. The training dataset comprises non-overlapping patches of size 24 × 24 pixels, extracted from the input images. To evaluate generalization and mitigate overfitting, 1,000 patches are reserved for training, and the remainder are used for validation.

During training, batch normalization is applied to stabilize convergence. The loss function is defined as mean squared error (MSE), aiming to minimize reconstruction errors. The N2V manipulator is implemented using a median-based masking strategy, which replaces selected pixels with the median of their local neighborhood to prevent identity learning. The model is trained for 25 epochs with a batch size of 16. For noise estimation, 50% of the pixels in each patch are randomly selected, with a neighborhood radius of five pixels used to define local context during training.

Once the model is trained, as detailed in Table 1, it is integrated into an iterative reconstruction process to refine ghost images. Given a noisy ghost image G, the N2V model generates an estimated clean image L by identifying and suppressing noise. The residual noise component F is then computed F = G–L and further refined through a proximal operator, which applies soft-thresholding to enforce sparsity and enhance noise separation (Parikh and Boyd, 2014).

TABLE 1. N2VGI Algorithm: Applying the Noise2Void (N2V) model to enhance reconstructed ghost images, leveraging a U-Net architecture to capture and reduce noise. The stopping criterion is defined by the epsilon (ε) threshold set at 0.5.

To improve robustness, the refined noise estimate is iteratively averaged with previous estimates, yielding an updated noise estimate R. The final reconstructed ghost image is obtained by subtracting the refined noise estimate from the original input. We empirically evaluated convergence thresholds in the range ε = 0.3–0.6 and found that ε = 0.5 offered the best balance between denoising quality and original feature preservation. Consequently, the iterative process continues until the convergence parameter—defined as the ratio of the noise-suppressed image norm to the original image norm—falls below ε = 0.5 or the maximum iteration limit is reached.

This approach effectively enhances ghost image reconstruction by leveraging deep-learning-based self-supervised denoising while preserving fine structural details. The integration of N2V with iterative refinement provides a robust noise suppression mechanism, improving the quality and reliability of reconstructed images in quantum and computational ghost imaging applications. Figure 2 shows the workflow of the N2VGI algorithm.

2.4 Speckle size measurement and image quality evaluation criteria

To measure the speckle size, we employ a fast cross-correlation algorithm. This algorithm calculates the cross-correlation between consecutive input fields and then fits a Gaussian function to the cross-correlation result. The width and height parameters obtained from the Gaussian fitting provide information about the speckle size. The average speckle size within the input field is estimated based on the pitch size (13.6 μm) of the DMD used in the experiment. Using this methodology, we generated a total of 20 random patterns with varying speckle sizes ranging from 30 to 275 μm.

For a quantitative assessment of reconstructed image quality, the Structural Similarity Index (SSIM) is employed as a metric to evaluate the similarity between the reconstructed image and the reference image while considering luminance, contrast, and structural information. The SSIM is calculated using the following formula (Wang et al., 2004), as shown in Equation 2.Here, μ x and μ y represent the mean intensity values of images x and y, respectively, while σ x 2 and σ y 2 denote their variances. The term σ x y represents the covariance between the two images. The constants C 1 and C 2 are small values included to prevent instability when the denominators are close to zero. By evaluating the SSIM, the quality of the reconstructed images can be effectively compared and assessed based on their structural similarity to the reference image, offering a perceptually relevant measure of image fidelity.

S S I M x , y = 2 μ x μ y + C 1 2 σ x y + C 2 μ x 2 + μ y 2 + C 1 σ x 2 + σ y 2 + C 2 (2)

Figure 3 presents the resolution measurement of a ghost image. To evaluate the resolution of the ghost image in relation to changes in speckle size, we used the USAF resolution target as the object and measured the resolution based on the spatial resolution of the line pairs. The resolution value R is calculated using the following equation (Oh et al., 2023), as shown in Equation 3.where p 1 and p 2 are the positions of the two peaks, and W 1 and W 2 are their respective widths. As shown in Figure 3, we outlined three lines and obtained an object profile through Gaussian fitting of these lines. To quantitatively evaluate the resolution value, we compare our approach R to the following criteria:

R = 2 p 2 − p 1 W 2 + W 1 (3)

• • Rayleigh Criterion: Applies when the central maximum of one Airy disk coincides with the first minimum of another. This corresponds to an R value of 0.5.
• • Abbe Criterion: Determined by the full width at half maximum (FWHM) of the two overlapping Airy disks. This corresponds to an R value of 0.41.
• • Sparrow Criterion: Observed when the overlapping Airy disks exhibit no discernible difference in their superimposed intensities across the entire resolution distance. This corresponds to an R value of 0.38.

The R value serves as a measure of resolution for the reconstructed images. Specifically, a resolution value exceeding R > 0.5 indicates that the image surpasses conventional criteria (Advanced Microscopy, 2025). By comparing the R values, we can effectively assess the quality and resolution of the ghost images obtained through our method.

3 Experimental results

3.1 Microscopic ghost imaging

Figure 4 depicts the reconstructed microscopic ghost images for various object targets, with line widths ranging from 15.6 µm to 2.2 µm. These images were captured using a ×40 objective lens, resulting in a magnification range from 21.6 to 83.2. All images demonstrate sufficient spatial resolution, with the three lines clearly discernible to the naked eye.

In Figure 4b, we present the quantitative resolution R values obtained by averaging 10 line-out sections per image. Images 5–1, 6–4, 7-3, and 7-6 all exhibit R values above the Rayleigh limit of 0.5, and, apart from case 6–2, every image exceeds the Abbe limit of 0.41. Case 6–2 corresponds to a 7 µm line width and the lowest effective magnification (∼21.6×), which decreases the sampling interval in the line-out analysis and leads to an underestimated R value. In contrast, all other cases benefited from higher effective magnifications, yielding R values that surpass conventional resolution criteria.

3.2 Exploring the influence of speckle size on microscopic ghost imaging

To further explore these effects, experiments were conducted by projecting random speckle patterns of known sizes onto resolution targets. Figure 5 shows this experimental result.

The upper section of Figure 5 illustrates the reconstructed ghost images for speckle sizes from 30 µm to 210 μm, demonstrating that larger speckles improve contrast but reduce sharpness. The lower section presents SSIM and R value curves, showing that increased speckle size enhances SSIM while diminishing R value. This trade-off arises because larger speckles reinforce low-frequency information—boosting overall image fidelity—whereas smaller speckles preserve high-frequency details essential for fine resolution. To balance these effects, an appropriate speckle size should be chosen to match the object’s spatial scale: use smaller speckles when maximum resolution is required and larger speckles when image contrast is paramount. Furthermore, following our earlier work on mixed‐scale speckle illumination (Oh et al., 2023), a multiscale approach—projecting speckles of varying sizes sequentially or in combination—can simultaneously capture both high- and low-frequency content, thereby maximizing resolution and SSIM in a single dataset.

3.3 Enhancing GI quality through N2VGI algorithm optimization

We utilized the N2V model to enhance the image quality of GI by optimizing the experimental setup and speckle size in the initial reconstructed GI. In all reconstructed GI cases, we conducted 10,000 iterations. The number of patterns used is 10,000 for the random speckle pattern, which is only 0.6% of the patterns used in the differential Hadamard method and only 0.3% of the patterns used in the 4-step Fourier method. All reconstructed ghost images have a dimensional size of XGA, same to DMD (1,024 × 768). Initially, we divided the training and validation datasets with a single noisy image based on patch parameters. The patch size, determined by the average noise size of the reconstructed ghost image and the image’s dimensions, played a crucial role in determining the most optimal modelling parameters. Through iterative experiments, we developed an effective denoising N2V model. Subsequently, employing the N2VGI algorithm outlined in Table 1, we conducted iterative refinement until achieving the desired level of denoising. Figure 6 illustrates the impact of our N2V algorithm on individual images, with all images demonstrating a substantial reduction in noise, confirming the efficacy of our approach.

In Figure 4, only four images surpassed the resolution threshold of 0.5, defined by the Rayleigh standard. However, after applying the N2VGI algorithm, all images exceeded the Rayleigh criteria. On average, the R value in all images subjected to the N2VGI algorithm was approximately 33% higher than the average R value in the initial GI. For image 5-6, the R value increased by approximately 78% compared to its previous value. When considering SSIM, the application of the N2VGI algorithm resulted in an increase in SSIM values for all images, with the average SSIM value across all targets increasing by 324% in N2VGI compared to GI. For image 7-3, the SSIM increased by approximately 854% compared to the original image.

Additionally, we conducted an experiment on GI reconstructed with 2000 iterations to validate the algorithm’s applicability to cases with extremely low image quality. Figure 7 presents the GI obtained with a low number of iterations and the results when the N2VGI algorithm is applied to the images.

Figure 7 displays a low-iteration GI alongside an image enhanced with the N2VGI algorithm. The average R value of denoised images increased by approximately 63% compared to GI. In terms of SSIM, the average value increased by approximately 247% compared to the original images. While the algorithm notably improved the image in noisy regions, it is evident that some information was lost in the denoised image due to the limitations of the original data. Nonetheless, the N2VGI model demonstrates its effectiveness in enhancing even very low-quality GI images.

4 Conclusion

In conclusion, we demonstrated microscopic ghost imaging by optimizing both lens position and speckle size, achieving a smallest resolvable line width of 2.2 µm—well beyond conventional Rayleigh criteria. Our investigation revealed that as speckle size increases, resolution decreases while SSIM improves for certain target widths, with an optimal speckle scale maximizing both metrics. By combining speckle-based CGI with our self-supervised Noise2Void (N2VGI) U-Net algorithm, we successfully denoised images reconstructed from as few as 2,000–10,000 speckle patterns (under 1% of the patterns required by differential Hadamard or 4-step Fourier methods), without any reference images or large training sets. Despite this drastic reduction, N2VGI consistently delivered high-quality XGA-resolution (1,024 × 768) reconstructions using less computational resources. Because N2VGI relies on single-image training, there is a risk of overfitting to speckle-specific noise, which may degrade performance under low-contrast or highly correlated noise conditions. Future work will address these limitations by incorporating regularization techniques, multi-image strategies, and broader dataset validation to further improve robustness and extend applicability to light-sensitive biological imaging.

Generative AI statement

The author(s) declare that Generative AI was used in the creation of this manuscript. For Proofreading.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.