Nanoemulsion formulation based delivery system enhances anticancer efficacy of pumpkin seed oil against prostate cancer

Prostate cancer is an extremely variable and multifaceted malignancy and has significant contributor to morbidity and mortality in males. Current therapies frequently resulting in significant adverse consequences. Pumpkin seed oil (PSO) has antibacterial, antiviral, cytotoxic, and anticancer properties. Nevertheless, its hydrophobic characteristics restrict bioavailability, which might be enhanced using nanoemulsion based delivery system. In our previous study, the formulation and characterization of pumpkin seed oil nanoemulsion (PSO-NE) was investigated and observed that PSO-NE induces cytotoxicity in breast cancer cells (MCF-7 and MDA-MB-231). Nonetheless, a PSO-NE has not been studied in prostate cancer research, whether in vitro or in vivo. The primary aim of this work was to evaluate PSO-NE cytotoxic and apoptotic effects on prostate cancer cell lines (PC3 and LNCaP). Cytotoxicity efficacy was assessed using MTT assay. Cellular uptake and nuclear morphology were evaluated through confocal microscopy. Mechanistic assays included apoptosis, proliferation, and cell cycle analysis were evaluated through flow cytometry. Differential gene expression was quantified by real-time PCR. In this study, PSO-NE exhibited dose-dependent cytotoxicity, demonstrating a markedly reduced IC 50 in comparison to free PSO and Doxorubicin after 24 h of exposure (p < 0.01) accompanied with reduced cytotoxicity towards non-cancerous cells (Vero). At IC 50 concentrations (PC3 = 90.22 μg/mL and LNCaP = 65.47 μg/mL), PSO-NE strongly induced nucleus morphological changes, reduced cell proliferation, increased phosphatidylserine exposure, and arrested the cell cycle at G0/ G1 phase (p < 0.001). The expression of genes (Bcl-2, STAT-3, and NF-kB), related to cell proliferation, was significantly downregulated (p < 0.001), whereas the expression of genes (p53, Bax, and caspase 3), associated with apoptosis, was significantly upregulated compared to the untreated cell lines (p < 0.001), indicating the activation of apoptotic gene expression. The enhanced cytotoxicity of PSO-NE against prostate cancer cells, together with its capacity to cause apoptosis and cell cycle arrest, highlights its potential as an effective anticancer agent.

Prostate cancer is an extremely variable and multifaceted malignancy and has significant contributor to morbidity and mortality in males, with 1.3 million diagnoses and 358,989 fatalities occurring annually (Wasim et al., 2022; Litwin and Tan, 2017). The GLOBOCAN estimates in 2022 that prostate cancer was the most frequently diagnosed cancer among males globally, with 1.47 million new cases after lung cancer (Filho et al., 2025). The management of prostate tumors poses considerable clinical difficulties, owing to the disease’s pronounced invasive capability, particularly to essential organs such as the bones, liver, lungs, and brain. The inherent diversity of prostate cancer cells, marked by varied hereditary, genomic, and phenotypic features, hinders traditional treatments, underscoring the necessity for enhanced diagnostics and therapy methods (Saadh et al., 2025).

There are several treatment options, such as surgery, hormone therapy, chemotherapy, and radiation therapy. Although these methods can exhibit effectiveness, they frequently have restrictions and possible adverse effects (Nagireddy et al., 2021). Consequently, exploring natural compounds as a valuable source of safer pharmaceuticals for cancer treatment is prudent. Compounds produced from plants that may trigger apoptosis via several signaling pathways have been proposed as an alternate strategy for cancer therapy (Salehi et al., 2022). Currently, several plant-derived anticancer pharmaceuticals are available on the market, such as vinblastine, etoposide, paclitaxel, and camptothecin (Huang et al., 2021; Omara et al., 2020). These pharmaceuticals possess an herbal individuality, comparable efficacy, and less adverse effects in contrast to manufactured medications.

Pumpkin seed oil (PSO) is darkish green in color and includes a significant concentration of free polyunsaturated fatty acids, comprising four predominant fatty acids: oleic (43.8%), linoleic (33.1%), palmitic (13.4%), and stearic (7.8%) (Shaban and Sahu, 2017). PSO exhibit several biological properties that can be categorized as antibacterial, antidiabetic, antiviral, cytotoxic, anticancer, antioxidant, anti-inflammatory, cytoprotective, and anti-mutagenic effects (Medjakovic et al., 2016; Chen and Huang, 2019; Al-Okbi et al., 2017; Shayesteh et al., 2017). It is also abundant in minerals and liposoluble vitamins, specifically α- and γ-tocopherol, as well as carotenoids (Procida et al., 2013). Studies indicates that the PSO provides protection against prostate cancer by mitigating inflammation of prostatic tissues and alleviating micturition disorders associated with benign prostatic hyperplasia, and it may help inhibit the progression of kidney stones (Marks et al., 2001). This action’s process is associated with its strong antioxidant properties, due to high levels of vitamin E and carotenoids, particularly zeaxanthin and lutein (Anderson, 2005). Recently, options for chemical modification have been investigated to augment the biological activity of pumpkin seed oil. Ozone-modified PSO has been shown to possess enhanced antibacterial and bioactive potential due to structural modifications in unsaturated fatty acids, resulting in the generation of ozonides and peroxide derivatives with increased activity (Alsalamah et al., 2025). A plethora of studies has been undertaken to examine the anticancer capabilities of PSO against different cancers, including prostate and breast cancer, due to the presence of carotenoids, tocopherols, and sterols (Abdulsalam and Fathy, 2024).

Recent years have demonstrated an increasing interest in exploring the therapeutic properties of plant seed oil-based nanoemulsions, attributed to its beneficial characteristics such as improved bioavailability and stability (Nirmala et al., 2020). Nanoemulsions function as a delivery system due to their aqueous composition, enabling intravenous administration, enhanced cytotoxicity, increased therapeutic properties, extended circulation time in the bloodstream, and other therapeutic benefits (Wilson et al., 2022). This is due to their small particle size, typically about 200 nm. Compared to liposomes, solid lipid nanoparticles (SLNs) and polymeric nanoparticles, the nanoemulsion exhibit superior drug stability and extended release, they are safer due to the lack of organic solvents used during the formulation process. Liposomes utilize phospholipid bilayers, whilst nanoemulsions employ surfactant-stabilized droplets, influencing their appropriateness for certain medications (Kumar et al., 2024; Viegas et al., 2023).

Multiple studies illustrate the therapeutic effectiveness of Pumpkin Seed Oil Nanoemulsion (PSO-NE) in the management of benign prostatic hyperplasia, steatohepatitis, and peptic ulcer (Al-Okbi et al., 2017; Sindi and Hosny, 2019; Alhakamy et al., 2019). The cytotoxic activity of PSO-NE in breast cancer cells (MCF-7 and MDA-MB-231) was examined in our earlier study by Ghosh et al. (2025), which also provided mechanistic insights into cell cycle arrest and apoptosis. However, no PSO-NE has been investigated in vitro or in vivo prostate cancer studies. This study’s main goal was to evaluate the cytotoxic and apoptotic effects of PSO-NE on PC3 and LNCaP prostate cancer cell lines. Since apoptosis is not connected to the inflammatory system, it is the ideal method of cell death for anticancer medications (Weerapol et al., 2022). Nuclear fragmentation, phosphatidylserine exposure on the cell surface, and related mRNA expression levels were examined using DAPI staining, annexin V-FITC/DAPI staining, and qPCR, respectively, to determine the type of cell death caused by the PSO-NE.

Materials

For this investigation, we acquired Cucurbita maxima (pumpkin) seed oil of Indian origin from Cyrus Enterprises in Chennai, India. The cold press technique was used for oil extraction according to the manufacturer’s datasheet. We acquired the non-ionic surfactant Polyoxyethylene sorbitan monooleate (Tween 80) from Merck, India. MTT, DAPI, propidium iodide, RNase, and rhodamine were obtained from HiMedia Laboratories, India. The Annexin V-FITC Early Apoptosis Detection Kit was procured from eLabScience, India. We acquired the CFSE proliferation kit, Verso cDNA kit, SYBR Green real-time PCR master mix, and DNase I from ThermoFisher Scientific-Life Science Solutions, India. The Carbunn RNA Purification kit was acquired from JITM C Genes, UP, India. All water used in the technique was double-distilled using a Milli-Q system (Millipore Corporation).

Long term stability

PSO-NE was synthesized and characterized in our prior research. Ghosh et al. (2025), Briefly, the research developed PSO-NE with PSO, Tween 80 as surfactant, and MilliQ water. A coarse emulsion was first formed by combining 6% oil with different oil-to-surfactant ratios (1:1, 1:2, 1:3) at 500 rpm. The material was thereafter subjected to probe ultrasonication (750W, 13 mm probe, 50% amplitude) for 10–20 min. The cavitation energy generated by sonication decreased droplet size to the nanoscale. The resultant nanoemulsions were assessed for droplet size, shape, and stability prior to in vitro evaluation. The optimization of PSO-NE was affected by surfactant content and sonication time, resulting in a droplet size of 152.0 nm, PDI of 0.189 and Zeta Potential of −23.5 mV (Supplementary Material S1-S5). The ultimate optimum oil-to-surfactant ratio employed in the study was 1:3 and stored at 4 °C, and this composition was uniformly upheld throughout all characterization and biological investigations.

In our prior investigation, short-term and thermodynamic stability studies were performed but the PSO-NE’s long-term stability study was limited. Thus, a long-term study was examined in this study. For 6 months, equal volumes (10 mL) of optimized PSO-NE were placed in transparent screw-capped glass vials and left upright on the bench at room temperature (25 °C ± 2). The samples were regularly examined visually throughout this period to look for any indications of physical instability, including oil separation or creaming. To confirm the stability of the nanoemulsion particles, the droplet size, polydispersity index (PI), and zeta potential were then tested once again at the end of the storage period by Zeta Sizer Ver. 7.12 (Malvern instrument) at 25 °C.

To quantify stress-induced instability in nanoemulsions, we computed the percentage change in droplet sizes (d z) as follows: where d z,f represents the z-average droplet size of the nanoemulsion following the application of the stressor, and d z,i denotes the initial value of dz.

The equivalent quantity for fluctuations in ζ potential values is delineated as follows: where ζ f represents the zeta potential of the nanoemulsion after the application of the stressor, and ζ i denotes the initial value. In this study, negative values of %Δζ signify destabilization due to a movement in the ζ potential value towards zero from either direction.

Cytotoxic activity

Cell culture

The prostate cancer cell lines PC3 and LNCaP, together with the non-cancerous Vero cell lines, were obtained from NCCS (National Centre for Cell Science), Pune, India. The PC3 cells were grown in Ham’s F12K (HiMedia, Mumbai, India) medium, LNCaP cells were grown in RPMI medium (HiMedia, Mumbai, India) and Vero cells were grown in DMEM medium supplemented with 10% FBS (Fetal Bovine Serum, HiMedia, Mumbai, India) and antibiotics (penicillin and streptomycin) (100X, HiMedia, Mumbai, India) at 37 °C in an environment of 95% air and 5% CO 2.

Cytotoxicity assay

The MTT (3-(4, 5-Dimethyl-2-thiazolyl)-2, 5-diphenyl-2H-tetrazolium bromide) experiment was conducted to evaluate the cytotoxic effectiveness of PSO, formulated PSO-NE, blank NE (formulation of Tween 80 and MilliQ without encapsulated PSO), and the control medication Doxorubicin on LNCaP and PC3 cell lines. The cytotoxicity of PSO-NE was assessed against non-cancerous Vero cell lines. The cells were seeded onto 96-well culture plates at a density of 1 × 10^4 cells per well and incubated for 24 h. Subsequently, the cells were subjected to PSO, Doxorubicin, PSO-NE, and blank NE at various concentrations (20, 40, 60, 80, 100, 150, 200, 250 μg/mL) and incubated for 24 h. Following this, 10 μL of MTT solution (5 mg/mL in PBS) was added to each well and incubated for 4 h. Subsequently, the MTT was removed, and the resulting formazan crystals were dissolved in 100 μL of DMSO, followed by gentle agitation. Absorbance was measured at a wavelength of 570 nm using the microplate reader (Spectra Max 340 PC, Molecular Devices, United States). The study was performed thrice, and the IC 50 values were calculated by plotting the percentages of cell viability for prostate cancer cells versus the sample concentrations using GraphPad Prism 9 software. The calculated IC 50 concentrations of PSO-NE and PSO were applied to all subsequent experiments.

Cellular uptake study

Cancer cells (3 × 10^5 cells/well) were cultured in 35 × 60 mm plates on sterile 18 mm round cover slips (Deckglässer) and incubated for 24 h. After incubation, the cells were treated with 17 μL of the 1 mg/mL Rhodamine-B loaded PSO-NE (Rhod-PSO-NE) and PSO (Rhod-PSO), incubated for 3–4 h, and then underwent 2-3 washes with PBS. The cells were fixed with a 4% aqueous paraformaldehyde solution, washed with PBS, and then examined using confocal microscopy (Leica DMRE) equipped with a TCS-SPE confocal head (Leica, Wetzlar, Germany). Image analysis was performed utilizing ImageJ/Fiji (NIH, UNITED STATES) to analyze Mean fluorescence intensity (MFI). Initial raw photos underwent background subtraction, with a fixed radius evenly applied across all groups. Images were subsequently converted to 8-bit format, and a consistent threshold was used across all samples to guarantee impartial segmentation. Whole-cell segmentation was executed via the “Analyze Particles” tool, with size exclusion criteria established to exclude trash while including all intact cells inside the field.

The % uptake was calculated as:

Fold increase in uptake:

Nuclear morphology assay

The DAPI experiment was conducted to detect morphological alterations in the nucleus. In this study, both the control cells and the PSO-NE and PSO treated cells (at the IC 50 concentration) were first fixed using methanol. Subsequently, DAPI (1 μg/mL) was introduced to the cells and kept at 37 °C for 15 min. The modifications in the nuclear morphology were examined and observed using confocal microscopy (Leica DMRE) outfitted with a confocal head TCS-SPE (Leica, Wetzlar, Germany). The quantitative analysis of DAPI-stained nuclei was conducted using ImageJ/Fiji (NIH, United States). Following background correction and consistent thresholding, nuclei were automatically segmented, mean fluorescence intensity (MFI) were quantified to assess apoptosis-related chromatin condensation.

Cell proliferation assay

In this investigation, cells were introduced at a density of 1 × 10 6 cells/mL in 25 cm 2 flasks and allowed to incubate for 24 h to promote adhesion. After incubation, the cells were trypsinized and placed into 15 mL polypropylene tubes and rinsed 2-3 times with 1X DPBS. The CFSE (Carboxyfluorescein diacetate succinimidyl ester) dye (5 μM) was introduced to the solution and incubated at 37 °C for 15 min in a water bath. The material was then centrifuged at 1000 rpm for 5 min. The supernatant was discarded, and the pellet was mixed with 10 mL of complete medium. The combination was then centrifuged, and the supernatant was removed. The pellet was carefully mixed and re-suspended in entire medium. The CFSE-labeled cells were then positioned in a six-well plate and incubated for 24 h. Following incubation, cells were administered PSO-NE and PSO at the IC 50 concentration and let to incubate for 24 h. Thereafter, the CFSE dye-labeled cells underwent cytometric examination with the BD-LSR II flow cytometer.

Apoptosis analysis

In this study, cells were inoculated at a density of 1 × 10 6 cells/mL in 25 cm 2 flasks and incubated for 24 h to facilitate adhesion. After incubation, cells were administered PSO-NE and PSO at the IC 50 concentration, with untreated cells acting as controls. Post-treatment, both control and treated cells (1 × 10 6 cells/mL) were collected and re-suspended in 500 µL of Annexin-V binding buffer. 5µL of Annexin-V FITC and propidium iodide were added to each tube. The samples then underwent incubation in darkness at ambient temperature for 15–20 min. Apoptosis analysis was conducted with a BD LSR II Flow Cytometer, with the acquisition of 25,000 events per sample. The apoptotic distribution was examined using FlowJo_v10 software.

Cell cycle analysis

Cell-cycle analysis was conducted by assessing the DNA content in both treated and untreated cells by flow cytometry after propidium iodide (PI) staining. In this experiment, cells were inoculated at a density of 1 × 10^6 cells/mL in a 25 cm 2 flask and incubated for 24 h. Following incubation, cells were administered the IC 50 concentration of PSO-NE and PSO, and then incubated for an additional 24 h. Subsequently, the cells were trypsinized and subjected to centrifugation at 12,000 rpm for 10 min. The supernatant was removed, and 4 mL of 70% ethanol was added to the pellet, which was then incubated on ice for 30 min for fixation. Subsequently, the cells underwent centrifugation, the supernatant was discarded, and the pellet was reconstituted in 200 μL of PBS. 50 μL of RNase was added to the solution and maintained on ice for 30 min to facilitate RNase activity. Subsequently, 5 μL of propidium iodide was introduced and incubated for 5 min in dark. The resultant mixture was then examined with the BD-LSR II Flow Cytometer. Each sample recorded 25,000 instances. Cell cycle arrest was evaluated with FlowJo_v10 software.

Quantitative reverse transcription-polymerase chain reaction (qRT-PCR)

Extraction of Total RNA: Total RNA was extracted from Untreated (Control), PSO-NE, Doxorubicin and PSO treated cells (at IC 50 concentration) at a density of 1 × 10 6 cells using the Carbunn RNA Purification kit, following the manufacturer’s guidelines. The assessment of RNA content and quality was performed using the Nanodrop 2000c Spectrophotometer (Thermo Fisher Scientific), emphasizing the 260/280 ratio.

Transcription of RNA to complementary DNA (cDNA): First-strand cDNA was synthesized from 600 μg of total RNA using a Verso cDNA Kit in accordance with the manufacturer’s instructions.

Real-Time Polymerase Chain Reaction assays: The Real-Time PCR experiments were conducted with the SYBR Green PCR kit, accompanied by unique forward and reverse primers for each gene sourced from Integrated DNA Technologies (IDT). The nucleic acid sequences for the forward (F) and reverse (R) primers targeting the Caspase 3 (Cas3), p53, Bax, Bcl-2, NF-kB, and STAT-3 genes, with GAPDH as a housekeeping gene, are shown in Table 1 (Ghosh et al., 2025). The real-time PCR was performed using SYBR green dye chemistry on the CFX96 Touch Real-Time PCR Detection System (UNITED STATES). The 10 μL real-time PCR experiment consisted of 5 μL of Maxima SYBR Green qPCR Master Mix (Thermo Fisher Scientific), 0.5 μM of each forward and reverse primer (see study Table 1), 2 μL of one-hundredfold diluted cDNA, and nuclease-free water (NFW). The thermal cycler parameters were (i) 95 °C for 5 min (initial denaturation), (ii) 39 cycles at 95 °C for 30 s, 62 °C for 30 s, and 72 °C for 30 s, followed by (iii) melt curve analysis throughout a temperature range of 55 °C–95 °C. The Ct values were determined using a consistent threshold value for all analyzed genes. The comparative ΔCt (2 −ΔΔCT) approach was used for quantifying comparative gene expression, with GAPDH serving as the normalization gene.

Statistical analysis

The research was performed in three biological and experimental duplicates. The GraphPad Prism program (Version 9.0) was used to ascertain statistically significant results (p < 0.05). An unpaired t-test was employed to compare two groups, whilst a one-way ANOVA with a Tukeypost test was utilized to analyze three groups. Data is presented as the mean value ±standard deviation (n = 3).

Long term stability

During 6-month storage, the PSO-NE showed an increase in the droplet size from 152.0 nm to 241.4 nm and PDI values exhibited a tight distribution, ranging from 0.189 to 0.342. as shown in Table 2 and Figure 1. After 6 months, the zeta potential values decreased from −23.5 to −18.2, becoming less negative. Visual inspection of the nanoemulsions at regular intervals during the storage period revealed no signs of opacity and phase separation.

Cytotoxicity assay

Prostate cancer cell proliferation was shown to be strongly inhibited by PSO-NE, with IC 50 values of 90.22 μg/mL for PC3 and 65.47 μg/mL for LNCaP. PSO-NE showed no detrimental impact on non-cancerous Vero cells. PSO’s cytotoxicity, demonstrated by its IC 50 value, indicates that 50% of LNCaP and PC3 cells must be eliminated at doses of 254.03 μg/mL and 300 μg/mL, respectively (Figure 2). There was no inhibitory effect of blank NE on prostate cancer cells.

Uptake study

Rhod-PSO-NE was regularly and effectively absorbed inside the cytoplasm of PC3 and LNCaP cells, as shown in Figure 3. Cells treated with Rhod-PSO-NE exhibit a greater fold increase in cellular uptake compared to those treated with Rhod-PSO, with increases of 2.86 in PC3 and 2.82 in LNCaP, respectively.

Nuclear morphology assay

The nuclear morphological differences between control, PSO-treated, and PSO-NE-treated cells were shown by DAPI nuclear staining. Figure 4 and Figure 5 showed that both PC3 and LNCaP cells have condensed nuclear material in cells treated with PSO-NE. Contrary to the smooth nuclei seen in control and PSO-treated cells, the dense nuclear morphology in PSO-NE treated cells demonstrates apoptotic changes. Moreover, PSO-NE treated cells shows higher MFI than control and PSO treated cells.

Cell proliferation assay

The CFSE fluorescent dye was used to validate the results of the MTT experiment in the cell proliferation analysis. Figure 6 depicts the proliferation pattern of PSO-NE tagged with CFSE in the treated cells in comparison to the control and PSO-treated cells. The cell proliferation rate decreased from 98.5% to 75.6% in control and PSO-treated cells, respectively, to 52.2% in PSO-NE treated PC3 cells, and from 97.3% to 70.2% in control and PSO-treated cells, respectively, to 56.6% in PSO-NE treated LNCaP cells.

Apoptosis analysis

Compared to the control cells, the PSO-NE treated cells showed a much higher proportion of PI-Annexin-V FITC + cells, which indicate early apoptosis, and a lower percentage of PI-Annexin-V FITC cells, which represent alive cells. The PSO-NE treated PC3 and LNCaP cells showed an increase from 0.42% to 63.28% and from 0.05% to 58.44%, respectively, in contrast to the PSO-treated cells, where the percentage of cells in the early stages of apoptosis increased from 0.42% to 17.90% and from 0.05% to 10.82% in the treated PC3 and LNCaP cells, as shown in Figure 7.

Cell cycle analysis

The study showed that the proportion of cells in the G0/G1 phase rose from 52.7% in control cells to 81.3% in PSO-NE treated PC3 cells. The proportion of cells in the G2/M and S phases dropped from 22.9% to 24.5% in control cells to 8.3% and 11.3% in PSO-NE treated PC3 cells. The proportion of cells in the G0/G1 phase rose from 57.9% in control cells to 76.9% in PSO-NE treated LNCaP cells. In contrast, the proportion of cells in the G2/M and S phases decreased from 21.7% to 20.4% in control cells to 8.3% and 18.3% in PSO-NE treated LNCaP cells, respectively (Figure 8). The results from the cell cycle investigation demonstrate that both cell lines underwent cell cycle arrest in the G0/G1 phase. In PSO-treated PC3 cells, the G0/G1 phase grew to 60.5%, whereas the S and G2/M phases rose to 20.3% and 19.2%, respectively. Moreover, in PSO-treated LNCaP cells, the G0/G1 phase rose to 65.9%, while the S and G2/M phases increased to 19.4% and 14.7%, respectively.

Quantitative reverse transcription-polymerase chain reaction (qRT-PCR)

In the real-time PCR analysis of PC3 and LNCaP cell lines, the expressions of genes associated with cell proliferation (Bcl-2, STAT-3, and NF-kB) were significantly downregulated, whereas the expressions of genes linked to apoptosis (p53, BAX, and Caspase 3) were significantly upregulated in comparison to the untreated and PSO-treated cell lines, as depicted in Figure 9.

This study examines the long-term stability of PSO-NE. PSO-NE exhibited a rise in droplet size and polydispersity index (PDI), and a reduction in zeta potential as shown in Figure 1. A high PDI and droplet size indicated diameter variability across particle populations compared to the fresh sample due to deterioration, aggregation, or disintegration (McClements and Rao, 2011). The observed rise in droplet size and polydispersity index after 6 months likely indicates progressive phenomena such as droplet coalescence or Ostwald ripening. Ostwald’s ripening arises due to the migration of oil molecules from smaller to bigger droplets across the intervening fluid (McClements and Rao, 2011), potentially accounting for the modest increase in particle size during storage. The system did not display apparent phase separation or creaming. Similar results were observed by Lozada et al. (2021) that nanoemulsion of three species of pumpkin seed oils exhibited an increased droplet size and PDI after 120 days of storage, irrespective of temperature and by Zainuddin et al. (2019) which showed that after storage for 60 days, a slight increase in terms of particle size and PDI was observed in palm oil nanoemulsion.

Conversely, Zeta potential is a stability metric related to the surface potential of emulsion particles. After 6 months, the zeta potential values decreased from −23.5 to −18.2, becoming less negative. Similar results observed by Lozada et al. (2021) that nanoemulsion of three species of pumpkin seed oils showed negative values from Zeta-potential in the range of −18 to −31, far from zero after 120 days of storage. Droplets of nanoemulsions with zeta potentials below ±30 mV demonstrate only satisfactory short-term stability, with a propensity for particle aggregation (Mishra, Shaal, Müller and Keck, 2009).

The cytotoxic efficacy of PSO-NE against prostate cancer cells was assessed using the MTT test. Figure 2 demonstrates a substantial (P < 0.05) dose-dependent reduction in the proliferation percentage of LNCaP and PC3 cancer cells after a 24-h treatment with PSO-NE compared with untreated control cells. The PSO-NE formulation had a superior cytotoxic impact on the cell lines compared to either Doxorubicin or PSO when used independently. The enhanced antiproliferative action of PSO-NE is ascribed to its capacity to improve cellular uptake and facilitate the transport of PSO. Noteworthy that the long-chain fatty-acid elongase has been demonstrated to be overexpressed in prostate cancer and is essential for the steroid synthesis, including androgens (Santos and Schulze, 2012). While the overexpression of elongases has not been documented in other tumors, hence the prostate cancer cells may internalize and metabolize these lipid components more effectively, hence augmenting cytotoxic effects. The findings from previous study of Ahmad et al. (2017) indicate that rutin nanoemulsion therapy drastically decreases the viability of PC3 cancer cell lines in a dose-dependent way and study of Botelho & Queiroz. (2024) indicated a dose-dependent effect of testosterone nanoemulsion on the metabolic activity of PC3 and LNCaP cells. The principal disadvantage of chemotherapeutic drugs is their lack of selectivity, leading to detrimental effects on healthy tissues (Abd-Rabou and Edris, 2022). Our findings demonstrate a significant cell-specific cytotoxic effect on LNCaP and PC3 cells and significant moderate cytotoxic effect is seen in normal Vero cells under the same treatment conditions. Moreover, blank NE did not have an inhibitory impact on prostate cancer cells, indicating that the reported cytotoxic effects are due to PSO rather than the delivery vehicle.

IC 50 values in this range (65–90 μg/mL) align with those documented for other plant-derived oils and extracts assessed in cancer cell models (Atanasov et al., 2021). From a nanomedicine standpoint, the formulation aims to enhance pharmacological efficacy and safety, hence underscoring its potential translational significance as a supplemental or adjunct treatment approach (Patra et al., 2018). To elucidate the biological significance of the observed cytotoxicity, a selectivity index (SI), defined as the ratio of IC 50 in normal cells to IC 50 in cancer cells, was computed. The SI is a commonly utilized metric for assessing the therapeutic window of anticancer drugs, since it offers a comparative assessment of tumor selectivity beyond mere IC 50 values (Rayan et al., 2017). In this investigation, SI values of PSO-NE for PC3 and LNCaP is 5 and 6.15, respectively. Compounds with SI values above 2 are typically regarded as having preferred cytotoxicity against cancer cells and a more advantageous safety profile (Kaminsky et al., 1996).

Studies indicate that nano formulations with a surface charge ranging from −20 to −50 mV may be internalized by cells, facilitating their entry into cellular compartments via the endosomal route (Cunha et al., 2020). Moreover, the incorporation of lipids in the fabrication of nanoemulsions augments the interaction with cell membranes, enhances the permeability of compounds (Niu et al., 2016). In our findings, Rhod-PSO-NE was uniformly and effectively internalized throughout the cytoplasm of PC3 and LNCaP cells (Figure 3) and show greater fold increase in cellular uptake compared to those treated with Rhod-PSO suggesting emulsification of PSO enhance the bioavailability of oil. Our findings align with a previous study of Choudhury et al. (2014) that Paclitaxel nanoemulsion exhibited enhanced uptake of paclitaxel in breast cancer cells. Bayoumi and Alkhatib (2022) also had demonstrated that incorporation of topotecan into sesame oil nanoemulsion enhances cellular accumulation of topotecan in MCF-7, HCT116 and HepG2 cells.

DAPI is a nuclear morphology indication often used to differentiate apoptotic cells based on the size and roundness of the nucleus (Pajaniradje et al., 2014). Our study reveals that the condensed nuclear morphology indicates apoptotic alterations in PSO-NE treated cells, contrasting with the smooth nuclei seen in control and PSO treated cells (Figures 4, 5). Our study aligns with the previous results of Shanmugapriya et al. (2019) and Khan et al. (2025). They observed more shrunken nuclei and nuclear fragmentation, membrane blebbing and apoptotic bodies when cells treated with nanoemulsion formulation of Chicorium intybus L. seed extract on human breast cancer cells and astaxanthin nanoemulsion against colorectal cancer, respectively.

During cell division, the CFSE fluorescence intensity (FI) is almost uniformly distributed throughout the two subsequent daughter cells. Thus, the measurement of FI functions as an indirect evaluation of cellular divisions (Banks et al., 2011). In our investigation, the fluorescence intensity (FI) decreased in cells treated with PSO-NE, compared to those treated with PSO and the control cells, showing a reduction in cell proliferation after PSO-NE treatment (Figure 6). These results were congruent with previously published observation of Sharma et al. (2025) that myricetin nanoemulsion significantly reduces the rate of TNBC proliferation and enhances tumor cell death.

An effective method used by cytotoxic medications is the activation of apoptosis and cell cycle arrest to impede cellular proliferation (Salehi et al., 2022). Annexin V-FITC examined the cell death pathways caused by PSO-NE, focusing on early and late apoptosis. This research observed that most cells had experienced apoptosis (Annexin V+/PI−) or had already succumbed to death (Annexin V+/PI+) after the treatment of prostate cancer cells with IC 50 value of PSO-NE as compared to PSO treated cells (Figure 7). The findings may stem from the augmented accumulation of PSO and improved penetration of the nanoemulsion, which might have altered the expression of many cellular genes linked to the apoptotic pathway, leading to heightened cytotoxic effects on cancer cells (Farhoudi et al., 2022). Similar results have been reported by Al-Otaibi & AlMotwaa. (2022) that the percentage of apoptosis in HT-29 cells treated with 0.3 µM Oxaliplatin was significantly lowered, when compared with cells treated with Oxaliplatin-loaded nanoemulsion (p ≥ 0.05).

The G0/G1 and G2/M phases of the cell cycle serve as primary checkpoints and are essential for cell cycle control (Al Saqr et al., 2022). Our observations indicate that compared with PSO treated cells, PSO-NE treated halted the cell cycle at G0/G1 in prostate cancer cells (Figure 7) by obstructing the transition from G0/G1 to S, thus leading to apoptosis induction (Figure 8). Similar results observed by Alotaibi et al. (2023) that after treatment with a metformin-loaded coconut oil-based nanoemulsion, MCF-7, HepG2 and HCT-116 cancer cells experienced impediments in their cell cycle progression, resulting in accumulation in the G1 phase and by Etminan et al. (2022) that cellular arrest is only found in the SubG1 phase, indicating the incidence of apoptosis in cells subjected to Apricot Kernel Oil nanoemulsion treatment.

The assessment of apoptosis frequency in both PSO-NE treated and untreated cells indicated that PSO-NE significantly elevated the rate of cell apoptosis. The elevation of cell apoptosis indicates that PSO-NE impede the cell cycle and trigger cell death. To explore this theory, the expression of apoptotic and anti-apoptotic genes in cells treated with PSO-NE were examined. The real-time PCR experiment indicated that the expression of the apoptotic genes BAX, p53 and Cas 3 in cells treated with PSO-NE was significantly upregulated, while proliferative genes STAT3, NFκB, and Bcl-2 was significantly downregulated as compared to untreated and PSO treated cells. This data was also supported by the same previous study of Abd-Rabou and Edris (2022), which indicate that frankincense essential oil nanoemulsion specifically induces lung cancer apoptosis through activation of apoptotic genes (DR5, FAAD, Caspase 8, p53, and Bax) and inhibition of anti-apoptotic genes (Bcl-2, NF-kB, and STAT-3).

PSO-NE showed highly effective for PSO delivery to prostate cancer cells, since they have proven to be safe and capable of increasing PSO bioavailability, hence enhancing therapeutic efficacy. However, The PSO-NE instability may occur during on-site storage in long-term applications. Therefore, extended stability studies (12–24 months) in accordance with ICH criteria are necessary for long-term commercialization. Mohite et al. (2023) also stated the similar limitations for the nanoemulsion application in long-term application.

This study’s greater absorption efficiency indicates potential for better treatment results in patients. Considering that PSO is often deemed safe and has an application in prostate cancer, its nano formulation may enhance clinical translation (Nasrullah et al., 2025). Nonetheless, comprehensive in vivo pharmacokinetic, biodistribution, and toxicity investigations are necessary prior to clinical use. The use of PSO-NE in oral dosage forms may provide health advantages due to the benefits of polyunsaturated fatty acids. Gavin et al. (2015) had investigated that lipid-based nanoemulsion nanocarrier systems loaded with the drug genistein were developed into buccal tablet formulations using chitosan as a mucoadhesive interface and showed significant anticancer efficacy against oropharyngeal carcinomas. Thus, this study’s findings provide more insights into the use of PSO-NE as a possible adjuvant formulation for prostate cancer therapy. However, more research is required to evaluate and validate the safety and cell-dependent anticancer efficacy, including studies on other cell lines and focused in vivo investigations.

This in vitro research demonstrates the cytotoxic and apoptotic effects of Pumpkin seed oil nanoemulsion on prostate cancer cell lines. The formulated PSO-NE increased the cytotoxicity and bioavailability of the pumpkin seed oil, as shown by the inhibition of cell growth, reduction of IC50, promotion of apoptosis and cell cycle arrest, and upregulation of apoptotic gene expression. PSO-NE demonstrated safety on normal cells, indicating selective cytotoxicity.