Evaluation of the Japan urgent stroke triage score in a North American population

Introduction

There are two main categories of acute stroke, ischemic and hemorrhagic, with 87% being ischemic in nature (1). Hemorrhagic stroke encompasses both intracranial hemorrhage (ICH) and subarachnoid hemorrhage (SAH), with their estimated incidences in the United States at 10% and 3%, respectively (1). Ischemic and hemorrhagic strokes require highly specialized and procedurally different treatments. Intravenous (IV) thrombolysis remains the standard of care for most ischemic stroke patients, and mechanical thrombectomy evaluation and treatment should occur as rapidly as possible for those with a large vessel occlusion (LVO) (2). In contrast, American Heart Association (AHA) guidelines for the treatment of hemorrhagic stroke focuses on blood pressure control, immediate reversal of anticoagulation therapy, and potential neurosurgical intervention for hematoma evacuation or decompression craniotomy (3).

In the United States, there are a variety of prehospital stroke screening tools, including the Cincinnati Prehospital Stroke Scale (CPSS), Miami Emergency Neurologic Deficit stroke exam (MEND), and Face Arm Speech Test (FAST). However, none distinguish between ischemic and hemorrhagic strokes. The RACE screening tool has been shown to detect intracranial hemorrhage in addition to LVOs, but it does not distinguish between stroke types (4). Given different treatment capabilities of receiving hospitals, with only some stroke centers being capable of mechanical thrombectomy or other neurosurgical interventions, a reliable screening test to distinguish stroke types may help ensure more targeted hospital transport and better patient outcomes.

In 2018, Uchida et al. (5) identified 21 variables that were used to distinguish between LVO, SAH, and ICH in the prehospital setting. These variables formed the Japan Urgent Stroke Triage (JUST) score (Figure 1). JUST was subsequently broken down to seven key variables that maintained the predictive ability of the full JUST score (6). When compared to the CPSS, the JUST-7 score showed a superior predictive ability for acute LVO, and it also was able to differentiate between hemorrhagic and ischemic strokes (6). The objective of this study was to reconstruct a stroke screening scale that could reliably differentiate between hemorrhagic and ischemic strokes in a North American population using the JUST and JUST-7 variables.

Methods

Study design

This was a retrospective study of prehospital encounters from 44 unique EMS agencies transporting to 31 different hospitals across Colorado and southern Wyoming from January 1, 2020, to December 31, 2022. The population included any adult patient (aged 18 years or older) who: (1) was identified in the prehospital setting of having a primary or secondary impression of cerebrovascular accident (CVA), stroke, or transient ischemic attack (TIA), or in whom a prehospital stroke alert was called and (2) had a final hospital diagnosis of stroke or TIA (Supplementary Figure 1). This study was approved by the Colorado Multiple Institutional Review Board (COMIRB #23-10-35) and adhered to the principles of the Declaration of Helsinki.

Data collection

Variables were manually extracted from the prehospital and in-hospital electronic health record (EHR) and local Get with the Guidelines-Stroke (GWTG-S) registry data via standardized data collection forms created and validated in a HIPAA-compliant, secure Research Electronic Data Capture (REDCap) database. Data was extracted by trained student researchers. As part of the training process, at least 30 charts were extracted by all students and compared for accuracy. Data extractors had to reach over 90% accuracy before they were allowed to extract data independently. In addition to the components of the JUST Score (Figure 1), patient demographics, the primary and secondary impression, computer assisted dispatch (CAD) times (call received, dispatch, en-route, on-scene, at-patient, transporting, at-destination), prehospital treatments, and clinical presentation were extracted from the prehospital EHR. Signs and symptoms, which were extracted from the prehospital documented physical assessment and narrative, included unilateral limb weakness, facial droop, speech changes, numbness, dizziness, altered mental status, headache, nausea/vomiting, vision changes, generalized weakness, and gait disturbance. Unilateral spatial neglect and dysarthria (compared to aphasia) were not consistently identified and, thus, were not reported. Additionally, we were unable to consistently distinguish between paralysis of the upper or lower limbs and instead focused on unilateral or generalized weakness. Presence of an arrythmia was extracted when a cardiac rhythm was documented in the prehospital EHR, or a patient was documented to have a history of atrial fibrillation in the GWTG-S registry or in-hospital EHR. This was the only JUST variable that was also determined based on the in-hospital EHR when it was not available in the prehospital EHR. While this may create leakage and improve model performance, this was considered a necessary proxy given that the heart rhythm is obtained and reviewed by providers in many cases with suspected stroke (i.e. prehospital completion of ECG), but the specific rhythm not reliably documented. The following additional variables were extracted from the GWTG-S registry or in-hospital EHR: patient demographics, past medical history, initial National Institutes of Health Stroke Scale (NIHSS) score, discharge disposition, receiving facility stroke certification level, and the time of last known well (LKW), arrival, and computed tomography (CT).

Data analysis

Strokes were classified as either ischemic or hemorrhagic based on hospital records. If a stroke was unable to be classified, it was removed from the data set. Separate complete data was obtained for the full JUST variables, the JUST-7 variables and for JUST variables plus sex, numbness, comorbidities, and AHA guidelines. The final model was selected to help account for known sex-based differences in hemorrhagic and ischemic stroke incidence and clinical presentations, differences in risk factors and presentations associated with hemorrhagic and ischemic strokes, and the supposition that better AHA guideline compliance may also be associated with more detailed stroke assessments which could facilitate better discrimination between hemorrhagic and ischemic strokes (6–9). Variables were compared between stroke types using a chi-square/Fisher's exact association test for categorical variables, and with two sample T-tests or Wilcoxon rank sun tests for continuous or scale variables. Logistic regression models for hemorrhagic vs. ischemic stroke were fit on encounters with complete data using three different sets of explanatory variables: (1) JUST-7, (2) JUST, and (3) JUST, plus sex, comorbidities, and AHA guidelines. For the third model, the final set of variables was determined using backwards elimination with an alpha = 0.10. EMS sudden onset, improve, and progress were omitted as they had only one level. Race was considered as a starting variable for model 3 but was rejected because many of its values were missing, which would have reduced the effective sample size, without providing much explanatory value. Only encounters with complete data were included in the analyses. Missing data was not imputed to prevent adding additional complexity to the models. Missingness is reported in Supplementary Table 1. For the JUST and JUST-7 Score variables and past medical history, if it was not documented in the prehospital or hospital EHR, it was considered to not to be present. Receiver Operating Characteristic (ROC) curves and area under the curve (AUC) statistics were obtained for the full model for each set of explanatory variables, and AUCs of the full models were compared. Odds ratios and 95% confidence intervals were obtained for each of the explanatory variables in each model: two-sided univariate alpha = 0.05. An omnibus test was performed for the entire model. Because an ideal test to differentiate ischemic and hemorrhagic strokes in the prehospital setting would balance sensitivity and specificity, a Youden's (J) index was calculated for each model, representing the maximal sum of both sensitivity and specificity for the model.

Results

Study population

Out of 2,981 patient encounters, 85.7% (n = 2,555) of patients were diagnosed with an ischemic stroke and 14.3% (n = 426) were diagnosed with a hemorrhagic stroke (Table 1). Patients presenting with a hemorrhagic stroke were younger [mean age 68.8 years, standard deviation (SD) 14.9] than patients diagnosed with an ischemic stroke (mean age 73.3 years, SD 14.0) (P < 0.0001). A history of dyslipidemia was more common in patients with ischemic strokes compared to those with a hemorrhagic stroke (43.6% versus 36.4%) (P = 0.0008). Patients with a hemorrhagic stroke had a higher initial NIHSS score [median 13, interquartile ration (IQR) 6–21] compared to patients with an ischemic stroke (median 7, IQR 3–14) (P < 0.0001). The mean EMS documented systolic blood pressure (SBP) was 176mmHg (SD 37.2mmHg) in patients with a hemorrhagic stroke compared to 157.6mmHg (SD 32.6mmHg) in patients with an ischemic stroke (P < 0.0001). Only 12.9% (n = 55) of hemorrhagic stroke patients were discharged home and 21.4% (n = 91) died. Comparatively, 37.2% (n = 950) of ischemic stroke patients were discharged home and 5.4% (n = 138) died (P < 0.0001).

Table 1. Subject characteristics.

^*P < 0.05 by chi-square/Fisher's exact association test or by T-test.

Abbreviations: TIA, transient ischemic attack; CAD, coronary artery disease; CHF, congestive heart failure; CKD, chronic kidney disease; ESRD, end stage renal disease; NIHSS, National Institutes of Health Stroke Scale; CT, computed tomography; UTD, unable to determine.

Patient characteristics in original JUST prediction rule

Patients with hemorrhagic and ischemic strokes differed with regards to the JUST Score variables (Table 2). A greater proportion of patients with an ischemic stroke had aphasia according to EMS clinicians (68.9% vs. 57.5%, P < 0.05). However, patients with hemorrhagic stroke more frequently presented with a headache (19.7% vs. 9.7%, P < 0.05), altered consciousness (46.0% vs. 36.5%, P < 0.05) or nausea and vomiting (22.3% vs. 7.5%, P < 0.05) compared to patients with an ischemic stroke. Similarly, a greater proportion of patients with a hemorrhagic stroke had an SBP≥ 165mmHg (59.6% vs. 38.4%, P < 0.05) or diastolic blood pressure (DBP) ≥95mmHg (54.0% vs. 36.5%, P < 0.05) compared to those with an ischemic stroke.

Table 2. Patient characteristics in original JUST prediction rule.

^*P < 0.05 by chi-square/Fisher's exact association test.

Differentiating hemorrhagic and ischemic strokes based on JUST score variables

Using a combination of the JUST Score variables resulted in a model with only moderate discriminatory ability (Figure 2a), with an AUC of only 0.699 (95% CI: 0.671–0.728). Maximizing the sum of the sensitivity and specificity for the model resulted in a J-index of 0.296 (sensitivity 50.2%, specificity 79.4%). Patients experiencing a hemorrhagic stroke had 1.44 (95% CI 1.07–1.94) times the odds of atrial fibrillation compared to ischemic strokes. Compared to patients with ischemic stroke, those with hemorrhagic strokes also had greater odds of presenting with an SBP≥ 165mmHg (OR 2.21, 95% CI 1.72–2.85), altered mental statues (OR 1.82, 95% CI 1.46–2.29), headache (OR 1.70, 95% C5 1.26–2.30), nausea or vomiting (OR 2.78, 95% CI 2.05–3.78) (Table 3).

Table 3. Differentiating hemorrhagic and ischemic stroke based on JUST score variables.

Abbreviations: CVA, cerebrovascular accident; DBP, diastolic blood pressure; SBP, systolic blood pressure; TIA, transient ischemic attack.

AUC: 0.699 (95% CI 0.671–0.728).

J-index: 0.296 (sensitivity 50.2%, specificity 79.4%).

Differentiating hemorrhagic and ischemic strokes based on JUST-7 score variables

Using components of the JUST-7 score (Figure 1) resulted in a model with a slightly lower discriminatory ability and AUC (0.677, 95% CI: 0.649–0.704) (Figure 2b). The J-index remained poor (0.299, sensitivity 72.3%, specificity 57.6%). Within this model, an SBP≥165mmHg (OR 2.45, 95% CI 1.98–3.03), altered mental status (OR 1.80, 95% CI 1.45–2.23), and presence of a headache (OR 2.32, 95% CI 1.75–3.08) were associated with greater odds of being diagnosed with a hemorrhagic stroke compared to an ischemic stroke. Presentation with aphasia was associated with 0.59 (95% CI 0.48–0.74) times the odds having a hemorrhagic compared to ischemic stroke (Table 4).

Table 4. Differentiating hemorrhagic and ischemic stroke based on JUST-7 score variables.

Abbreviations: AMS, altered mental status; CVA, cerebrovascular accident; DBP, diastolic blood pressure; SBP, systolic blood pressure; TIA, transient ischemic attack.

AUC (95% CI): 0.677 (0.649, 0.704).

J-index: 0.299 (sensitivity 72.3%, specificity 57.6%).

Differentiating hemorrhagic and ischemic strokes by symptoms, medical history, and AHA prehospital guidelines

Starting with the JUST Score variables and adding patient sex, presence of numbness, patient comorbidities, and components of prehospital stroke care, we sought to identify a potential superior model for differentiation between ischemic and hemorrhagic strokes using the variables commonly collected by prehospital providers. In this final model all available prehospital variables were included at onset and then extraneous covariates were eliminated using backwards elimination with an alpha = 0.10. Despite being the highest AUC of the models, this model only achieved moderate discriminatory ability (AUC 0.719, 95% CI: 0.689–0.749) and a poor J-index (0.318, sensitivity 72.3%, specificity 59.5%) (Figure 2c). The presence of an SBP≥165mmHg, altered mental status, headache, nausea or vomiting, and histories of hypertension and chronic kidney disease were associated with greater odds of being diagnosed with a hemorrhagic compared to ischemic stroke (Table 5). Conversely, age, presentation with aphasia or dizziness, a histories of diabetes mellitus and dyslipidemia, and EMS documented stroke scale were associated with lower odds of having a hemorrhagic compared to ischemic stroke.

Table 5. Selective model identifying hemorrhagic vs ischemic stroke including symptoms, medical history, and AHA prehospital guidelines.

Abbreviations: CKD, chronic kidney disease; DBP, diastolic blood pressure; SBP, systolic blood pressure; SpO2, pulse oximetry.

AUC (95%): 0.719 (0.689, 0.749).

J-index: 0.318 (sensitivity 72.3%, specificity 59.5%).

Discussion

In Japan, the JUST and JUST-7 scores both showed good positive and negative predictive value in determining not only the presence of stroke, but also further classifying the stroke as LVO, ICH, or SAH (5, 6). In the United States, there exists no such scoring system or prehospital stroke scale that has been proven to differentiate between stroke subtypes. The retrospective application of the JUST and JUST-7 Score variables to this North American population resulted in clinical prediction model with only minimal discriminatory ability in differentiating between ischemic and hemorrhagic strokes, AUC 0.70 and 0.68, respectively. Evaluating all patient and encounter level factors using backwards elimination resulted in a 13-variable model with only modest discriminatory ability (AUC 0.72). While there is no set performance level that has been established a clinically actionable, implementation of a prehospital tool to differentiate between ischemic and hemorrhagic strokes would ideally match or exceed the AUC of the JUST and JUST-7 Scores in a Japanese population which exceeded an AUC of 0.8 in most cases (5, 6). These results are relatively poor, especially considering the performance of the JUST and JUST-7 scores in a Japanese population. This performance is likely multifactorial and potential explanations include, but are not limited to, population differences, EMS documentation differences, and variable mismatch.

The limited predictive ability of the JUST and JUST-7 Score variables in a U.S. population may not be unexpected given underlying differences in these populations both in terms of their stroke risk and their stroke systems of care. Asian populations have a higher proportion of hemorrhagic stroke compared to the general US population (7). Additionally, Asian patients are more likely to have diabetes mellitus and hypertension as stroke risk factors compared to other ethnicities (7). The data presented here included only 48 patients (1.6% of the total study population) who identified as Asian. Of these Asian patients, 22.9% experienced a hemorrhagic stroke. Comparatively, only 13.5% of Caucasian patients experienced a hemorrhagic stroke. While racial and ethnic differences between the two populations may contribute to differences within predictive models, this alone is unlikely to be responsible for the low discriminatory ability of the JUST and JUST-7 score in a US. population.

Additionally, American and Japanese EMS systems vary regarding EMS provider training and system structure. Prehospital providers in Japan are trained to assess for symptoms such as hemispatial neglect and conjugate gaze deviation. These symptoms are not part of a standard assessment for prehospital providers in the United States. Prehospital providers in the United States are taught to use simple prehospital stroke scales such as the CPSS and FAST when assessing for stroke. Neither of these scales assess more complex presentations such as neglect and conjugate gaze deviation, nor do they differentiate between dysarthria and aphasia. Additionally, this study was unable to assess for three variables initially characterized in the original JUST Score: sudden onset, progression, and improvement of symptoms. These characterizations are not routinely documented by EMS providers in the United States and were undetermined in over 90% of encounters. While these factors were characterized in the original JUST Score, they were not included in the final JUST-7 and are unlikely to explain the differences noted within our predictive model. Additionally, due to both the retrospective nature of this study and the differences in training and documentation between Japanese and United States EMS systems, there was variable mismatch within our analysis that could have further contributed to the performance of the model.

In the United States, patients with a hemorrhagic stroke tend to present to the prehospital clinician with higher blood pressures compared to patients with ischemic stroke (8). Recent research from China showed hypertension, headache, vomiting, facial droop, arm weakness, and slurred speech to be more frequently associated with hemorrhagic strokes (9). Similarly, our data revealed an association between high systolic blood pressure, headache, and nausea or vomiting and the presence of hemorrhagic stroke. However, only aphasia was not associated with increased odds of a hemorrhagic stroke within our study population. Unfortunately, not all symptoms that are associated with hemorrhagic strokes in both populations are routinely assessed in US. prehospital stroke screening tools. For example, the RACE score, which has been shown to detect ICH and LVOs, includes the symptoms: facial palsy, arm motor function, leg motor function, head and gaze deviation, as well as aphasia or agnosia (10). The CPSS assesses for facial droop, speech changes, and upper extremity weakness (11). However, neither assess for altered mental status, headache, or nausea and vomiting, all variables that showed strong correlation with hemorrhagic strokes within this study. This potential failure to recognize symptoms associated with hemorrhagic strokes can lead providers to miss these stroke patients, potentially impacting treatment and transport decisions. When EMS recognizes hemorrhagic strokes, providers spend less time on-scene and door-to-CT-times are shorter (12). Additionally, this study included patients who were identified by EMS as having a stroke and who also had a final diagnosis of stroke. This study did not include patients who were stroke mimics or strokes missed by EMS providers. Given that prehospital providers in the United States are trained primarily in LVO stroke detection, it is possible that these scores would perform better in a prospective study design.

The highest AUC attainable for this data set was 0.72. This model was derived from the JUST Score variables and adding patient sex, presence of numbness, patient comorbidities, and components of prehospital stroke care and narrowed down to variables that showed significance in predictive value. While this model showed better predictive value than chance alone, there is significant room for improvement to recapitulate the discriminatory ability achieved by the JUST and JUST-7 scoring systems. Previous research has indicated that there may be a benefit in transporting hemorrhagic stroke patients to a comprehensive stroke center (CSC) (13). Hemorrhagic stroke patients admitted to CSCs are more likely to receive neurosurgical and endovascular treatments and be alive at 90 days than patients admitted to other hospitals (13). Given this potential ability to improve patient outcomes and decrease mortality, it seems prudent to provide prehospital providers with an assessment tool that can, with relative accuracy, enable them to identify hemorrhagic strokes in the field and route these patients to facilities more capable of handling these emergencies.

The need for better field identification and routing applies to both ischemic and hemorrhagic strokes. It remains unknown as to whether transporting a suspected ischemic stroke to the nearest thrombolytic-capable facility or extending transport to take to a CSC for LVOs is the best option (14–17). However, without more definitive data, many current prehospital protocols prioritize transport to the nearest stroke center for patients with ischemic stroke in order to facilitate timely administration of intravenous thrombolysis. While intravenous thrombolysis is available at most emergency departments, only CSCs are guaranteed to have the specialty services needed for treatment of hemorrhagic strokes. Taking a suspected hemorrhagic stroke to a CSC and bypassing a primary stroke center could lead to quicker neurosurgical intervention and thus better patient outcomes. The frequency of inter-hospital transfer for hemorrhagic stroke is increasing and long-term prognosis after hemorrhagic stroke is critically dependent on the quality and speed of treatment a patient receives during the acute phase of care (18). Providing prehospital clinicians with a way to distinguish between ischemic strokes which may need to prioritize transport to the nearest hospital for intravenous thrombolysis versus hemorrhagic strokes which need the neurosurgical evaluation of CSCs can optimize patient transport to the most appropriate facility in cases of both ischemic and hemorrhagic stroke. Future prospective research is needed to determine the ability of the identified predictors of hemorrhagic stroke in helping make these important decisions and to better evaluate their impact of patient outcomes.

Limitations

This study is limited due to its retrospective design, which is heavily reliant on the accuracy of information documented in a patient's EHR. Due to the retrospective nature of this study, certain symptom substitutions and alterations were necessary. Any mention of vision changes or eye deviation was grouped under “conjugate deviation.” The original JUST study used dysarthria, whereas this study grouped all alterations to language under aphasia. Unilateral spatial neglect was not used in this study due to lack of provider training in assessing this sign. Additionally, the JUST study distinguished between paralysis of upper and lower limbs, whereas this study was unable to reliably distinguish between the two and instead used unilateral paralysis. Inclusion criteria for this study was limited to encounters with a suspected stroke in the prehospital setting and final diagnosis of stroke. This inclusion criteria eliminates stroke mimics as well as strokes missed in the prehospital setting from the dataset and has the potential to both inflate apparent performance and limit generalizability to actual EMS triage. Furthermore, the original JUST study focused primarily on Large Vessel Occlusion (LVO) for ischemic stroke whereas this study included both LVO and non-LVO in its characterization of ischemic stroke. The above substitutions and alterations likely alter the results to some capacity and limit the generalization of this study to the previous JUST and JUST-7 studies. Symptom characterization of sudden onset, progression, or improvement was not included in this model due to lack of consistent documentation. This study is limited geographically to two contiguous Mountain West states which also potentially limits generalizability to the entire United States.

Conclusions

In a North American population, the JUST and JUST-7 Score variables only had moderate predictive ability to distinguish between hemorrhagic and ischemic stroke. The best AUC attainable with this data set was 0.72. Patients with a systolic blood pressure ≥165mmHg, headache, altered mental status, and nausea or vomiting had a greater odd of having a hemorrhagic stroke in this study. Current prehospital stroke scales in the US. are not designed to detect nor distinguish hemorrhagic strokes from ischemic strokes. Prehospital providers need a tool that can allow them to triage stroke patients with relative accuracy to provide transport to the facility most equipped to handle their specific neurologic emergency.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/femer.2026.1812004/full#supplementary-material

Flow diagram of patient encounter inclusion in study. Supplementary material at the source

Supplementary material Supplementary material at the source

Supplementary material Supplementary material at the source

Footnotes

Edited by: Korakot Apiratwarakul, Khon Kaen University, Thailand

Reviewed by: Joanna Harazny, University of Erlangen Nuremberg, Germany

Yuriy Flomin, P. L. Shupyk National University of Health Care of Ukraine, Ukraine