Quality of systematic reviews on teaching English as a foreign language: a methodological review protocol

1 Introduction

This protocol describes a transparent and systematic approach for searching and assessing the methodological quality of existing systematic reviews (SRs) in the field of teaching English as a foreign language (i.e., non-native, second or additional). Throughout this review, the term EFL is used as an umbrella term encompassing English taught outside native-language contexts, including studies describing ESL, EAL and L2 when classroom instruction represents learning English as a non-native language. The need for this review stems from current pedagogical practice, which requires access to the best available evidence and its effective use in decision-making (Sedláčková et al., 2025). Teachers, researchers, policymakers, parents, school principals, and other pedagogical staff may benefit from the results of the proposed study through improved identification and interpretation of methodologically trustworthy systematic reviews.

Systematic reviews represent the primary type of evidence synthesis (White, 2019) and form the main pillars of evidence-based education (Kantor et al., 2022). According to Munn et al. (2018), they are “conducted by groups of reviewers with specialized skills who aim to identify and retrieve international evidence relevant to a specific question or questions and to appraise and synthesize the results of this research to inform practice, policy, and in some cases, further research” (p. 2). Because SRs provide more robust evidence than individual primary studies, educational policy should prioritize this type of study, provided they are high-quality and well-conducted.

Methodological advancement in evidence synthesis over the last 30 years has been driven by four leading international organizations: the Cochrane Collaboration, JBI, the Campbell Collaboration, and the GRADE Working Group. These organizations have established standardized methodologies for various types of SR and for assessing the evidence in the form of handbooks, manuals, and appraisal tools. These resources are regularly updated and publicly available, specifically the JBI Manual for Evidence Synthesis (Aromataris et al., 2024), JBI Checklist for Systematic Reviews (Aromataris et al., 2015), the GRADE Handbook (GRADE Working Group, n.d.), and the Cochrane Handbook series, e.g., the Cochrane Handbook of Systematic Reviews of Interventions (Higgins et al., 2022) or the Cochrane-Campbell Handbook for Qualitative Evidence Synthesis (Noyes and Harden, 2024).

For example, the JBI Manual for Evidence Synthesis describes recommendations for creating the following types of reviews (Aromataris et al., 2024): Although systematic review methodology originated in healthcare, it is now well established in education. Therefore, decision-making in pedagogical practice should prioritize high-quality evidence syntheses, both in educational practice and in education policy (White, 2019). This approach places greater emphasis on cognitive science than on traditional teaching methods, personal experience and opinions (Weinstein et al., 2018). Consequently, teachers may need to reconsider their instructional beliefs, leading to changes in their professional identities (Jones, 2024). Searching for SRs and interpreting them is demanding and impractical for teachers and other pedagogical staff in daily practice. Similarly to the whole concept of evidence-based education, if not understood and executed correctly, it may be seen as a drawback, instead of a facilitator (Holloway and Larsen Hedegaard, 2023). For this reason, some researchers and research organizations have created evidence overviews of existing pedagogical methods and strategies. Examples include John Hattie's well-known “Visible Learning” project (2008, 2023), numerous overviews of the most impactful educational approaches and methods (Friberg et al., 2024; Hott et al., 2021; Mitchell and Sutherland, 2020; Petty, 2018), as well as digital platforms and apps in the form of educational toolkits. Among these, the most internationally widespread is the Teaching and Learning Toolkit created by the British organization Education Endowment Foundation (2026). Most of these resources offer information on the effectiveness of pedagogical methods and strategies through the synthesis of existing primary studies.

• systematic reviews of effectiveness,
• systematic reviews of qualitative evidence,
• systematic reviews of economic evidence,
• systematic reviews of etiology and risk,
• mixed-methods systematic reviews,
• systematic reviews of text and opinion,
• umbrella reviews (systematic reviews of systematic reviews),
• scoping reviews or systematically processed overview studies (Peters et al., 2020).

If available, trustworthy SRs are the most appropriate and valid source of information for both practitioners and decision makers. However, numerous meta-studies from healthcare suggest that studies published under the title of “systematic review” may greatly differ in methodological quality (Campbell et al., 2017; Kelly et al., 2001; Pölkki et al., 2014; Van Der Pol et al., 2015), which reduces their utility for informing relevant users. Similarly, in the field of inclusive education, a recent methodological review (Li, 2025) found that many studies labeled SRs followed methodologies more typical of mapping reviews (such as scoping reviews) and were often biased by low adherence to established methodological and reporting standards.

That is why practitioners and policymakers should not only be able to identify relevant systematic reviews, but also to assess the methodological quality of these studies, in order to support their practice/policy by trustworthy evidence (Djulbegovic et al., 2015; Vukovic, 2021).

Language acquisition distinguishes between L1 (the native or first language) and L2 (any language learned after the first language). Therefore, the following areas of language acquisition can be differentiated: Our study focuses primarily on EFL, while also considering ESL/EAL. In the Czech Republic and many countries worldwide, the acquisition of the English language primarily involves EFL, with language proficiency typically described using the A1-C2 levels by the Common European Framework of Reference for Languages (Council of Europe, 2020).

• English as a Native Language (ENL)
• English as a Second/Additional Language (ESL/EAL)
• English as a Foreign Language (EFL)

There are fundamental differences in the mechanisms of teaching native and foreign languages. Native language acquisition is predominantly implicit and occurs in response to linguistic input and interactions in everyday activities (Guasti, 2017). In the EFL context, a combination of implicit mechanisms and explicit instruction is employed, utilizing various teaching methods and strategies based on curriculum-defined goals and approaches. At the highest theoretical level, an approach is understood as the fundamental philosophy of teaching, a set of theoretical principles regarding how we learn language. It thus determines the direction and principles of teaching, not its specific steps. A method is understood as the overall framework of teaching derived from the approach. It establishes teaching goals, the type of syllabus, the roles of the teacher and the student, materials, and methods of assessment. Within a method, teaching strategies, techniques, and activities are implemented as specific classroom procedures and practices (Anthony 1963; Larsen-Freeman, 2000; Richards and Rodgers, 2014, 2022).

English language teaching is rooted in various curricular documents. In the Czech Republic, English is a foreign language. The Framework Educational Programs for kindergartens, elementary schools, and high schools are the most important among these documents. These framework programs do not prescribe specific methods for teaching foreign languages, but rather define target competencies, placing strong emphasis on didactic principles, competency goals, and teaching strategies that teachers are expected to apply in the given level of education.

An example may be provided by the Framework Education Program for Primary Education (Národní pedagogický institut ČR, 2025c). The expectations of active communication and interaction, active student participation in class, gradual progression of language skills, differentiated approach to students with diverse needs, the use of language in authentic situations, and other examples may be found there. This creates opportunities for the use of methods such as Communicative Language Teaching, Total Physical Response, Task-Based Language Teaching, and Content and Language Integrated Learning. Furthermore, the model school curricula for English language teaching in primary and lower secondary education (Národní pedagogický institut ČR, 2025a, b) provide an overview of teaching strategies for achieving the target competencies in the Framework Education Program. Those include working with authentic materials and digital technologies (Klímová et al., 2023), games, drama activities, discussions, feedback, error correction, self-assessment, peer learning and assessment, and many others.

The emphasis on authentic interaction in the Czech Framework Education Program (Národní pedagogický institut ČR, 2025c) aligns with a broader international shift toward functional language use reflected in the Common European Framework of Reference for Languages (European Commission: European Education and Culture Executive Agency, 2023). National curricula, however, operationalize these principles in different ways. For example, the Norwegian curriculum places particular emphasis on critical reflection and critical literacy, whereas recent curriculum documents and analyses from other European contexts indicate varying interpretations of communicative language education and communicative competence (Ministry of Education of the Republic of Poland, 2020; Ministerio de Educación, Formación Profesional y Deportes, n.d.; Hällgren and Palm, 2025; Secretariat of the Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany, 2023; Schipor and Hammershaug, 2022). In the United States, policy and practice have increasingly emphasized evidence-based language and literacy support for English learners (National Center on Improving Literacy, 2022). Such variation highlights the value of systematic reviews, which provide transparent syntheses of research evidence and support educational decision-making based on empirical findings rather than tradition or personal preference (Sedláčková et al., 2025; White, 2019).

Resources based on evidence are available for teaching English, such as the Teaching and Learning Toolkit (Education Endowment Foundation, 2026) or Visible Learning (Hattie, 2023). However, both syntheses explicitly state that studies concerning the teaching of English as a foreign or second/additional language were not included.

Identifying this potential knowledge gap, we conducted a preliminary search in two databases (Epistemonikos and EBSCO host accessed from Palacký University Olomouc) to find evidence syntheses that could best inform educational practice in this area. We used the keywords “English language” AND “teaching OR learning” in the title/abstract (limited to SRs; publication period unlimited). Based on the preliminary search results, we formulated eligibility criteria for the studies to be included. The studies relevant to the Czech context were most frequently focused on ICT-supported instructions and digital tools, interventions related to reading skills, social and support interventions, pronunciation and the development of phonological awareness, and strategies for teaching writing. Other represented areas included gamification in English language instruction and the flipped classroom approach, motivational strategies, inquiry-based learning, peer-learning, vocabulary teaching strategies, and the use of humor. Examples of the studies identified in the preliminary search are provided in Table 1.

Table 1. Examples of studies retrieved from Epistemonikos.

Legend: EAL, English as a second/additional language; EFL, English as a foreign language; L2, second language; SR, systematic review.

authors reported these databases as examples (“such as…” or “like…”).

During the screening of the identified syntheses, we found that the studies varied greatly in their methodological quality. Many studies used “systematic review” in their titles, but their methodologies more closely resembled literature reviews. We have decided to conduct a new methodological systematic review because practitioners need to rely on most methodologically trustworthy evidence syntheses. In doing so, we first verified (via searches in the Epistemonikos database, the Cochrane Database of Systematic Reviews, JBI Evidence Synthesis, Web of Science and PROSPERO) that no methodological review of this type has been published to date, nor does a prospectively published study protocol exist on this topic.

The forthcoming methodological SR therefore aims to: a) identify existing SRs focusing on teaching English as a foreign (second/additional) language, b) evaluate the methodological quality of these reviews, and c) provide recommendations that could have a positive impact on the development of future SRs.

Review Question: Objectives:

• What is the methodological quality of systematic reviews published in the field of teaching English as a foreign language?

• To identify SRs on educational methods and strategies for teaching English as a foreign language.
• To examine the methodological quality of the identified SRs according to the JBI critical appraisal checklist (Aromataris et al., 2015).
• To provide recommendations for further research and practice.

2 Methods

The future review will follow the guidelines from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (Page et al., 2021). Any possible substantial deviations from this protocol will be reported in the full review.

2.1 Inclusion criteria

Participants: The planned SR focuses on children, pupils, and students learning the English language from pre-primary to higher education. Participants will be included regardless of school type, age, special educational needs, nationality, or socio-cultural background.

Concept: The concept of this study involves the teaching of English as a foreign language, including English as a non-native, second, or additional language. To be included, SRs need to provide information relevant to classroom-based English teaching (English self-study or study relying solely on digital technologies with no application in classroom teaching will not be included).

Context: The context will not be restricted in demographic and socio-cultural terms, nor by the type of educational institution.

Types of studies: We will include SRs of quantitative, qualitative, and mixed designs. To be included, SR will need to report, at minimum, a review question/objective, eligibility criteria for identifying relevant evidence, and a search strategy, without considering their appropriateness or methodological rigor. Reviews that contain a mapping-type question, such as scoping reviews, integrative reviews, critical reviews, literature reviews, and all types of non-systematic overviews, will be excluded.

2.2 Search strategy

Following JBI recommendations, a three-step search strategy will be used (Svobodová and Kantor, 2024), aimed at locating all published studies on the given topic. After the initial search on the Epistemonikos and EBSCOhost databases, examples of relevant studies were identified, and a search strategy will be tested to achieve an optimal balance of specificity and sensitivity. Next, search strategies will be developed for the following sources: Epistemonikos, Web of Science, APA PsycINFO (EBSCOhost), ERIC (EBSCOhost), OpenAlex, and ProQuest One Education. Generally, the search terms will contain “systematic review” in title/abstract, as well as “English as a foreign language”, “learning English”, “teaching English”, “English language learners”, “English language development”, together with synonyms and abbreviations of the search terms. The specific features of each search engine will be considered during this process; then a separate search strategy will be created for each information source (Supplementary material/Appendix 1). The publication period and language of publication will not be limited. Protocols or other records not providing a completed systematic review will not be included. After the identification of relevant studies, a tertiary search of their reference lists will also be conducted.

2.3 Study selection

Records from the search will be imported into Rayyan citation manager (Khabsa et al., 2015), and duplicates will be removed. Two reviewers will independently screen the records in two stages. First, they will assess the same ten records as a calibration exercise. If a pre-set threshold of Cohen's kappa of 75%, as an inter-rater agreement level, is not reached, the calibration exercise will be repeated until the level of agreement is satisfactory. In the first stage, the relevance of titles and abstracts will be assessed against the inclusion criteria. Later, a full-text screening of all potentially relevant studies will be performed. Disagreements during the screening process will be resolved by discussion or by an additional reviewer. There will be one final decision on each identified study. Reasons for the exclusion of any full texts will be referenced with justification in the final manuscript.

2.4 Data extraction

Before the data extraction phase, training sessions with reviewers will take place, during which the inclusion criteria will be studied and the information to be extracted will be discussed. This phase will use both included studies and studies not included in our review, so that the differences can be seen. The extraction will be carried out using an extraction tool developed by the authors, based on the JBI critical appraisal checklist (Aromataris et al., 2015).

To ensure precise data extraction, a pilot test will be conducted in which the two reviewers will use the form to extract data from a randomly chosen sample of the included studies. If needed, the extraction form will be amended before the data extraction from all other studies. Data from the relevant studies will be extracted in a similar manner to the screening phase by two independent reviewers. Possible disagreements will be resolved by an additional reviewer. We will extract the following information: Further, the following details concerning the methodological quality of SRs will be extracted (not assessed; in this stage, only Yes/No information or quotes of, e.g., review questions, will be reported. This descriptive data extraction records the methodological characteristics in each review, whereas the further methodological-quality assessment evaluates these characteristics using the JBI assessment tool):

• Study details (author, year, title of the study)
• Type/design of the SR
• Demographic and other study contexts, including educational environment, language environment, and sociocultural factors, as all these may be essential for a deeper understanding of the study results (Kvintová et al., 2024) – these will be extracted solely for descriptive purposes and potential subgroup analyses; they will not contribute to the methodological quality assessment

• Information on the prospectively published protocol
• The review/research question
• Inclusion criteria
• Search strategy
• Resources used to search studies
• Screening process
• Critical appraisal details
• Data extraction process
• Publication bias assessment
• Methods used for data synthesis
• Certainty of evidence assessment
• Formulated recommendations

2.5 Data analysis, presentation, and methodological quality assessment

The methodological quality of SRs included will be assessed by two independent reviewers using the JBI Checklist for SRs and Research Syntheses (Aromataris et al., 2015). This Checklist consists of 11 questions focused on the methodology of the SR (e.g., “Is the review question clearly and explicitly stated?”), which the assessors answer Yes, No, Unclear, or Not applicable: We chose the JBI checklist as the main aim of this methodological review is to assess the methodological quality of systematic reviews, not the effectiveness of educational interventions. It can be applied to systematic reviews regardless of the type of evidence. In educational literature, quantitative, qualitative, and mixed forms of evidence are often synthesized. Although the JBI checklist was developed within the healthcare context, we do not plan to adapt any of its items, because it focuses on methodological quality assessment, not the effectiveness of interventions. The JBI tool also enables item-level assessment, not only one single methodological quality judgement.

• Was the review question clearly stated?
• Were the inclusion criteria sufficiently specified for all elements of the research question acronym?
• Were the search strategy and terms used to search for studies appropriate? Are language/publication period restrictions appropriate?
• Which information sources were included? Were they appropriate?
• Were the criteria for critical appraisal of the studies appropriate?
• Was the critical appraisal conducted by two or more independent reviewers?
• Was data extraction performed by two or more independent reviewers? What methods were applied to minimize errors in data extraction?
• Were the methods used to combine included studies appropriate?
• Was the likelihood of bias assessed?
• Were practice recommendations supported by the reported data?
• Were there any recommendations for future research appropriate?

At first, each reviewer will assess a sample of four SRs as a calibration exercise. If the inter-rater agreement does not reach a pre-set threshold of Cohen's kappa of 75%, the calibration exercise will be repeated with other four studies. Further, any disagreement between the two reviewers during the quality assessment will be initially resolved by discussion (the reviewers will provide their reasons for rating a particular item as fulfilled or not). If consensus cannot be reached, a third reviewer will make the final decision. In the final manuscript, we will report the final numbers of Yes/No responses for each item, alongside narrative descriptions of the items rated as No or Unclear. We do not plan to present the total cumulative scores or overall quality classification. For each checklist item, the number and proportion of included systematic reviews with each response category will be reported. If different results occur in different types of included systematic reviews, we will present appropriate subgroup analyses. These will be undertaken only where each subgroup contains at least five reviews to permit meaningful descriptive comparison. The characteristics that may be considered for subgroup comparisons include, with regards to a methodological review, review design/type (e.g., quantitative versus qualitative/narrative systematic reviews) and population (e.g., different levels of education). The results of the quality assessment will be presented in tabular and graphical formats, supplemented by a narrative summary in line with the objectives of this methodological review.

The results of this methodological review will be discussed from two primary perspectives. First, recommendations will be provided regarding the interpretation and use of systematic reviews in this field. We anticipate that these conclusions could be useful to the researchers and practitioners in various educational and national contexts. Second, this review will offer recommendations for better design and development of future SR studies. The methodological quality of these studies is crucial for fulfilling the purpose of SRs as primary instruments for informing professionals in pedagogical practice and educational policies.

3 Discussion

We believe the planned methodological SR will provide some meaningful and useful information for teachers, school principals, researchers, policymakers, parents, and other pedagogical staff regarding methodological quality in evidence in teaching English as a foreign language. Incorporating the expected results into practitioners’ decision-making may lead to changes strengthening the awareness of methodologically sound evidence and improving the quality of future evidence in the field. This corresponds to the core principles of evidence-based education: the integration of the best available evidence, the perspectives and experience of experts, and the values of various groups of learners (American Physical Therapy Association, 2020; American Speech-Language-Hearing Association, n.d.; European University Association, 2020; U.S. Department of Education, 2016).

Possible limitations that may occur in our planned review: While our inclusion criteria place no restrictions on publication language, however, during the search phase, there remains a risk that SRs published in non-English languages may go undetected if they do not have an English translation for abstract, at least. This potential omission could consequently constrain the international reach of our findings. Further, the use of a critical appraisal instrument developed primarily for healthcare might be considered a limitation of the study. However, the items are general, without any specification regarding the research area. Rather, the issue could lie in the quantity and quality of primary studies available within the field of education compared to the healthcare field. Subsequently, the possible low quality of available SRs would lead to a lower certainty of conclusions and recommendations developed by our research team. Also, the included SRs may be of such great variety that any generalization of the results might not be supported enough by the evidence. However, even when quality of the included studies is limited, the review may provide useful information and recommendations for future authors of SRs, which could improve the quality of future research in education.

4 Ethics and dissemination

Since all analyses in the project will be performed using existing published data, there are no known ethical or legal obstacles that could impact their use and sharing. We will extract only relevant information and accompanying citations from the original studies based on data already published. No personal or sensitive data will be collected or processed. The planned systematic review is a secondary research study and therefore neither participants’ informed consent nor approval of an ethics committee are applicable. The full text of this methodological review will be submitted for publication in a peer-reviewed educational journal. For unrestricted use, the publication will be published under the terms of the latest available version of the Creative Commons Attribution International public license (CC BY 4.0 at the time of the project preparation). In addition to publication on the publisher's website, all metadata and appendices (such as data extraction tables) will be submitted in a public repository (e.g., Zenodo or OSF).

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2026.1801057/full#supplementary-material

Supplementary material Supplementary material at the source

Footnotes

Edited by: Raúl Ruiz Cecilia, University of Granada, Spain

Reviewed by: Yayu Heryatun, Sultan Maulana Hasanuddin Banten State Islamic University, Indonesia

M. Galuh Elga Romadhon, Universitas PGRI Wiranegara, Indonesia