Enhancing PISA-aligned argumentation skills of prospective biology teachers through generative artificial intelligence

1 Introduction

In the third decade of the 21st century, during an era of notable within this sample advancement in science and technology, the concept of biological literacy is undergoing fundamental transformation. The PISA 2025 conceptual framework established by the Organization for The Organization for Economic Co-operation and Development (OECD) requires modern education systems to move beyond rote memorization of fixed scientific facts and instead focus on helping students develop the ability to make evidence-based decisions on complex socio-scientific issues (SSI) such as global warming, genetic modifications, and pandemics (OECD, 2023). In today's world, where truth often feels elusive and information can be uncertain, the way we argue scientifically has grown beyond just a mental skill. As Höttecke and Allchin (2020) highlight, it now lies at the heart of what we call “scientific media literacy,” a vital part of how a healthy civil society function (Chinn et al., 2020).

However, there's a stubborn obstacle standing in the way: many future biology teachers themselves aren't fully comfortable with argumentation. They find it challenging to analyze opposing views or craft strong rebuttals. Research in biology education shows that teachers don't usually struggle with collecting scientific evidence; instead, they often find it hard to organize that evidence into clear, logical arguments or to weigh different hypotheses against each other (Sampson and Blanchard, 2012; Zeidler, 2014). In countries like Uzbekistan, where education still leans heavily on memorizing facts, these difficulties in developing discussion skills become even more pronounced among future teachers. This creates a ripple effect—if teachers can't build solid arguments, they'll find it tough to prepare students for international exams and standards.

In this regard, the Technological Pedagogical Content Knowledge (TPACK) framework, which has become relevant in contemporary pedagogy, requires teachers not only to know their subject matter but also to solve pedagogical problems through technology (Mishra and Koehler, 2006). Teachers' digital competency now extends beyond mere computer usage to encompass the management of complex cognitive processes (Siddiq et al., 2016). Approaches to teaching that use dialogue, rooted in social constructivist principles and the Socratic method, are widely recognized for fostering argumentation skills. Nevertheless, the comprehensive meta-analyses conducted by Hattie and Timperley (2007) highlight that the single most impactful factor in improving educational efficacy is prompt, personalized feedback. Unfortunately, in traditional higher education settings with hundreds of students in lecture halls, it is physically impossible for instructors to work individually with each participant and provide what Hattie termed the “feedback level.” The result of this situation is that most students tend to remain in a passive position, and argument development does not progress beyond the primary stage.

Future Generation Computer Systems is interested in contributing to the hype and the research concerning the applications of Emerging capabilities in Generative Artificial Intelligence (GenAI). The recent surveys on how AI is being used in higher education (Crompton and Burke, 2023) show that GenAI is actually far more adaptable than previous technologies and has learned how to understand the context of what is required and can tailor its communication to the pedagogical approach specified for the educational task. This study rests on the premise that GenAI can bridge the deficit in personalized feedback characteristic of conventional instruction. We advocate for employing this technology not simply as an access point for data, but fundamentally as a “Socratic Tutor” (or an Epistemic Agent). Following this model, the tool refrains from providing direct answers; instead, it engages students' assertions with critical inquiry and counterevidence, thereby stimulating deeper intellectual processing.

The central aim of this investigation is to conduct an empirical assessment of the efficacy of Generative Artificial Intelligence when deployed as a teaching facilitator to cultivate skills in scientific argumentation among individuals training to become biology educators. The study seeks to answer the following two primary research questions:

RQ1: How does intervention with artificial intelligence configured as a Socratic tutor affect the overall argumentation quality (based on Toulmin's model) of prospective biologists compared to traditional group discussions?

RQ2: Does the AI-mediated instructional package—comprising individualized Socratic questioning, systematic counter-argumentation, and immediate feedback—produce statistically significant differences in students' ability to evaluate counterevidence and provide rebuttals, which are considered the most complex competencies in PISA standards, compared to an unstructured peer-discussion protocol?

2 Literature review

2.1 Scientific argumentation: epistemic core and contemporary crisis

The common image of science education has undergone a drastic change in recent years from the model of “learning as the acquisition of knowledge” to “learning as participation in knowledge construction processes”. Against the backdrop of these changes, scientific argumentation is recognized not merely as a rhetorical skill but as a fundamental tool for understanding science's epistemic core—how scientific knowledge is formed and validated through social consensus (Osborne, 2010). Research indicates increasing trends in integrating argumentation into science education (Erduran et al., 2015).

In argumentation structures based on Toulmin's (1958) model, students often succeed in making claims and providing evidence. However, as Deanna Kuhn (2019) emphasizes, a serious systemic crisis is evident in the “rebuttal” component, considered the highest cognitive level of argumentation, and in the critical analysis of others' opinions. As Chinn et al. (2020) noted in their “AIR” (Aims, Ideals, and Reliable processes) model, this problem has intensified in today's “post-truth” world; students face difficulties not only in providing evidence but also in evaluating information reliability and matters of “epistemic trust.” Rebuttals require not only defending one's opinion but also “thinking differently” and cognitive flexibility.

2.2 Pedagogical approaches and limitations of traditional methods

Cognitive conflict theory is taken as an instructional strategy to promote argumentative competence. Limón (2001) pointed out that there will be no real conceptual change in students' minds if they do not experience the conflict caused by their current mental frameworks and the learning materials. In order to assist the process of cognitive conflict and lead students to realise their own mistakes via in-depth and strategic questioning, as suggested by Paul and Elder (2007), Socratic questioning should be adopted.

In the context of Vygotsky's ‘Zone of Proximal Development’ (ZPD) it is necessary that the approach to the development process is individualized. It is not possible for teachers in conventional classrooms to create individual cognitive conflicts for each student. Time does not allow for this, and the argumentation in the classroom often remains on the surface of ideas rather than going to the depth of the subject (Henderson et al., 2022). claim that in conventional group discussions students tend to hold confirmation bias. They present only those arguments for their point of view that are in line with their beliefs and leave out the ones that are contradictory. Kreijns et al. (2013) observe that the social loafing in group work negatively affects the argumentation process.

2.3 AI evolution in education: from rules to generative models

Over the past decade, artificial intelligence (AI) in education has evolved into a diverse range of different forms. Taking the years 2010–2020 as an example, the research conducted by Zhai et al. (2021) revealed that in the early days of education AI, the relevant education systems would mainly apply a rule-based approach. Although Chen and Wu (2020), Noroozi et al. (2020) affirm to some extent the role of Computer-Supported Collaborative Argumentation (CSCA) systems and gaming technologies that can support students, these systems are not really able to allow students to engage in uninhibited free dialogue. Wecker and Fischer argued in a meta-analysis that the script of the strictly automatic and static behavior of an AI is not adaptable to the various learning styles of students. So, this tool is not really suitable for handling the types of complex and argumentative discussions which frequently occur at higher levels of discussion with students.

The advent of Large Language Models (LLMs) such as ChatGPT in 2022 changed the game once again. Recent research on Generative AI in education (Chiu, 2024; Hwang and Chang, 2023; Kasneci et al., 2023; Tlili et al., 2023) concludes that current applications fall into three main categories: (1) content creation for students, (2) intelligent assessment systems, and (3) intelligent learning assistants. Additionally, enhancing teachers' AI competencies remains relevant (Kim and Kwon, 2023).

2.4 GenAI: “cognitive offloading” or “hybrid regulation”?

There has been a flurry of recent papers in academic literature critiquing GenAI. Most revolve around the search-answer paradigm and the perceived impacts on student cognition, academic integrity, and learning outcomes (Dwivedi et al., 2023; Sullivan et al., 2023). According to Lodge et al. (2023) in this latter case we are in the presence of a phenomenon known as “cognitive offloading”. The expression used in this context highlights how students delegate to the GenAI the cognitive activities necessary for the act of learning. In this way, students are less engaged in the learning process and are relegated to being simply consumers of the output produced by the system. As we have already seen, Rouet and Britt (2018) affirm that in the context of source evaluation, students need to develop particular sets of skills and strategies.

Recently, Zhai et al. (2020) and Cooper (2023) argued that rather than just being an evaluation tool, AI should be used as a thinking skills aid in the learning process of students. Molenaar (2022) in his “Hybrid Human-AI Regulation” theory, suggests that the use of AI can improve the self-regulation skills of students. In their study, Using AI in Surgery, Bearman et al. (2024) wrote that “using AI as a simulated opponent in a training environment opens up a wealth of opportunities in surgical education particularly for teaching evaluative judgements.”

2.5 Research gap

Presently, the majority of empirical studies dealing with the use of AI in learning environments have focused upon the role of the AI system as an information provider. There is a lack of research on AI's function as a “cognitive dissonance inducer”—a role grounded in Festinger's (1957) cognitive dissonance theory, whereby exposure to contradictory input compels conceptual reorganization—or as a “Socratic opponent,” drawing on the Socratic questioning tradition (Paul and Elder, 2007). In particular, rebuttal skill development mechanisms—which are thought to be the most intricate aspect of argumentative competency among aspiring biology teachers—remain on the periphery of scientific discussions. Furthermore, no empirical research integrates GenAI prompts tailored to PISA rubrics for training future biology teachers — a gap confirmed by recent systematic reviews of GenAI applications in education, which document content generation and general tutoring as dominant use cases without addressing rubric-aligned cognitive-conflict induction — overlooking how AI can scaffold complex argumentation in discipline-specific scenarios amid rising PISA AI literacy emphases. In this research, the lack of knowledge for the effectiveness evaluation of the debating artificial intelligence is complemented by means of prompt engineering with the goal that the AI that is used is treated as a ‘debating agent’ and its effectiveness evaluated according to Toulmin's Argument Pattern (TAP). The innovative character of this research lies in that it is conceived under a new instructional model of “discursive collaboration with the AI”, which at the pedagogical level transcends the traditional “answering” AI model that nowadays is taught and that reduces the role of the learning technology to the role of an educational tool. Hence, at this level, a technological innovation is used to obtain that AI assumes a pedagogical role which affects the students' epistemic beliefs.

2.6 Research novelty and theoretical contributions

This study introduces pedagogical, theoretical and methodological innovations that distinguish it from previous research: Unlike existing studies (Baidoo-Anu and Ansah, 2023; Zawacki-Richter et al., 2019) that employ AI as an information provider or assessment tool, this research operationalizes AI as a “Pedagogical Antagonist” using systematic prompt engineering. This represents a paradigm shift from “AI as knowledge source” to “AI as a mediator of a structured cognitive-conflict-inducing instructional package,” fundamentally altering the human-AI interaction dynamic in educational contexts. Previous argumentation studies in biology education (Erduran et al., 2015; Sampson and Blanchard, 2012) have primarily focused on peer-to-peer collaboration or teacher-mediated discussions. This study is the first to systematically integrate GenAI with Toulmin's Argument Pattern (TAP) framework using PISA-aligned assessment rubrics, providing more fine-grained precision in measuring complex argumentation competencies than prior peer-discussion-only studies, within the scope of this single-institution sample. The research advances Vygotsky's Zone of Proximal Development theory by demonstrating how AI can provide individualized cognitive scaffolding on a scale. While traditional applications of ZPD require intensive human mentorship, this study suggests that appropriately configured AI can replicate the “more knowledgeable other” function, enabling personalized epistemic development in large educational settings.

• Pedagogical Innovation

• Methodological Innovation

• Theoretical Innovation

Taken together, these innovations respond to the gap between theoretical frameworks for argumentation development and the practical realities faced by higher education institutions.

3 Research methodology

3.1 Research design and approach

The main purpose of this study is to verify the teaching effect of GenAI on the construction of counterarguments, namely the rebutter's skill, for biology teacher training students. Based on the theory of hybrid intelligence, this study has adopted the cognitive augmentation theory, which regarded artificial intelligence as a cognitive augmenter instead of a cognitive replacement. As pointed out by Zawacki-Richter et al. in the two systematic reviews (2019), the application of AI in education can enhance the pedagogical process, and AI becomes an auxiliary tool for teaching, which indicates that it is more of a tool in the teaching process rather than a teaching method itself.

This study employed a quasi-experimental pretest–posttest control group design to investigate the impact of AI-mediated Socratic tutoring on students' scientific argumentation skills. The study employed pre-test and post-test experimental design with control groups to measure the performance of the participants before and after implementing the proposed intervention as highlighted by Cohen et al. (2018). This design was employed to obviate the influence of extraneous variables in the educational settings and at the same time maintain ecological validity for the research setting since it is not practical to carry out full randomization in the classroom or school setting due to the structured nature of the class groups. Within the experimental parameters, the differentiating teaching methodology, either AI-facilitated Socratic cross-examination or standard peer-to-peer group deliberation—was established as the independent variable under scrutiny. The measured dependent variable was the caliber of the students' scientific reasoning, evaluated systematically based on the criteria set forth in Toulmin's argumentation model.

This study followed a six-phase methodological framework that facilitated a coherent integration of intervention development, implementation, measurement, and data analysis (Figure 1). Phase 1 involved pre-assessment of 124 participants using TAP, critical thinking, and biology knowledge instruments. To understand baseline cognitive abilities of the participants, critical thinking was assessed exclusively during the pre-test phase, as its purpose was to control pre-existing individual differences in higher-order thinking rather than to measure intervention-induced change; it was subsequently used as a covariate in ANCOVA analyses. Phase 2 established baseline equivalence through homogeneity testing. Phase 3 implemented the intervention by assigning participants to control and experimental groups: the control group received traditional collaborative learning, weekly seminars, and offline strategies, whereas the experimental group engaged in AI-mediated Socratic dialogue and cognitive conflict activities. Phase 4 monitored the intervention using TAP scores, cognitive load, discourse quality, and fidelity measures. Phase 5 involved post-assessment, with analysis based on TAP outcomes, a PISA-aligned argumentation evaluation, ANCOVA, and effect size estimation. The PISA-aligned evaluation refers to the TAP 5-Level Rubric instrument, which was developed based on OECD PISA scientific literacy descriptors and Toulmin's Argument Pattern (TAP) framework (described in detail in Section 3.4.2). This instrument was administered exclusively as a post-test measure because its purpose was to assess the outcomes of argumentation skill development following the intervention, rather than to establish baseline equivalence. Content validity was confirmed through expert review (CVI = 0.92), and inter-rater reliability was high (Cohen's Kappa κ = 0.87). The phase 6 integrated quantitative and qualitative analyses to compare pre- and post-intervention outcomes.

3.2 Participants and sample

This section describes the sample and procedures used to investigate the impact of AI-mediated Socratic tutoring on prospective biology teachers' scientific argumentation skills. The sample of the study consisted of 124 (N = 124) third-year students (N = 62 students for experimental group, N = 62 for control group) from the Biology Education program at the leading pedagogical University of Uzbekistan. To check the adequacy of the sample size, the statistical power analysis was done using the GPower 3.1 program with medium effect size (f = 0.25), and α = 0.05 error level and 1-β = 0.80 statistical power (Faul et al., 2007). The characteristics of the participants were as follows: average age - 21.4 years (SD = 1.24); sex distribution (Female: 68%; Male: 32%). Initially, achievement of internal validity of the experiment was investigated by determining the average values of the students' academic achievement criteria (GPA) and their critical thinking levels, and they were found statistically sufficient (p > .05). The study employed a quasi-experimental nonequivalent-groups design. Two intact class sections were assigned to the experimental and control conditions through administrative allocation rather than individual random assignment. Accordingly, participants were not individually randomized. Randomization was applied at the level of intact class sections rather than individual students, since full individual-level randomization was not feasible within the institution's existing class structure (see Section 3.1). All 124 enrolled participants completed the eight-week intervention and post-test assessment. Attrition was therefore 0%, no post-test outcome data was missing, and all 124 participants were included in the analyses.

3.3 Experimental process and conditions

The experimental process lasted 8 weeks and included intensive 90-minute weekly seminars. Both groups analyzed socio-scientific issues (SSI) rich in cognitive conflict, including “Global warming and anthropogenic factors,” “Genetically modified organism (GMO) safety,” “Vaccines and community immunity,” and “Gene editing (CRISPR) ethics,” based on a unified curriculum (Sadler et al., 2016). Sessions adhered to a conventional collaborative learning structure rooted in social constructivist principles. Participants were segmented into compact teams of four to five members, initiating reciprocal discussions concerning provided scenarios. The educator's function was strictly supervisory, offering direction without direct involvement in the substance of the exchanges. In order to mitigate the risk of “group contamination” affecting the internal validity of the study, these control sessions occurred entirely “offline,” thereby precluding internet connectivity. In addition, all students signed an “Academic Integrity Contracts” before the trial, pledging not to use any external generative AI tools for the tasks. This cohort used a novel methodology in which every student participated in real-time dialogue with dedicated artificial intelligence agents powered by the GPT-4 framework. The AI's programming deviated from that of a simple “intelligent search repository”; instead, it assumed the function of a “Pedagogical Antagonist” (as described by Hwang and Chang, 2023). The goal was to remove students from their “comfort zone” and compel them to reconsider their arguments. To ensure fidelity of the intervention process, all dialogues between experimental group students and the AI were anonymously recorded. Researchers analyzed 15% of randomly selected dialogues and confirmed that 94% of students were working according to the designated argumentation protocol.

• Control Group

• Experimental Group

Control-group participants signed an integrity agreement stating that they would not use generative-AI tools for the assigned activities. No objective device, browser, or account-level monitoring was available. Consequently, unreported AI use in the control condition cannot be completely excluded, and the integrity agreement should be interpreted as a self-reported rather than independently verified fidelity measure.

3.4 Research instruments and prompt engineering

The study employed a range of pre-test and post-test instruments to evaluate participants' baseline characteristics and learning outcomes. The pre-test measures included the Pre-Test TAP Assessment, Critical Thinking Level Test, and Prior Biology Knowledge Test, whereas the post-test measures comprised the PISA-Aligned TAP Post-Test, TAP Sub-Rubrics for Component Analysis, Cognitive Load Scale, and Discourse Quality Framework (Table 1).

Table 1. Instrument classification by development origin, administration phase, and psychometric properties.

3.4.1 Pre-test instruments: baseline measurement and covariate control

The primary analysis compared post-test TAP scores between the experimental and control conditions while controlling only for baseline TAP scores. Treatment condition was entered as the fixed factor, post-test TAP score as the dependent variable, and pre-test TAP score as the sole covariate. Critical-thinking and prior-biology-knowledge scores were not included as covariates in this primary ANCOVA; these variables were examined separately in supplementary predictor and moderation analyses. With 124 participants, two treatment conditions, and one covariate, the primary ANCOVA had 121 error degrees of freedom, consistent with the model reported in Table 6.

• Pre-Test TAP Assessment
• Critical Thinking Level
• Prior Biology Knowledge
• Assessment of Assumptions

Before conducting the primary ANCOVA, the linear relationship between baseline and post-test TAP scores was evaluated using scatterplots and partial-residual plots. Homogeneity of regression slopes was assessed by testing the Baseline TAP × Treatment interaction. Homogeneity of error variance was evaluated using Levene's test. Residual normality and influential observations were examined using Q–Q plots, standardized residuals, and Cook's distance. The Critical Thinking × Treatment interaction reported in Table 17 was part of a separate moderation analysis and was not an assumption test for the primary ANCOVA because critical thinking was not included as a covariate in that model.

Before conducting the primary ANCOVA, linearity between baseline and post-test TAP scores was examined using scatterplots and partial-residual plots. Homogeneity of regression slopes was evaluated by testing the Baseline TAP × Treatment interaction. Homogeneity of error variance was assessed using Levene's test. Residual normality and influential observations were examined using Q–Q plots, standardized residuals, and Cook's distance.

3.4.2 Post-test and process instruments: outcome measurement

Four instruments were used to measure intervention outcomes and process indicators. These instruments were administered either at post-test (Week 8) or at multiple points during the intervention (Weeks 4 and 8). Each instrument is described below with its development basic, structure, and role in addressing research questions.

• PISA-Aligned TAP Post-Test (Primary Outcome Measure — RQ1)
• TAP Sub-Rubrics — Component Analysis (RQ1 & RQ2)
• Cognitive Load Scale (Supporting Analysis)
• Discourse Quality Framework (Qualitative Validation — RQ2)

Each instrument component directly corresponds to PISA 2025 scientific literacy competencies1, ensuring international assessment alignment. The TAP rubric operationalizes PISA Levels 1–5, with particular emphasis on Levels 4–5 representing advanced counter-argumentation skills essential for scientific citizenship in the 21st century. The rubric was an adapted TAP–CER framework rather than a complete operationalization of Toulmin's original six-element model. It assessed claim, evidence, warrant/reasoning, rebuttal, and counter-evidence, but did not score backing and qualifier as separate dimensions. Accordingly, references to TAP scores in this study denote scores obtained using this adapted TAP–CER rubric.

3.4.3 Prompt engineering

To ensure research replication possibilities, specialized systematic instructions based on “Prompt Engineering” methodology were developed as follows: Prior to the main experiment, a one-week pilot study involving 12 students was conducted to test the effectiveness of the proposed AI system instructions. The initial version revealed that AI provided overly aggressive rebuttals to students, and the prompt tone was softened from “extremely critical” to “Socratic doubt” level.

• Your role: A strict, skeptical, and critically thinking biology professor. Your goal is not to provide direct answers to students but to expose logical errors in their arguments.
• Interaction algorithm: 1) Deconstruct the student's claim according to Toulmin's model (claim, evidence, warrant); 2) If the student speaks without evidence, strictly demand scientific sources; 3) Most importantly, present strong scientific rebuttals against the student's position (For example: ‘If antibiotics kill all bacteria as you claim, then why are resistant strains multiplying?’); 4) Always conclude conversations with open questions that place students in cognitive dissonance.
• Constraints: Rely only on facts with scientific consensus, avoid discussing personal beliefs, and refrain from providing incorrect information.

To manage primary risks associated with generative AI integration in education—false facts and algorithmic dishonesty—the “Human-Controlled-AI” principle was implemented (UNESCO, 2023). First, the system instruction imposed an imperative on AI to rely only on information from indexed databases such as PubMed and ScienceDirect. Second, students were assigned “Fact Verification” tasks as a metacognitive strategy; they were required to compare each AI rebuttal with textbooks or reliable external sources.

The intervention used the consumer ChatGPT interface identified at the time as GPT-4. The researchers did not access the model through an API; therefore, API-level parameters such as temperature, top_p (text-generation sampling parameter), token limit, and frequency or presence penalties were not available for researcher control or reporting. The exact system-level model snapshot could also change during platform updates.

Illustrative anonymized pre- and post-intervention response excerpts are presented in Table 2.

Table 2. Anonymized pre- and post-intervention response excerpts (illustrative example, GMO crop safety scenario).

3.5 Data analysis

3.5.1 Argumentation scoring procedure

Students' written responses were coded using a 5-level analytical rubric, based on the Toulmin's Argument Pattern (TAP) model (McNeill and Krajcik, 2011). The levels were: Level 1 – simple claim; Level 2 – claim with basic evidence (evidence or warrant present, but no rebuttal); Level 3 – evidence and warrant present but no rebuttal; Level 4 – includes basic rebuttal; Level 5 – strong claim incorporating empirical evidence and deep scientific rebuttal against alternative views. Higher levels represent increasingly sophisticated argument structures, particularly through: Integration of multiple evidence sources, Use of counter-arguments, and Inclusion of rebuttals and limitations. To ensure assessment reliability, 30% of submissions were examined “blind” by two independent experts, confirming high inter-rater agreement (Cohen's Kappa, κ = 0.87). Each student completed five open-ended biological argumentation scenarios. Every individual scenario response was assigned a single TAP level (1–5) representing the highest argumentation quality demonstrated in that response. For Table 9, level-specific mean scores were computed by counting, for each TAP level k, the total number of responses across all participants that were scored at level k, then divided by the number of participants (n = 62 per group). This produces a mean frequency score indicating how often, on average, a student demonstrated a given TAP level across the five scenarios. This item-frequency metric is distinct from the holistic proficiency classification used in Table 9, where each student is assigned a single overall PISA proficiency level based on aggregated post-test performance.

3.5.2 TAP component assessment framework

To provide comprehensive analysis of students' argumentation development, the TAP assessment employed both holistic scoring (Levels 1–5) and component-based evaluation. The five TAP components were assessed individually using specific sub-rubrics derived from the main 5-level framework: Claim formulation assessed the clarity and specificity of students' scientific claims on a 5-point scale (1 = vague/absent claim, 5 = precise, testable scientific claim with clear scope and limitations). Evaluated the appropriateness and strength of scientific evidence provided to support claims on a 5-point scale (1 = no evidence, 5 = multiple, high-quality empirical sources with proper citation and contextual relevance). Warrant construction measured students’ ability to connect evidence to claims through logical reasoning on a 5-point scale (1 = no logical connection, 5 = explicit, scientifically sound reasoning that clearly links evidence to claim). Rebuttal skills assessed students' capacity to address alternative viewpoints and construct counterarguments on a 5-point scale (1 = no consideration of alternatives, 5 = sophisticated refutation of opposing views with scientific justification). This analysis evaluated students' ability to critically examine evidence that contradicts their position and integrate this analysis into their argument on a 5-point scale (1 = ignores contradictory evidence, 5 = systematically addresses conflicting evidence and explains its limitations or contextual boundaries).

• Claim Formation

• Evidence quality:

• Warrant Construction

• Rebuttal Skills

• Counter-Evidence Analysis

Each component was scored independently, with values representing mean performance levels rather than frequency counts. For example, a score of 3.12 indicates that students averaged Level 3.12 performance on that particular component. The component-based assessment complemented the holistic 5-level evaluation to provide detailed insight into specific areas of argumentation strength and weakness, enabling targeted pedagogical interventions.

Furthermore, Inter-rater reliability was maintained across all components, with the same two independent experts achieving Cohen's Kappa κ=0.87 for component-based scoring. This dual assessment approach (holistic + component-based) provided both overall argumentation quality measures and diagnostic information about specific skill development patterns.

3.5.3 Overall argument quality calculation

Argument quality was evaluated using a hybrid scoring approach that integrated component-based analysis with holistic level classification based on Toulmin's Argument Pattern (TAP). Each student response was first examined for the presence and quality of key argumentation components, including claim, evidence, warrant (reasoning), rebuttal, and counter-evidence analysis. Subsequently, an overall argumentation level (Levels 1–5) was assigned to reflect the structural completeness, coherence, and sophistication of the argument.

The assessment followed Toulmin's progressive framework, whereby advancement to higher levels required mastery of foundational argumentation elements. Level 1 represented a simple claim; Level 2 required the inclusion of supporting evidence; Level 3 required both evidence and a warrant connecting the evidence to the claim; Level 4 required the incorporation of a rebuttal addressing alternative viewpoints; and Level 5 required a well-supported claim strengthened by empirical evidence and a sophisticated scientific rebuttal that critically evaluated competing explanations.

The overall TAP score was determined holistically and reflected the highest argumentation level demonstrated by the student according to the rubric criteria. Rather than averaging performance across individual components, the scoring emphasized the extent to which the response satisfied the requirements of each progressive level. Thus, students were required to demonstrate all prerequisite components before being assigned a higher-level score.

This integrated scoring procedure preserved the theoretical integrity of Toulmin's framework while providing a comprehensive measure of argument quality. By capturing both the structural components and the overall sophistication of students’ arguments, the approach enabled reliable pre–post comparisons and between-group analyses using ANCOVA.

3.5.4 Statistical analysis

The data were analyzed using IBM SPSS Statistics 28.0 (Field, 2018). The primary hypothesis was tested using ANCOVA, with post-test TAP score as the dependent variable, treatment condition as the fixed factor, and pre-test TAP score as the sole covariate. Statistical significance was set at p < .05, and effect size was reported using partial eta squared (ηp²). Before the primary analysis, linearity, homogeneity of regression slopes, homogeneity of error variance, residual normality, and influential observations were assessed. Critical-thinking and prior-biology-knowledge scores were analyzed separately as individual-difference predictors and moderators.

3.6 Data validity and reliability

Students' argumentative competency was measured using open-ended biological cases adapted from the PISA framework's scientific literacy assessment system (OECD, 2023). Initially, three independent experts evaluated task adaptation to local context, achieving a Content Validity Index (CVI) of 0.92. In addition, the pilot test designed for verifying the psychometric characteristics of the instrument gave Cronbach's alpha coefficient (α=.84) a very reliable metric which showed high internal consistency. In addition, the pre-test and post-test items of SCME designed in an isomorphic fashion required the students not only to justify their choices but also in scientific manner, reject opposite arguments and proofs.

3.7 Research hypotheses

Based on cognitive conflict theory and Vygotsky's Zone of Proximal Development framework, the following directional hypotheses were formulated:

Primary hypothesis (H₁): Students receiving AI-mediated Socratic tutoring will demonstrate significantly higher overall argumentation quality scores (measured by TAP rubric) compared to students in traditional collaborative learning conditions, after controlling for baseline differences.

Secondary hypothesis (H₂): The AI intervention will produce significantly larger effect sizes in complex argumentation competencies (rebuttal skills and counter-evidence evaluation) compared to foundational components (claim formation and basic evidence provision), reflecting AI's particular effectiveness in generating cognitive conflicts that enhance higher-order thinking skills.

Tertiary hypothesis (H₃): Students in the experimental group will show significantly greater advancement to PISA Levels 4–5 (advanced counter-argumentation) compared to control group students, demonstrating AI's effectiveness in developing internationally recognized scientific literacy competencies.

Null hypothesis: No significant differences exist between groups across all measured variables (H₀₁, H₀₂, H₀₃).

These hypotheses directly address the research questions while providing testable predictions grounded in established cognitive and pedagogical theory. The relationship between the research questions and the corresponding hypotheses is presented in Table 3, demonstrating the logical alignment of the study's research design.

Table 3. Mapping between research questions and hypothesis.

3.8 Ethical considerations

All procedures performed in this study involving human participants were in accordance with the ethical standards set by the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study has been conducted in accordance with the ethical guidelines advised by the Institutional Review Board of our institutions and in line with institutional requirements. Informed consent was obtained from all individual participants involved in the study. All the relevant information and possible issues were disclosed to participants. The research was performed in accordance with relevant guidelines. General Data Protection Regulation (GDPR) was followed to protect the data.

4 Results

4.1 Descriptive statistics and pre-test analysis

124 third year biology education students took part in this research study (N = 124). Students were allocated to the two groups (n = 62 each) via clustered random sampling at the class-section level, consistent with the quasi-experimental design described in Section 3.1. The mean age of students in the study was 21.4 (±1.24) years old and 68% of the students were females and 32% males (Table 4).

Table 4. Descriptive statistics and group homogeneity testing.

Key terms: M, mean; SD, standard deviation; GPA, grade point average; TAP, Toulmin's argument pattern; CI, confidence interval. Pre-test TAP score was used as the sole covariate in the primary ANCOVA; critical-thinking and prior-biology-knowledge scores were examined separately in supplementary analyses.

A pre-test analysis showed that the homogeneity between the experimental groups was also statistically valid (p > .05). This internal validity was an important element for the constructionist approach taken in the study. The baseline argumentation skills of the students in both groups were roughly the same. The results were categorized at level 2 of the TAP, which pertains to basic claim with evidence (Table 5).

Table 5. Pre-test distribution across TAP components (mean component scores on 5-point scales).

4.2 Overall argumentation quality and TAP performance

The research question 1 focus is Intervention Research focused on investigating the effects of AI-based Socratic tutoring on prospective biologists' argumentation skills according to Toulmin model of scientific argumentation as compared to their argumentation skills in traditional group discussions.

This section presents more detailed and thorough evidence related to the impact of the AI intervention on the overall argumentation quality of students' written arguments across all TAP activities within the scope of the different learning paths identified in the study.

4.2.1 Primary hypothesis testing: overall argumentation quality

Analysis of Covariance (ANCOVA) was used to test the primary research hypothesis, with pre-test TAP scores as covariates and post-test overall argumentation quality as the dependent variable, calculated according to the composite scoring method described in Section 3.5.

ANCOVA revealed a significant condition effect, F (1,121) = 14.23, P < .001, with ηp² indicating that 10.5% of the variance in argumentation quality was explained by the condition variable (Table 6). This F means that participants in the AI condition outperformed participants in the control condition. An ηp² of this size is generally considered to denote a medium-to-large effect size, according to Cohen (Table 7).

Table 6. ANCOVA results for overall argumentation quality.

Key terms: SS, sum of squares; df, degrees of freedom; MS, mean square; F, F-statistic; η²p, partial eta squared (proportion of variance explained by treatment); Observed power = statistical sensitivity to detect the effect.

p < .001.

Table 7. Adjusted post-test means by group.

Key terms: Unadjusted mean = raw post-test group mean; Adjusted mean = post test mean after controlling for pre-test TAP score via ANCOVA; SE , standard error; CI, confidence interval. Mean difference = Experimental adjusted M−Control adjusted M.

Bold value indicates the statistically significant adjusted mean difference between the experimental and control groups.

*p < .001.

Given the number of significance tests conducted across TAP components, discourse indicators, and moderator analyses, findings should be interpreted with this multiple-comparisons risk in mind; a Bonferroni-adjusted threshold is reported alongside uncorrected p-values in the relevant tables.

4.2.2 Results substantiation and theoretical alignment

The significant condition effect (F (1,121) = 14.23, p < .001, η²_p = .105) provides robust empirical validation of cognitive conflict theory in AI-mediated learning environments. This finding aligns with Limón's (2001) theoretical predictions while extending beyond traditional face-to-face applications. The medium-to-large effect size (η²_p = .105) exceeds typical educational intervention outcomes reported in meta-analyses by Hattie and Timperley (2007), suggesting that the structured, AI-mediated Socratic protocol may be particularly effective for argumentation skill development, although this study's design cannot isolate the technology's contribution from that of the structured practice it embodies.

Unique data point analysis: the experimental group's adjusted mean (M = 3.91) represents advancement from baseline Level 2 (claim + basic evidence) to approaching Level 4 (includes rebuttal), indicating qualitative transformation in argumentation sophistication. This progression typically requires multiple semesters in traditional pedagogical approaches yet was achieved within 8 weeks through AI-mediated cognitive conflicts.

The 95% confidence interval [0.45, 0.95] for mean differences confirms practical educational significance, as this range represents approximately one full TAP level improvement—a threshold indicating meaningful pedagogical impact according to McNeill and Krajcik (2011) argumentation development standards.

4.2.3 Comprehensive TAP component analysis (All five components)

Each TAP component was examined individually to determine where the effect of AI was most evident.

Our study discovered that the deployment of an AI-based educational module on TAP led to a statistically significant increase in performance on all argumentation components assessed by the instrument (Table 8). Claim formation, evidence quality and warrant construction, while all building blocks of arguments and improved upon, showed the smallest gains relative to the rebuttal and counter-evidence assessment tasks, suggesting that the complex skills taught by the module were developed to a greater extent.

Table 8. ANCOVA results for individual TAP components.

Key terms: adj. M = ANCOVA-adjusted mean, controlling for pre-test TAP score, critical thinking level, and prior biology knowledge; SE, standard error; η²p, partial eta squared; Cohen's d = standardized mean difference effect size. Effect size benchmarks (Cohen, 1988): small = 0.20, medium = 0.50, large = 0.80. p-values are reported without correction for multiple comparisons. Applying a Bonferroni correction across the 13 tests conducted in this study (5 TAP components + 5 discourse indicators + 3 moderator interactions; adjusted α = .05/13 = .0038) would render the following results non-significant at the adjusted threshold: Evidence quality (p = .015), Warrant construction (p = .012), and Prior GPA × Treatment (p = .018). Rebuttal skills, Counter-evidence evaluation, and Critical thinking × Treatment remain significant under the corrected threshold.

p < .05,

p < .001.

4.2.4 TAP rubric level distribution (levels 1–5)

TAP levels represent the structural complexity of arguments based on Toulmin's framework. The evolution of students' argumentation skills over the course of a senior-level ethics class was investigated, with particular emphasis on their ability to meet the criteria set forth in all of the levels described by Toulmin (Table 9).

Table 9. Detailed scoring rubric results by TAP level.

Key terms: TAP Level = Toulmin's Argument Pattern structural complexity level (1–5); Max points = maximum possible mean value per level, equal to the total number of scenarios (5); M = mean frequency of level demonstrations per student across five scenarios; SD = Standard Deviation; t-test = independent samples t-test comparing group means at each TAP level.

p < .01,

p < .001.

The mean scores reported in Table 9 were calculated based on students' performance across five open-ended biological argumentation scenarios, each independently scored using the TAP 5-level rubric. Importantly, each scenario was scored for the highest TAP level demonstrated within that response — not as a cumulative sum across levels. The mean score for each TAP level therefore represents the average number of scenarios (out of five) in which a student's response met the criteria for that specific level.

The formula applied was:The “Max points” column indicates the maximum possible mean value for each level, corresponding to the total number of scenarios administered (i.e., if all 62 students in a group achieved a given level on all five scenarios, the maximum possible mean would equal 5). Accordingly, Max = 1 for Level 1 reflects that Level 1 is the baseline threshold achievable in every scenario, while Max = 5 for Level 5 reflects that the highest level could theoretically be achieved across all five scenarios.

MeanLevelkScore=(TotalnumberofresponsesscoredatLevelkacrossallparticipants)÷(Totalnumberofparticipants)

This item-level frequency approach is conceptually distinct from the holistic proficiency distribution presented in Table 10. Table 10 reports the number of students classified at each PISA proficiency level based on their overall post-test performance, whereas Table 9 reports the mean frequency of level demonstrations per student across individual scenarios. These two measures are complementary: Table 9 captures within-student variability across items, while Table 10 captures between-student proficiency distribution at the group level. Both are consistent with the scoring framework described in Section 3.5.

Table 10. Distribution of students across PISA argumentation levels (post-test).

Key terms: PISA Level = PISA 2025 scientific literacy proficiency level (1–5); n = number of students; % = percentage of total group (n = 62); Residual χ² contribution = each cell's contribution to total chi-square statistic (positive = more than expected, negative = less than expected); Cramér's V = effect size for chi-square test of independence = 0.36.

Bold values indicate the total number and percentage of participants in each group.

p < .001, df = 4.

The distribution demonstrates statistically significant improvements across the higher TAP levels (Levels 3–5), indicating that students in the experimental group achieved more sophisticated argumentation structures.

4.2.5 PISA-aligned scientific argumentation assessment

PISA levels reflect competency-based performance aligned with scientific literacy standards. Students' overall performance was evaluated using a 5-level analytical rubric based on PISA scientific literacy standards to contextualize improvements within international assessment frameworks (Table 10).

Experimental group students scored higher on tasks at the level of PISA proficiency scales in all achievement levels and showed a clear shift to the right.

4.2.6 Learning trajectory over time: weekly progress tracking

TAP scores provide a numerical representation of argument quality derived from TAP level classification. Students' argumentation quality was measured over the whole 8-week intervention period to look at learning trajectories and to see whether gains were sustained (Table 11).

Table 11. Weekly TAP score progression by group.

p < .05,

p < .01,

p < .001.

Compared with the control group, the experimental group learned with a 2.2-fold faster speed and showed accelerated improvement in argumentation quality throughout the intervention period. The mean differences between the two groups gradually expanded, reflecting the effect of AI-based Socratic dialogue on improving students' argumentation skills (Figure 2).

4.3 Complex competencies - rebuttal skills and counter-evidence evaluation

This section provides a deeper analytical interpretation of argumentation components, particularly those related to advanced competencies such as rebuttal and counter-evidence evaluation.

While Section 4.2 addressed overall argumentation quality improvement (RQ1), this section specifically examines the two most cognitively demanding TAP components—rebuttal skills and counter-evidence evaluation—to answer RQ2 regarding complex competencies. According to PISA frameworks, these represent the highest-order thinking skills in scientific argumentation. The following analysis focuses specifically on performance at TAP Levels 4–5, which requires integration of sophisticated counter-argumentation abilities that distinguish advanced scientific thinkers from novice learners. Although indicators such as the use of scientific terminology and multiple evidence sources are commonly associated with general argument quality, they were included in this analysis because they contribute to the epistemic sophistication required for effective rebuttal construction and counter-evidence evaluation.

This targeted analysis is necessary because RQ2 specifically investigates whether cognitive conflicts generated by AI produce statistically significant differences in the most complex competencies, rather than general argumentation improvement. The effect sizes and statistical outcomes for these advanced skills provide distinct evidence for AI's effectiveness in developing higher-order cognitive abilities.

4.3.1 Rebuttal skills and counter-evidence evaluation: TAP component focus

Detailed analysis of the two most cognitively demanding TAP components reveals the magnitude of improvement in complex argumentation competencies. The experimental group achieved significantly superior performance in rebuttal formation, F (1,121) = 33.89, p < .001***, η_(*p*2) = .219, d=0.881. Control group adjusted M = 1.83 (SE = 0.11), experimental group adjusted M = 2.74 (SE = 0.11). The large effect size (d = 0.88) indicates highly significant practical improvements. Counter-evidence evaluation demonstrated substantial improvement in the experimental group, F (1,121) = 30.47, p < .001**, η_(*p*2) = .201, d = 0.981. Control group adjusted M = 1.95 (SE = 0.12), Experimental group adjusted M = 2.89 (SE = 0.12). This very large effect size (d = 0.98) clearly points to real and highly significant educational gain.

• Rebuttal skills performance

• Counter-evidence evaluation performance

The effect sizes for the TAP components of reasoning through rebuttal and reviewing counterevidence are the highest of all the TAP components measured by PISA, which indicates that the potential of AI to boost students' advanced argumentation skills is particularly high.

4.3.2 Advanced TAP levels (levels 4–5) distribution

The performance on the most advanced TAP levels, which demand highly developed ability to construct counterarguments and to integrate refutations, is the indicator of the effect of AI on advanced skills.

Both advanced levels showed statistically significant differences favoring the experimental group, with Level 5 (the highest complexity tier requiring integration of rebuttal and counterevidence) demonstrating particularly large differences (Table 12).

Table 12. Performance analysis.

*p < .001.

4.3.3 PISA levels 4–5 achievement: advanced counter-argumentation skills

Based on the PISA framework, Levels 4–5 pertain to elementary and advanced forms of counter-argumentation competencies, thus directly addressing RQ2 for complex skills.

The fact that scores are clustered towards the higher end of the PISA scale in the experimental group shows that the AI made a meaningful impact on students' ability to reach the advanced level of counter-argumentation skills that PISA labels the most sophisticated (Table 13).

Table 13. Distribution analysis.

Chi-square: χ² = 16.34*, Cramer's V = 0.36 (large effect).

Discourse quality was assessed using five binary-scored indicators operationalized in Section 3.4.2. Results are presented in Table 14 and discussed below.

Table 14. Discourse quality indicators — percentage of students demonstrating each feature in post-test responses (n = 62 per group).

Key terms: Quality indicator=binary-scored discourse feature (1 = present/0 = absent); % = percentage of students in each group demonstrating the indicator (Number of students with indicator present ÷ 62 × 100); χ² = chi-square statistic for between-group comparison; All indicators scored from post-test written responses only (Week 8). P-values are reported without correction for multiple comparisons. Applying a Bonferroni correction across the 13 tests conducted in this study (5 TAP components + 5 discourse indicators + 3 moderator interactions; adjusted α = .05/13 = .0038) would render the following results non-significant at the adjusted threshold: Evidence quality (p=.015), Warrant construction (p=.012), and Prior GPA × Treatment (p=.018). Rebuttal skills, Counter-evidence evaluation, and Critical thinking × Treatment remain significant under the corrected threshold.

p < .01,

p < .001.

4.3.4 Discourse quality results: qualitative indicators of complex argumentation sophistication

The analysis was conducted using students’ post-test written responses as the primary data source. Only responses collected after the 8-week intervention were included. Each response was evaluated by two independent raters using binary scoring (present = 1/absent = 0) for each of the five discourse quality indicators described in Section 4.3.4.

The unit of analysis was the individual student response (n = 62 per group). The percentages reported in Table 14 represent the proportion of students within each group whose responses contained a given discourse feature, calculated as:For example, Counter-argument integration at 61.5% in the experimental group means that 38 out of 62 students produced responses that explicitly acknowledged and addressed opposing viewpoints. This student-level proportion is consistent with the binary scoring procedure described in Section 4.3.4 and directly comparable across groups. Inter-rater reliability was κ = 0.84, confirming consistent application of the coding framework. Raters were not formally blinded to group condition, since response content (e.g., references to AI dialogue vs. peer discussion) often revealed group membership implicitly; this is acknowledged as a limitation potentially affecting scoring objectivity.

Percentage=(Numberofstudentsdemonstratingtheindicator÷Totalnumberofstudentsingroup)×100

By focusing on coded discourse segments, the analysis provided a fine-grained assessment of argumentation quality and structure. This method allowed for the identification of subtle variations in students' use of claims, evidence, reasoning, rebuttals, and other argumentative elements, thereby offering a more comprehensive understanding of their argumentation performance at the discourse level.

We found that student arguments measured by our content analysis showed qualitative improvements corresponding to the increased presence of rebuttals and counterevidence in their feedback (Table 14).

These indicators demonstrate sophisticated rebuttal and counter-evidence construction: These qualitative indicators corroborate quantitative findings, demonstrating that cognitive conflicts generated by AI specifically enhanced complex argumentation competencies.

• Counter-argument integration (9.6% → 61.5%): Students in the experimental group demonstrated dramatically improved capacity to incorporate opposing viewpoints into their arguments.
• Acknowledgment of limitations (15.4% → 78.8%): The substantial increase indicates students’ enhanced ability to recognize constraints in their own position, a hallmark of sophisticated rebuttal.
• Epistemic hedging (31.2% → 72.1%): The dramatic improvement reveals students’ sophisticated understanding of knowledge uncertainty, enabling nuanced counterevidence evaluation.

4.3.5 Effect size analysis for complex competencies

The detailed effect size analysis for rebuttal and counter-evidence skills shows the practical effect of improvements in the most demanding argumentation skills. For rebuttal skills and counter-evidence evaluation the effect sizes were found to be d = 0.88 and d = 0.98, respectively, which were considered to represent large educationally significant effects as defined by Cohen (Table 15). Thus, the fact that such large effects were found confirms RQ2 according to the specifications made for RQ2 above. That is, the cognitive conflicts generated by technology artefacts like an AI that can make counterarguments are of such a nature that students benefit from being given such tasks statistically and educationally in a substantial way in the most advanced PISA competencies.

• PISA Level Achievement: refers to the average proficiency level attained by participants based on rubric scoring aligned with PISA scientific literacy descriptors. Scores were derived by mapping each response onto defined proficiency levels and calculating group-level averages to represent overall competency.
• PISA Level Achievement Operationalization: this measure represents the proportion of students achieving PISA Levels 4–5 (elementary and advanced counter-argumentation skills) as defined by international scientific literacy standards. Students were classified according to their highest demonstrated proficiency level based on post-test performance, with Level 4 requiring basic rebuttal skills and Level 5 requiring sophisticated counter-evidence analysis and integration. The effect size calculation compares the experimental group's advancement rate to higher PISA levels against the control group's achievement distribution.

Table 15. Effect size analysis for complex competencies (excerpt).

4.3.6 Illustrative qualitative examples

To substantiate the quantitative gains reported in Table 9 and Table 14 with concrete evidence of the qualitative shift in students' reasoning, Table 2 presents anonymized excerpts of student written responses to the same socio-scientific argumentation scenario, before and after the intervention, drawn from both groups.

Compared with the control group excerpt, which remains at a claim-plus-basic-evidence level even after the intervention, the experimental group excerpts show the qualitative markers identified in the discourse analysis in Table 14 — explicit counter-argument integration (“however, one could argue that..”), acknowledgment of limitations (“has not been fully evaluated”), and epistemic hedging (“appear to be,” “it is worth acknowledging”) — consistent with the TAP Level 4–5 gains reported quantitatively in Section 4.3.3.

4.4 Supporting analyses

This section contains supplementary analyses which shed further light upon the results for RQ1 and RQ2.

4.4.1 Cognitive load and engagement measures

To assess whether AI intervention created cognitive overload and to validate the learning effectiveness, students' perceived cognitive load and engagement were measured using established instruments. Cognitive load was assessed using Sweller's (1988) three-component model via 7-point Likert scales: Intrinsic cognitive load measured task-inherent difficulty, extraneous cognitive load assessed irrelevant processing demands, and germane cognitive load evaluated productive learning effort. Task engagement was measured using Fredricks et al.'s (2004) behavioral engagement scale, while perceived learning employed self-report measures of knowledge gain perception. All instruments demonstrated adequate reliability (α >.75) in our sample (Table 16).

Table 16. Cognitive load and engagement indicators.

p < .01,

p < .001.

Results indicate that AI intervention reduced extraneous cognitive load while significantly increasing germane (learning-related) cognitive load and student engagement.

4.4.2 Overall effect size summary

This comprehensive effect size analysis synthesizes results across both research questions to provide educators and policymakers with practical significance indicators for AI implementation decisions. While individual component analyses addressed specific research questions, this summary enables comparison of intervention effectiveness across different skill domains and informs resource allocation decisions for educational technology integration.

The overall effect size for the AI intervention was d = 0.67 (i.e., a medium to large sized effect) and was therefore considered to be practically significant, i.e., being greater than statistically significant (Table 17). In addition, the effect sizes for the individual intervention outcomes were deemed to be educationally significant.

Table 17. Comprehensive effect size analysis (Cohen's d computed from ANCOVA-adjusted post-test means; overall TAP score effect size corresponds to the group comparison reported in Table 7).

For descriptive interpretation, we used an adapted convention based on Cohen's (1988) approximate reference points: values near 0.20 were described as small, values near 0.50 as medium, and values near 0.80 as large. These labels are heuristic rather than rigid thresholds.

Cohen's d for the overall adjusted group comparison was calculated using the pooled within-group standard deviation rather than the pooled standard error. The adjusted means were 3.91 for the experimental group and 3.21 for the control group, with SE = 0.09 and n = 62 in each group. The corresponding within-group SD was estimated as SE×n=0.09×62=0.709. Thus, Cohen's d was 3.91−3.210.709=0.99. Standard errors were not used as the denominator.

4.4.3 Moderation analysis: individual differences

Further analyses were undertaken to investigate whether the intervention had a differential effect on different student characteristics (Table 18).

Table 18. Moderation analysis results.

p < .05,

p < .001.

The significant Critical Thinking × Treatment interaction (β = 0.67, SE = 0.19, t = 3.53, p < .001, ΔR² = .089) indicates that the association between treatment condition and argumentation performance varied according to baseline critical-thinking level. This interaction does not invalidate the primary ANCOVA because critical thinking was not included as a covariate in that model. It nevertheless demonstrates treatment-effect heterogeneity and therefore qualifies interpretation of the overall adjusted condition effect. Consequently, the ANCOVA results in Table 6 should be interpreted with caution, as the overall condition effect may mask meaningful subgroup variation. Follow-up analyses using the Johnson-Neyman technique are recommended to identify the range of critical thinking scores across which the treatment effect is statistically significant. P-values are reported without correction for multiple comparisons. Applying a Bonferroni correction across the 13 tests conducted in this study (5 TAP components + 5 discourse indicators + 3 moderator interactions; adjusted α = .05/13 = .0038) would render the following results non-significant at the adjusted threshold: Evidence quality (p = .015), Warrant construction (p = .012), and Prior GPA × Treatment (p = .018). Rebuttal skills, Counter-evidence evaluation, and Critical thinking × Treatment remain significant under the corrected threshold.

Inferential statistics regarding the effectiveness of the applied AI-based instructional module on posttest results are presented. Higher initial achievement and better critical thinking skills predicted greater benefit from the instructional activity that applied the AI technology. These findings suggest that prerequisite skills may be critical for learning from such AI applications.

5 Discussion

This study aimed to empirically evaluate the didactic potential of generative artificial intelligence in developing scientific argumentation skills, specifically rebuttal skills—considered among the most cognitively demanding components of the PISA scientific literacy framework (OECD)—among prospective biology teachers. Based on experimental design principles emphasized by Creswell (2014), covariance analysis (ANCOVA) of obtained data demonstrated that AI intervention modeled as a “Socratic tutor” achieved statistically significant advantages in developing students' argumentative competency compared to traditional collaborative learning (Tabachnick and Fidell, 2013). Below, pedagogical, cognitive, and technological interpretations of these results are analyzed.

5.1 Cognitive conflict and epistemic cognition changes

This study yields three noteworthy findings that may contribute to AI education theory and argumentation pedagogy, pending replication in other contexts: Unlike traditional interventions (Chen and Wu, 2020; Wecker and Fischer, 2014) that show uniform improvement across all argumentation components, AI-mediated instruction produced dramatically asymmetric gains. Rebuttal skills (d = 0.88) and counter-evidence evaluation (d = 0.98) showed large effects, while foundational skills (claim formation, d = 0.33) demonstrated modest improvement. This pattern suggests that the structured Socratic protocol delivered through the AI-mediated instructional package may be particularly effective at targeting cognitively demanding processes, while assuming students possess basic argumentation skills. The experimental group demonstrated 2.2-fold faster learning rates (0.214 vs. 0.096 points/week) compared to traditional methods. This acceleration contradicts concerns about AI-induced “cognitive offloading” and instead supports Molenaar's (2022) “hybrid regulation” theory, suggesting AI can enhance rather than replace student cognitive processing. Qualitative discourse analysis revealed notable improvements within this sample in epistemic sophistication: acknowledgment of limitations increased from 15.4% to 78.8%, and epistemic hedging improved from 31.2% to 72.1%. These indicators suggest promising short-term gains in epistemic sophistication (Sinatra et al., 2014) that are directionally consistent with patterns typically associated with extensive graduate-level research training (Sandoval, 2014); however, given the single 8-week, single-institution design, these should be interpreted as preliminary and warrant longitudinal follow-up before concluding lasting epistemic transformation. The dual assessment approach (quantitative TAP + qualitative discourse analysis) provides convergent validity unavailable in previous studies, offering a methodological approach that may usefully inform future argumentation research.

• Asymmetric Skill Development Pattern

• Accelerated learning trajectory

• Epistemic Transformation Indicators

• Methodological Strength

An important alternative explanation is that the observed gains reflect targeted deliberate practice rather than a unique effect of AI. The AI condition received prompts explicitly designed to elicit rebuttals, counter-evidence evaluation, and cognitive conflict, whereas the comparison condition participated in less structured peer discussion. Therefore, the present comparison estimates the effect of the complete AI-supported instructional package and cannot isolate the contribution of AI technology from the contribution of structured rebuttal practice.

5.2 Self-regulated learning and AI

While this study did not directly measure self-regulated learning constructs, several indicators from our data suggest metacognitive development aligned with Vygotsky's (1978) Zone of Proximal Development theory. The significant increase in germane cognitive load (d = 1.22) indicates students engaged in productive cognitive processing rather than superficial task completion. Additionally, the improved acknowledgment of limitations (15.4% → 78.8%) and epistemic hedging (31.2% → 72.1%) demonstrate enhanced metacognitive awareness of knowledge boundaries.

The weekly progression data showing accelerated learning rates (experimental group: 0.214 points/week vs. control: 0.096 points/week) suggests that the AI-mediated instructional package — through its structured, cognitive-conflict-inducing design — created optimal challenge levels within students' developmental zones, although this effect cannot be isolated from that of the structured practice it embeds. Each AI interaction required students to examine their reasoning processes, effectively providing the “mirror” function that Zimmerman (2002) identifies as crucial for metacognitive development.

These findings, while not definitive measures of self-regulated learning, provide empirical support for AI's role in fostering reflective thinking processes essential for independent learning capabilities.

5.3 “Cognitive offloading” and information ethics

In Baidoo-Anu and Ansah (2023), they mention that new publications suggest that AI could be the cause of students becoming lazy. We have demonstrated that this is not the case when using the API in a way where the student uses the API as a sparring partner, rather than a knowledge source. In order to answer the AI counterattacks the students had to consult external scientific sources (PubMed and ScienceDirect) and provide evidence for their statements, which we consider as developing the type of textual literacy described by Rouet and Britt (2018). This scenario also follows the human first concept of “AI ethics” as defined by Floridi and Cowls (2019).

6 Recommendations and conclusions

6.1 Policy recommendations

Based on empirical research results and their analysis, the following scientific-practical recommendations for integrating GenAI into higher pedagogical education systems have been suggested. These recommendations encompass micro (learning process), MESO (software provision), and macro (educational policy) levels. To implement GenAI that enhances pedagogical effectiveness, higher education institutions are advised to integrate the discursive partner GenAI into the Curriculum Assures Future Talent (CAFT) - Biology Teaching Methodology, and Research Foundations. Presently, technology is mainly used as a tool to search for information, even in institutions of higher learning. Future students will be required to utilize technology as a trainer for simulation-based learning, which would aid in honing their analytical, synthesizing and evaluation skills, thus equipping them for future assessments as demonstrated by the PISA and TIMSS tests.

• Curriculum Integration

Furthermore, Higher education institutions are recommended to incorporate GenAI “discursive partner” modules into “Biology Teaching Methodology” and “Research Foundations” curricula. Technology should be used not merely as an information search tool but as a simulation trainer preparing students for high-level thinking operations (analysis, synthesis, evaluation) required by international assessment programs like PISA and TIMSS. Replacing portions of traditional seminars with “Human-AI debate sessions” enables individualized approaches for each student amid instructor shortages. Research demonstrates that AI effectiveness directly depends on system instructions provided. Therefore, developing “Pedagogical Prompt Engineering” skills in prospective teachers is essential. This involves not merely writing technical commands but competently directing AI toward didactic goals (e.g., “role-playing,” “creating cognitive conflict,” “simplified explanation”). Universities should teach students to design their “virtual teaching assistants” and configure them methodologically correctly. Considering AI hallucination risks (false information generation), mandatory “Fact Verification Protocols” must be integrated into educational processes. Students should be taught to approach AI interaction based on “zero trust” principles. After each scientific debate, students should validate AI arguments against primary sources (textbooks, scientific articles) before acceptance, this should become an educational standard. This approach develops information consumption culture and critical immunity in students. Shifting primary emphasis in student knowledge assessment from “fact recall” to “argumentation quality” is recommended. This involves the digitization of the Argumentation TAP assessment rubrics and the integration of these into learning activities. It is important to note that students should not be marked on their essays that use the aid of technology but rather on the arguments they develop and the tactics they use while engaged in debates with technology and on their ability to refute technology's counterarguments. In our research, we used a global LLM (GPT-4). However, there are more than enough reasons to develop and implement local “Edu-AI” models that take into account the unique content, cultural context and academic language use in Uzbekistan. This not only removes language barriers but enables deeper study of regional biological characteristics (e.g., Uzbekistan's flora and fauna).

• Pedagogical Prompt Engineering

• Fact Verification Protocol

• Process-Oriented Assessment

• Local “Edu-AI” Models Integration

6.2 Conclusion and implications of this research

This study demonstrates that a structured, Socratic-style instructional package delivered through Generative Artificial Intelligence is associated with statistically significant and practically meaningful improvements in scientific argumentation skills among prospective biology teachers, although the specific contribution of the AI technology versus the structured pedagogical design embedded in the intervention cannot be isolated with the present design. The research contributions have immediate implications for educational practice, policy, and theory. The findings provide empirical evidence supporting AI integration into teacher preparation programs as a scalable solution for developing complex argumentation competencies. Universities can implement “Human-AI collaborative learning” models that reduce instructor workload while enhancing student outcomes, addressing the global challenge of expanding higher education access without compromising quality.

• Educational Practice Implications

Educational policy implications: results support policy shifts from traditional “content transmission” to “cognitive skill development” paradigms. The demonstrated effectiveness of the structured AI-mediated argumentation training package (d = 0.67 overall effect) supports institutional investment in educational AI infrastructure. However, this investment should be pursued alongside, not as a substitute for, investment in developing instructors' own pedagogical capacity to design and deliver structured Socratic and counter-argumentation protocols, since the present findings cannot determine how much of the observed benefit derives from the AI technology itself versus the structured pedagogical design it implements. The study contributes to three theoretical domains: (1) Cognitive Conflict Theory—suggesting that a structured, technology-mediated Socratic protocol may generate more targeted cognitive dissonance than an unstructured peer-discussion protocol; (2) Zone of Proximal Development—illustrating how AI-mediated delivery can provide individualized scaffolding at scale; (3) Epistemic Cognition—showing how AI interactions transform students' understanding of knowledge construction processes. These findings offer preliminary, context-specific evidence relevant to ongoing discussions about AI integration in teacher education (e.g., UNESCO's guidance). However, generalizability beyond this single institution, country, and discipline remains untested, and claims about national-scale or cross-cultural impact should be treated as hypotheses for future multi-site research rather than established conclusions. The PISA-aligned methodology enables international replication and comparison, enabling developing nations to rapidly improve educational quality without massive human resource investments.

• Theoretical Implications

• Potential Broader Implications (Requiring Further Validation)

Long-term educational transformation: this research establishes foundations for shifting from instructor-dependent to technology-augmented pedagogical models, enabling developing nations to rapidly improve educational quality without massive human resource investments.

7 Limitations and future research directions

Several limitations need to be acknowledged in this study. First, the intervention period was limited to 8 weeks, which may not have been sufficient for students to develop stable and transferable argumentative practices. A longer duration would likely provide a more accurate picture of the sustained effects of generative AI-supported instruction. Second, technological limitations remained, particularly the risk of AI hallucinations. This underscores the importance of using generative AI with careful pedagogical oversight, since issues of explainability and reliability remain significant concerns in educational settings, as noted by Luckin et al. (2016). Third, the findings may be difficult to generalize beyond biology education. It remains unclear whether the same instructional strategy would produce similar effects in other disciplines, such as the humanities or social sciences, and this should be examined in future studies. Control-condition non-use of generative AI was based on participant agreement rather than objective usage monitoring; possible contamination therefore cannot be ruled out. In addition, implementation fidelity was monitored asymmetrically across conditions: 15% of AI-condition dialogues were audited for protocol adherence, whereas no equivalent structured fidelity check was conducted for the control group's peer discussions. This asymmetry in monitoring rigor constitutes an additional limitation, as it cannot be ruled out that differences in observed outcomes partly reflect differences in the degree of structured oversight rather than the instructional medium itself.

Future research should therefore consider extended implementations, stronger safeguards against hallucination, and cross-disciplinary replication. Studies with larger and more diverse samples would also help clarify the broader applicability of generative AI for developing students' argumentation skills.

An alternative explanation for this asymmetric pattern must be acknowledged: the AI's structured Socratic prompts explicitly and repeatedly required students to construct rebuttals and evaluate counterevidence (see Section 3.4.3), while the control group's general peer discussion protocol did not include an equivalent, deliberately structured rebuttal task. This asymmetry means the comparison in this study is best understood as one between a technology-mediated, structured pedagogical package and an unstructured peer-discussion protocol, rather than a direct comparison between artificial intelligence and human tutoring capacity. A human tutor following the same Socratic, systematically structured protocol could plausibly generate similar cognitive conflict; this study cannot rule out that possibility. The observed gains in rebuttal and counter-evidence skills may therefore partly reflect targeted deliberate-practice effects specific to these skills, rather than a general property of AI-mediated cognitive conflict being superior to peer interaction per se. Future research should include a peer-discussion condition given an equivalent structured rebuttal protocol (without AI) to disentangle the medium effect from the practice-structure effect.

Future research should employ a factorial design that crosses medium (AI-mediated vs. human-delivered) with pedagogical structure (structured Socratic/counter-argumentation protocol vs. unstructured discussion), which would allow the unique contribution of the AI technology to be disentangled from that of the structured pedagogical protocol it delivers.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Ana Mouta, University of Porto, Portugal

Reviewed by: Adrian Bagas Damarsha, Surabaya State University, Indonesia

Tiara Swastika Putri, Yogyakarta State University, Indonesia