A techno-mathematical literacy scale for engineering students: development and validation (E-TmLS)

1 Introduction

The rapid development of digital technologies, automation systems, and data-driven decision-making processes has significantly transformed the nature and use of mathematics. Furthermore, the widespread adoption of artificial intelligence (AI)-supported systems has recently created an urgent need for individuals to critically evaluate mathematical outputs generated by automated technologies (Holmes et al., 2022; Luckin et al., 2016). This transformation highlights that mathematics should no longer be viewed solely as a procedure-oriented discipline, but rather as a context-sensitive, interpretative, and technology-integrated practice (Kent and Noss, 2001). In this context, techno-mathematical literacy (TmL) refers to individuals’ ability to interpret, evaluate, and apply mathematical knowledge in technology-rich environments (Hoyles et al., 2010).

TmL extends beyond performing correct mathematical procedures and includes the ability to make sense of and critically evaluate outputs generated through technological tools (Hoyles et al., 2010). Accordingly, TmL is conceptualized as a multidimensional construct consisting of interrelated components such as data literacy, representation, communication, error awareness, and technology use (van der Wal et al., 2017).

This multidimensional structure becomes particularly evident in engineering education, where mathematical processes are largely mediated by technological tools. Engineering problems typically involve complex datasets, modeling processes, and software-based analyses, which may obscure the visibility of underlying mathematical reasoning (Hoyles et al., 2010). Engineering students represent a distinct and critical group for this research because their discipline is uniquely characterized by high-stakes decision-making where mathematical reasoning is often embedded and obscured within complex ‘black-box’ software environments (Hoyles et al., 2010; Kent and Noss, 2001). Therefore, individuals are expected not only to perform mathematical procedures but also to interpret and critically evaluate technology-generated outputs.

Although interest in TmL has increased, the existing literature has largely focused on conceptual discussions and performance-based assessment approaches (van der Wal et al., 2017, 2023). Notably, the literature explicitly highlights the lack of psychometrically grounded measurement instruments for assessing TmL (van der Wal et al., 2023). Existing scale development efforts (e.g., Demir and Tortop, 2026) have primarily been validated with pre-service teachers and do not fully capture the domain-specific and multidimensional nature of engineering contexts. Moreover, techno-mathematical literacy is recognized as a professional competency that begins its foundational development specifically during the years of higher engineering education, rather than post-graduation (van der Wal et al., 2017; van der Wal et al., 2023). Therefore, there is a clear need for valid and reliable measurement tools that reflect the multidimensional and context-specific nature of TmL in engineering education. This study aims to develop and validate a TmL scale tailored to engineering students. Engineering students, rather than practicing engineers, were chosen as the target population because TmL is understood to begin developing during formal engineering education, making this group particularly suited to examining its early formation. The growing integration of artificial intelligence-supported tools into engineering software and modelling environments further underscores the urgency of equipping students with the ability to critically interpret and evaluate technology-generated mathematical outputs.

2 Literature review

2.1 Techno-mathematical literacy in engineering education

Advances in digital technologies, automation systems, and data-driven decision-making processes have significantly transformed the way mathematics is used and its role in engineering applications. Today, mathematics is no longer viewed merely as a discipline consisting of procedural operations; rather, it is regarded as a context-sensitive and interpretive field of application that is integrated with technology (Kent and Noss, 2001). Particularly in an engineering context, mathematical processes are largely carried out through software systems, modeling tools, simulation environments, and data analysis technologies (Hoyles et al., 2010). This situation requires individuals not only to perform mathematical operations but also to interpret and evaluate the mathematical outputs generated by technology. Technological tools not only support mathematical operations but also shape the ways in which individuals interpret, represent, and communicate mathematical knowledge (Kent and Noss, 2001).

TmL is defined as the ability of individuals to interpret, evaluate, communicate, and apply mathematical knowledge in technology-rich environments (Hoyles et al., 2010). Unlike traditional mathematical literacy, TmL focuses on how mathematical processes are represented, transformed, and used via technology. Therefore, TmL encompasses not only mathematical computational skills but also data interpretation, technology-supported reasoning, and critical evaluation processes (van der Wal et al., 2017).

One of the primary reasons for the emergence of TmL is the increasing invisibility of mathematics in contemporary work environments. Hoyles et al. (2010) note that mathematical reasoning processes are carried out embedded within technological systems and automated software environments, and thus mathematics is often not directly visible to the user. This situation demonstrates that individuals are often forced to interpret and evaluate mathematical outputs generated by technology without directly observing the underlying mathematical operations. Consequently, in today's engineering contexts, knowledge of mathematical procedures alone is insufficient; the ability to critically analyze technology-supported mathematical outputs has also become crucial.

Some studies highlight significant differences between school mathematics and the mathematical applications used in engineering work environments (Hoyles et al., 2010; Kent and Noss, 2001). In engineering applications, mathematics often does not take the form of direct use of formulas; rather, it manifests through data, graphs, simulations, software outputs, or modeling processes. This situation demonstrates that engineering students require not only formal mathematical knowledge but also the ability to interpret technology-supported mathematical processes.

The structure of TmL in an engineering context has been examined in detail, particularly by van der Wal, Bakker, and Drijvers (2017). Based on interviews with engineers, the researchers noted that TmL consists of seven core competency areas: data literacy, technical software skills, technical communication skills, sense of error, sense of number, technical creativity, and technical drawing skills. This classification demonstrates that mathematical competence in engineering applications cannot be reduced solely to the ability to perform calculations; rather, it must be considered in conjunction with technology-mediated processes such as interpreting data, using software, evaluating outputs, and engaging in technical communication (van der Wal et al., 2017). The six dimensions adopted in the present study build directly on this framework: data literacy and sense of number map onto Data Literacy, sense of error onto Error Analysis and Evaluation, technical software skills onto Technology-Supported Processes, technical drawing and creativity onto Mathematical Representation, and technical communication onto Mathematical Communication and Decision-Making and Justification, with the latter also reflecting the workplace decision-making emphasis of van der Wal et al. (2023).

Subsequent studies support the view that TmL is an important learning objective in engineering education. In particular, van der Wal et al. (2019) emphasize the importance of technology-supported and context-based mathematics teaching processes for developing TmL in engineering students. Similarly, van der Wal et al. (2023) view TmL as an integrated framework comprising mathematical knowledge, workplace knowledge, information and communication technology knowledge, and communication skills. The researchers note that it is not sufficient for engineering students to merely apply mathematical procedures; they must also possess the ability to interpret, evaluate, and communicate mathematical knowledge generated by technology (van der Wal et al., 2023). This approach demonstrates that TmL is a multidimensional, context-sensitive construct.

Engineering education, which makes extensive use of technology-supported mathematical processes, provides a highly suitable context for studying TmL. Engineering students frequently work with large datasets, computer-aided modeling systems, simulation tools, and technology-supported analysis processes (Hoyles et al., 2010; van der Wal et al., 2023). This necessitates that students integrate their mathematical reasoning processes with technological interpretation skills. Furthermore, the widespread adoption of digital technologies and AI-supported systems has further increased the need for individuals to critically evaluate mathematical outputs generated by technology (Holmes et al., 2022; Luckin et al., 2016). In this regard, TmL is considered a competency area closely related to 21st-century skills such as critical thinking, data literacy, and computational reasoning.

2.2 Measuring techno-mathematical literacy

Despite growing interest in TmL, it appears that research on measuring this construct remains limited. Early studies in the literature have largely focused on explaining the theoretical structure of TmL and identifying professional competencies in an engineering context (Hoyles et al., 2010; Kent et al., 2005; van der Wal et al., 2017). In particular, van der Wal et al. (2017) identified the basic TmL that engineers need in their work environments; however, they provided limited explanations regarding how these competencies can be measured from a psychometric perspective. This situation highlights the need for valid and reliable tools for measuring TmL.

One of the main reasons it is difficult to measure TmL is that the construct is multidimensional and context-sensitive. van der Wal, Bakker, and Drijvers (2023) note that TmL is a heterogeneous construct consisting of a combination of mathematical knowledge, workplace knowledge, information and communication technology knowledge, and communication skills. The researchers emphasize that in the assessment tool they developed for engineering students, some dimensions are more difficult to measure than others, and that TmL cannot be fully assessed using traditional mathematics achievement tests (van der Wal et al., 2023). This indicates that TmL cannot be reduced to procedural mathematical knowledge alone; rather, it is a multidimensional competency area that also encompasses technology-supported processes of interpretation, evaluation, and communication.

A significant portion of the literature on measuring TmL involves performance tasks, digital tool-based applications, or context-based problem situations. For example, Ovadiya (2023) notes that problem-posing and problem-solving processes using digital tools contribute to the development of TmL among teacher-researchers. Similarly, Kartika and Hastari (2022) examined which categories of TmL are important for teachers in the digital age and emphasized that data interpretation, digital tool use, and technology-supported mathematical communication skills stand out in particular. These studies demonstrate that TmL is not merely a theoretical construct; rather, it is a practical competency area that develops through interaction with digital tools.

Studies on the role of technology in mathematics education also offer significant theoretical contributions to the measurement of TmL. Sinclair and Robutti (2016) note that digital technologies not only support mathematical thinking but also transform mathematical representations and learning processes. Similarly, in their meta-analysis examining the functions of technology in mathematics education, Ladel and Kortenkamp (2019) state that technology restructures mathematical discovery, representation, and communication processes. Drijvers and Sinclair (2024), on the other hand, emphasize that the role of digital technologies in mathematics education is increasingly shifting toward critical evaluation, interpretation, and technology-supported reasoning processes. These studies demonstrate that TmL is not merely limited to the use of technology; rather, it is a framework that encompasses how technology shapes mathematical thinking.

The measurement of TmL is also linked to the literature on digital literacy and 21st-century skills. The DigComp 2.1 framework, developed by the European Commission (2017), emphasizes the importance of individuals’ ability to use digital technologies in a critical, safe, and problem-solving-oriented manner. Similarly, the OECD (2018) highlights data literacy, critical thinking, and technology-supported problem-solving skills as key components of future educational competencies. TmL's focus on technology-supported mathematical reasoning processes demonstrates that this framework is closely related to digital literacy and 21st-century skills.

However, existing studies on TmL have largely focused on theoretical discussions, performance tasks, and context-based applications. In particular, it is noteworthy that there is a limited number of psychometrically robust assessment tools capable of evaluating technology-supported mathematical processes in a multidimensional manner, especially for engineering students (van der Wal et al., 2023). Furthermore, although a significant portion of existing studies focuses on defining competencies in workplace contexts, there are very few scale-development studies assessing engineering students’ technology-supported mathematical competencies using self-report measures and a multidimensional approach. Demir et al. (2023) conceptualized techno-mathematical literacy as an integrated construct combining mathematical thinking and technological competencies and proposed a theoretical framework for the concept. However, the limited availability of psychometrically robust measurement tools that can evaluate technology-supported mathematical processes multidimensionally, particularly in engineering contexts, represents a significant gap in the literature. The widespread adoption of digital technologies, data-driven engineering applications, and artificial intelligence-supported systems makes the valid and reliable measurement of engineering students’ techno-mathematical literacy levels even more critical. Therefore, a technology-mathematics literacy scale that is specific to the engineering context, multidimensional, and psychometrically robust should be developed.

2.3 Research gap

A review of the literature on TmL reveals that the concept is largely addressed through theoretical frameworks and performance-based assessments. Studies highlighting the multidimensional nature of TmL (Hoyles et al., 2010; van der Wal et al., 2017) emphasize that this competence includes components such as data interpretation, representation, communication, and technology-supported mathematical processes. However, comparatively little attention has been paid to the development and psychometric validation of standardized measurement instruments.

Recently, some initiatives aimed at measuring TmL have attracted attention. In particular, the measurement tool developed by van der Wal et al. (2023) offers a significant contribution to evaluating the development of techno-mathematical literacy within a specific learning environment. However, these instruments are generally designed for specific instructional contexts and therefore cannot be readily used as standardized measures across diverse engineering education settings. Specifically, whereas van der Wal et al. (2023) validated their instrument within a single innovative mathematics course for future engineers and Demir and Tortop (2026) targeted pre-service teachers, the E-TmLS is validated across multiple engineering programmes and departments, positioning it as a more broadly applicable, standardized alternative.

Existing scale development studies have mostly been conducted with prospective teachers and do not adequately reflect the distinctive characteristics of engineering education. Engineering education provides a particularly appropriate context for examining techno-mathematical literacy because mathematical reasoning is routinely embedded within modelling software, simulation environments, data analytics, and other technology-supported engineering practices. These characteristics necessitate a re-examination of techno-mathematical literacy as a multidimensional construct. Accordingly, engineering students are expected not only to perform mathematical procedures but also to interpret, evaluate, communicate, and justify technology-generated mathematical information in support of engineering decision-making. Nevertheless, few measurement tools specifically developed for engineering students adequately capture the multidimensional nature of techno-mathematical literacy while also providing comprehensive psychometric evidence of validity and reliability. As a result, researchers currently lack a standardized instrument for consistently assessing techno-mathematical literacy across engineering education contexts. Furthermore, the absence of such an instrument limits empirical research examining the relationships between techno-mathematical literacy and important educational variables such as mathematical modelling, computational thinking, digital competence, and technology-supported mathematical practices.

Although previous studies have identified important TmL and provided valuable conceptual frameworks, relatively little attention has been paid to examining whether these competencies collectively represent a coherent multidimensional latent construct among engineering students. Consequently, the dimensional structure of techno-mathematical literacy in engineering education remains insufficiently understood. At the same time, the literature lacks psychometrically robust and standardized instruments capable of assessing techno-mathematical literacy across diverse engineering education contexts. Existing measurement initiatives have generally been developed for specific instructional environments and therefore provide limited opportunities for broader empirical investigation. Together, these theoretical and measurement-related limitations restrict the systematic study of techno-mathematical literacy and hinder the accumulation of comparable evidence across engineering education research.

Accordingly, this study aims to develop and validate the Techno-Mathematical Literacy Scale for Engineering Students (E-TmLS). By providing a psychometrically validated multidimensional instrument, this study seeks to facilitate future empirical research and contribute to a more systematic assessment of technology-supported mathematical competencies in engineering education.

Specifically, the study addresses the following research questions:

RQ1: What is the factor structure of the techno-mathematical literacy scale developed for engineering students?

RQ2: Does the proposed E-TmLS demonstrate satisfactory psychometric properties in terms of construct validity, reliability, convergent validity, discriminant validity, model fit, and temporal stability?

3 Method

3.1 Research design

This study employed a quantitative research design based on psychometric scale development procedures. The study aimed to develop and validate a TmL scale for engineering students by examining its factorial structure, reliability, and validity. The scale development process followed established procedures in measurement development literature, including item generation based on the theoretical framework, expert review for content validity, pilot implementation, exploratory factor analysis (EFA), and validation of the measurement model using partial least squares structural equation modeling (PLS-SEM) (DeVellis, 2017; Hair et al., 2019). The study adopted a cross-sectional survey design in which data were collected from engineering students at a single point in time. In line with recommended scale development procedures, separate subsamples were used for exploratory and validation analyses.

3.2 Participants

The study involved 360 engineering students selected through purposive sampling from different engineering departments at public universities in Türkiye. To strengthen the construct validation process, the sample was randomly divided into two independent subsamples. The first subsample (n = 172) was used for exploratory factor analysis (EFA), whereas the second subsample (n = 188) was used to validate the factor structure using partial least squares confirmatory composite analysis (PLS-CCA). Participants were drawn from industrial engineering, computer engineering, and electrical and electronics engineering programs. The majority of the participants were male (73.4%), and most were first-year students (74.5%). In addition, most participants reported daily technology use between 3 and 6 h. The size of each subsample was considered adequate given established guidelines for scale development: for the 20-item final version, the EFA subsample (n = 172) exceeded the commonly recommended ratio of five to ten cases per item, and the PLS-CCA subsample (n = 188) exceeded the minimum sample-size thresholds recommended for PLS-SEM models of comparable complexity (Hair et al., 2019). The predominance of first-year students is acknowledged as a feature of the sample that should be considered when interpreting the findings, and is revisited in the Limitations section.

Purposive sampling was preferred because the study focused on TmL within engineering education contexts. Engineering students were considered an appropriate sample because data interpretation, mathematical modelling, technology-supported analysis, and digital problem-solving processes play a central role in engineering education. Since TmL is closely associated with technology-rich mathematical practices, engineering education provides a relevant context in which to examine the construct's multidimensional structure. In addition, engineering students were preferred instead of practicing engineers because techno-mathematical literacy is considered a competency that begins to develop during engineering education (Hoyles et al., 2010; van der Wal et al., 2017). Therefore, examining engineering students may provide important insights into the early development of techno-mathematical literacy before transition into professional workplace settings (van der Wal et al., 2023).

3.3 Scale development process

3.3.1 Construct definition

The development of the E-TmLS began with a comprehensive review of the TmL literature. Previous studies conceptualized TmL as the ability to interpret, evaluate, communicate, and apply mathematical knowledge in technology-rich environments (Hoyles et al., 2010; van der Wal et al., 2017, 2023). Existing frameworks emphasize that TmL extends beyond procedural mathematical competence and encompasses data interpretation, technology-supported reasoning, mathematical communication, representation, and critical evaluation of technology-generated outputs.

Given the technology-intensive nature of engineering education, techno-mathematical literacy was conceptualized as a multidimensional construct reflecting engineering students’ ability to engage with mathematical information embedded within software systems, simulations, modelling environments, and data-driven technologies. Based on the literature, six dimensions were identified: Data Literacy, Error Analysis and Evaluation, Technology-Supported Processes, Mathematical Representation, Mathematical Communication, and Decision-Making and Justification. Collectively, these six dimensions were considered to represent the core manifestations of techno-mathematical literacy in engineering education because they capture how students interpret, evaluate, represent, communicate, and justify technology-generated mathematical information within engineering problem-solving contexts. Furthermore, these dimensions reflect the technology-mediated mathematical practices that engineering students frequently encounter in data analysis, modelling, simulation, and decision-making processes.

3.3.2 Item pool generation

Following construct definition, an initial pool of 35 items was generated. Item development was informed by the techno-mathematical literacy literature, particularly the work of Hoyles et al. (2010), van der Wal et al. (2017), van der Wal et al. (2023), and the theoretical framework proposed by Demir et al. (2023). The item pool was designed to reflect the multidimensional nature of techno-mathematical literacy in engineering educationParticular attention was given to representing engineering-specific technology-supported mathematical practices, including data interpretation, evaluation of software-generated outputs, mathematical representation, communication of quantitative information, and technology-assisted decision-making processes. The items were designed to capture students’ self-perceived competencies in technology-supported mathematical environments rather than traditional procedural mathematical skills. During the preliminary review process, conceptually overlapping items and items judged to be insufficiently aligned with the intended construct were removed, resulting in a 32-item draft version.

3.3.3 Expert review and content validation

To establish content validity, the draft scale was evaluated by five experts, including two mathematics education scholars, one engineering education specialist, one measurement and evaluation specialist, and one language expert. A structured evaluation form containing the categories “appropriate,” “should be revised,” and “should be removed” was used. Experts were also invited to provide qualitative feedback through open-ended comments. Based on expert recommendations, several items were revised linguistically, redundant items were removed, and wording modifications were made to improve clarity and contextual relevance. This process resulted in a 24-item version of the scale.

3.3.4 Pilot testing and item refinement

The revised 24-item instrument was administered to a pilot sample of 25 engineering students. The pilot study aimed to evaluate item clarity, comprehensibility, and preliminary psychometric performance. During the pilot implementation, participants were asked to read each item individually and explain how they interpreted its meaning. This process enabled the researchers to identify ambiguous expressions, unclear wording, and items that were interpreted differently from their intended meaning. Based on participant feedback, several items were revised to improve clarity and comprehensibility. In addition to this cognitive review process, item-total correlations, discrimination indices, and factor-loading patterns were examined. Items exhibiting weak psychometric properties, including low factor loadings or problematic cross-loadings, were evaluated using both statistical and theoretical criteria. As a result, four items were removed, yielding a final 20-item version of the scale. All items were scored on a five-point Likert scale ranging from Strongly Disagree (1) to Strongly Agree (5). In summary, the number of items decreased across four developmental phases: 35 items (initial pool) → 32 items (after preliminary review) → 24 items (after expert review) → 20 items (final version, after pilot testing and item refinement).

3.3.5 Validation study

The final 20-item version of the E-TmLS was administered to 360 engineering students enrolled in various engineering programs in Türkiye. Following recommended scale development procedures, the sample was randomly divided into two independent subsamples. The first subsample (n = 172) was used for EFA, whereas the second subsample (n = 188) was used for measurement model validation using PLS-CCA. Subsequent analyses focused on examining the factorial structure of the scale and evaluating evidence of reliability, convergent validity, discriminant validity, model fit, and temporal stability.

3.4 Data collection

Pilot and main application data for the study were collected in April and June 2025, respectively. Data were collected from engineering students enrolled in various state universities in Türkiye. During the pilot application, the clarity, comprehensibility, and applicability of the scale's items were evaluated, and necessary item analyses were performed on the data obtained. The pilot application was conducted in a classroom setting, and the purpose of the research was explained to the participants. Participants were clearly informed that participation in the study was voluntary; that they were not required to provide their names; that they should honestly select the option that best suited them; that the data would be used only for scientific purposes; and that their responses would be kept confidential. The application lasted 15–20 min. During the main application process, data were collected both face-to-face and online. The final form of the scale was administered to university students studying in different engineering departments. Consent for voluntary participation was obtained from all participants, and the researchers directly monitored the data collection process to ensure consistency and reliability in its implementation. The collected data were subjected to statistical analyses to examine the psychometric properties of the scale.

3.5 Data analysis

To obtain stronger evidence regarding construct validity, the total sample was randomly divided into two independent subsamples. The first subsample was used for EFA, and the second was used to validate the factor structure using PLS-CCA. Data were analyzed using IBM SPSS Statistics 29 and SmartPLS 4. Prior to factor analysis, the suitability of the dataset was evaluated using the Kaiser–Meyer–Olkin (KMO) measure and Bartlett's Test of Sphericity. EFA was conducted to examine the factor structure of the scale. Principal axis factoring with varimax rotation was employed during the extraction process to obtain a more interpretable factor structure. Items with low factor loadings or problematic loading patterns were evaluated and, when necessary, removed. Factor loadings of.50 and above were considered acceptable.

Principal axis factoring with Varimax rotation was used at the exploratory stage to identify the simplest, most interpretable item-factor structure for item reduction purposes, consistent with common practice in scale-development research (Tabachnick and Fidell, 2013). Varimax rotation was intentionally used at the exploratory item-reduction stage to obtain a simple and interpretable factor structure; the relationships among the resulting dimensions were subsequently examined independently in the validation subsample. The HTMT analysis for discriminant validity was conducted independently of the rotation choice, in the independent validation subsample. All HTMT values remained below the conservative.85 threshold, indicating that the six dimensions remained empirically distinguishable despite their shared higher-order variance.

In line with the conceptual structure of the scale, the constructs were specified as reflective measurement models because the indicators were considered manifestations of the underlying latent constructs rather than defining characteristics of those constructs. In reflective measurement models, changes in the latent construct are expected to cause changes in all indicators simultaneously (Hair et al., 2019). Following EFA, the resulting factor structure was validated using PLS-SEM with PLS-CCA. The significance of the PLS-CCA higher-order loadings was assessed using a bootstrap resampling procedure with 5,000 bootstrap samples. Bootstrap resampling was used to obtain standard errors, t-values, p-values, and 95% percentile-based confidence intervals for the higher-order loadings. A significance level of.05 was adopted. PLS-SEM was preferred due to its suitability for exploratory scale development studies, prediction-oriented measurement-model assessment, and validation of complex multidimensional and hierarchical constructs. PLS-CCA was selected because it was considered appropriate for the prediction-oriented validation strategy adopted in this study, namely exploratory scale development using separate EFA and validation subsamples, and for assessing the distinctiveness and joint representation of the dimensions within a multidimensional hierarchical structure (Hair et al., 2019; Henseler et al., 2016). This framing avoids implying that PLS-CCA is universally preferable to covariance-based CFA. In addition, PLS-SEM is considered appropriate for studies with relatively small sample sizes and does not require strict multivariate normality assumptions compared to covariance-based SEM approaches (Hair et al., 2019). In this study, techno-mathematical literacy was modeled as a higher-order hierarchical component model consisting of multiple interrelated reflective dimensions; therefore, PLS-CCA was particularly appropriate for simultaneously assessing the distinctiveness and joint representation of these dimensions within the higher-order E-TmLS construct.

The measurement model was specified as a reflective-reflective (Type I) higher-order construct, in which TmL is theorized to be reflected by its six first-order dimensions (DL, EAE, TSP, MREP, MCOM, DMJ), each of which is in turn reflected by its respective indicators. This specification was adopted because (a) at the first-order level, the items within each dimension are interchangeable manifestations of that dimension, consistent with the defining property of reflective indicators (Hair et al., 2019; Jarvis et al., 2003); and (b) at the second-order level, the six dimensions are conceptualized as correlated expressions of a single underlying propensity for technology-supported mathematical reasoning, rather than as distinct components that collectively define TmL. A formative (composite/Type II) higher-order specification was considered but not adopted because it would conceptualize the six dimensions as distinct components that collectively form the definition of TmL, which is not consistent with our theoretical framing of TmL as a unified, general competency (Hoyles et al., 2010; van der Wal et al., 2017). The reflective-reflective specification does not imply that the six dimensions are conceptually interchangeable; rather, they represent distinct but correlated manifestations through which the broader perceived techno-mathematical literacy construct is expressed.

The higher-order construct was estimated using the two-stage approach, which is particularly appropriate for higher-order models with an unequal number of indicators across first-order dimensions (Becker et al., 2012; Hair et al., 2017; Sarstedt et al., 2019). In the first stage, latent variable scores for each of the six first-order dimensions were obtained from the lower-order measurement model. In the second stage, these six latent variable scores were used as indicators of the second-order TmL construct. This approach was preferred over the repeated-indicators approach because the first-order dimensions contain an unequal number of items (three to four), which can otherwise bias the relative weighting of dimensions within the higher-order construct.

Reliability was evaluated using Cronbach's alpha and composite reliability (CR), whereas convergent validity was assessed using average variance extracted (AVE). CR values above.70 and AVE values above.50 were considered acceptable indicators of convergent validity. In addition, variance inflation factor (VIF) values were examined to assess potential multicollinearity among the constructs, and values below 5 were considered acceptable (Hair et al., 2019). Discriminant validity was examined using two approaches. First, the Fornell–Larcker criterion was employed to compare the square root of the AVE for each construct with the inter-construct correlations (Fornell and Larcker, 1981). Second, the heterotrait–monotrait ratio (HTMT) was calculated, with.85 adopted as the reference threshold. Internal consistency of the scale was assessed using Cronbach's alpha and composite reliability (CR). In addition, model fit was evaluated using the SRMR and NFI indices. Harman's single-factor test was also conducted to examine the potential presence of common method bias. In addition, test–retest reliability was assessed in a separate participant group to examine the temporal stability of the scale. This group consisted of 90 engineering students, independent of the main validation sample (N = 360). Participants were recruited from Kocaeli University, the same institution as the main sample. Of the 90 students who completed the first administration, 87 also completed the second administration and were retained for the test–retest analysis (attrition = 3.3%, n = 3); only matched pairs (n = 87) were included in the analyses reported below. The final form of the scale was administered twice, with a three-week interval between administrations. Paired-samples t-tests and Pearson correlation analyses were used to evaluate the stability of the measurements over time.

4 Findings

4.1 First working group: EFA

EFA was conducted to determine whether the scale items were grouped into meaningful, distinct factors. Principal axis factoring with Varimax rotation was employed to obtain a more interpretable factor structure. Items with factor loadings below.50 or problematic loading patterns were evaluated for potential removal based on both statistical and theoretical considerations. Additionally, item-total correlations were examined throughout the analysis. All 20 items demonstrated acceptable factor loadings (ranging from.583 to.837) and were retained in the six-factor structure.

Prior to EFA, the suitability of the dataset for factor analysis was assessed using the KMO measure and Bartlett's Test of Sphericity. The KMO value was.774, indicating acceptable sampling adequacy. Furthermore, Bartlett's Test of Sphericity was statistically significant (χ² = 758.092, df = 190, p < .001), demonstrating that the correlation matrix was appropriate for factor analysis.

Table 1 shows that the scale exhibits a six-factor structure, with all 20 items loading acceptably onto their intended factors (factor loadings ranging from.583 to.837). Across the six factors, the DL dimension showed the strongest representation (eigenvalue = 6.002, explaining 30.010% of the variance), whereas the DMJ dimension showed the weakest (eigenvalue = 1.179, explaining 5.893% of the variance); all six factors nonetheless met acceptable loading and eigenvalue criteria. When evaluated as a whole, the six-factor solution accounted for 67.236% of the total variance, indicating that the extracted factors captured a substantial proportion of variance in the observed item responses (Tabachnick and Fidell, 2013). These findings demonstrate the statistically supported multidimensionality of E-TmLS.

Table 1. Findings regarding the factor loading distributions of items based on factor analysis results.

DL, Data Literacy; EAE, Error Analysis and Evaluation; TSP, Technology-Supported Processes; MREP, Mathematical Representation; MCOM, Mathematical Communication; DMJ, Decision-Making and Justification.

4.2 Second working group: PLS-CCA

To validate the six-factor structure obtained from the EFA, PLS-SEM-based PLS-CCA was performed using SmartPLS 4. Accordingly, techno-mathematical literacy was modeled as a second-level structure consisting of interrelated sub-dimensions. Item loadings, Cronbach's alpha coefficient, CR, AVE, VIF, Fornell–Larcker criterion, and HTMT values were examined in the evaluation of the measurement model. Furthermore, model fit was assessed using the SRMR and NFI indices.

Table 2 shows that the factor loadings for all items on their respective constructs are above the acceptable threshold of.50. The Cronbach's alpha coefficients for the sub-dimensions of the scale range from.685 to.861, while CR values range from.810 to.916. Furthermore, all VIF values are below the recommended threshold of 5 (Hair et al., 2019). This finding indicates that multicollinearity is not present in the measurement model. Although Cronbach's alpha values for TSP (.689), MCOM (.685), and DMJ (.685) were marginally below.70, their corresponding composite reliability values exceeded.80, providing additional evidence of internal consistency. Moreover, AVE values above.50 indicate sufficient evidence of convergent validity for the scale. Overall, the findings indicate that the scale demonstrates acceptable psychometric properties with respect to reliability and construct validity.

Table 2. Factor loadings, reliability, and convergent validity findings related to the measurement model.

Examination of the Fornell–Larcker criterion in Table 3 indicates that the square roots of the AVE values for all constructs exceed the correlations between each construct and the others. This finding indicates that the sub-dimensions of the scale are distinguishable constructs. Therefore, the scale demonstrates sufficient discriminant validity.

Table 3. Findings regarding the Fornell–Larcker criterion.

Examination of the HTMT values in Table 4 shows that all values are below the recommended threshold of.85. This finding indicates that the scale has sufficient discriminant validity among its sub-dimensions. The results support the findings based on the Fornell–Larcker criterion and indicate that the scale has sufficient psychometric properties with respect to discriminant validity.

Table 4. Findings regarding HTMT values.

As shown in Table 5, analysis of the second-level structure indicated that all six first-order dimensions showed positive and statistically significant relationships with techno-mathematical literacy (TML). Based on 5,000 bootstrap resamples, all higher-order loadings were statistically significant (t = 8.89–22.27, p < .001), and none of the 95% confidence intervals included zero, providing empirical support for the significance of the higher-order structure. The higher-order loadings ranged from.582 (TSP) to.799 (EAE), with EAE showing the strongest empirical relationship with the higher-order construct and TSP the comparatively weakest, though still statistically significant, relationship.

Table 5. Findings regarding the second-level structure.

Figure 1 shows that techno-mathematical literacy is modeled as a second-level structure consisting of the dimensions DL, DMJ, EAE, MCOM, MREP, and TSP. All higher-order loadings in the measurement model were found to be positive and significant. Furthermore, the factor loadings for the sub-dimensions were at acceptable levels. These findings indicate that the techno-mathematical literacy scale is multidimensional and that its subdimensions significantly represent its superstructure. The R² values ranged from .338 for TSP to .638 for EAE and indicate the proportion of variance in each first-order dimension accounted for by the higher-order TML construct within the specified measurement model.

Examination of the model fit indices presented in Table 6 indicates that the SRMR value is close to the recommended threshold (.082), whereas the NFI value is slightly below the accepted limit (.884). It should be noted that well-established thresholds for the NFI have not yet been determined for composite-based (PLS-SEM) models, and its use in this context remains relatively uncommon and subject to ongoing methodological discussion (Henseler et al., 2014). However, the literature indicates that model fit indices, especially in PLS-SEM studies, should be evaluated within acceptable ranges rather than strict cutoff points (Hair et al., 2019). Furthermore, Hu and Bentler (1999) state that SRMR values below.10 may indicate acceptable model fit. Similarly, Marsh, Hau, and We (2004) emphasize that interpreting fit indices with strict threshold values may not be appropriate and that other validity and reliability indicators should also be considered in model evaluation. The model demonstrated adequate but not optimal global fit, with SRMR slightly above.08 and NFI below.90. These indices should therefore be interpreted cautiously alongside the broader measurement-quality evidence reported above, rather than as standalone indicators of model adequacy. However, Harman's one-factor test was applied to examine the potential impact of common method bias. Harman's single-factor analysis provided no indication that a single factor accounted for a dominant proportion of the variance (27.801%); however, because all variables were collected through self-report measures, common-method variance cannot be completely excluded.

Table 6. Findings regarding model fit indices.

Test-retest reliability refers to the stability of a measurement instrument over time and the consistency of scores obtained from the same individuals at different times (Cohen et al., 2018; DeVellis, 2017). A test–retest analysis was conducted to assess the temporal stability and consistency of the scale scores over time.

Table 7 presents results from the analysis evaluating the scale's stability over time. The findings indicate no statistically significant difference between the pre-test and post-test scores (p > .05). Furthermore, the Pearson correlation coefficients calculated between the two applications ranged from.534 to.855. The test-retest coefficients indicated moderate-to-strong temporal associations across dimensions, with the total E-TmLS score demonstrating a strong temporal association (r = .855). Overall, these findings provide evidence of temporal stability in E-TmLS scores over the three-week interval; however, the paired-samples t-tests and Pearson correlations assess mean-level change and temporal association, respectively, rather than direct agreement between measurements.

Table 7. E-TmLS test-retest findings.

5 Discussion

This study aimed to develop a self-report techno-mathematical literacy scale for engineering students and to examine its psychometric properties. The findings indicate that the E-TmLS demonstrates acceptable evidence of validity and reliability for assessing students’ perceived techno-mathematical literacy competencies. The final version of the scale consisted of 20 items grouped into six factors: Data Literacy (DL), Error Analysis and Evaluation (EAE), Technology-Supported Processes (TSP), Mathematical Representation (MREP), Mathematical Communication (MCOM), and Decision Making and Justification (DMJ). Together, these six factors explained 67.236% of the total variance, indicating that the six-factor structure accounted for a substantial proportion of the variance in the scale responses.

The factor structure obtained from the EFA was validated through PLS-CCA. The obtained values for CR, AVE, HTMT, the Fornell–Larcker criterion, and VIF indicate that the measurement model provides sufficient evidence of reliability and validity. Furthermore, second-level PLS-SEM results revealed that all sub-dimensions significantly represented the higher-order construct (techno-mathematical literacy). However, the use of separate subsamples for exploratory analysis and independent validation provided stronger evidence for the scale's construct validity, reduced the risk of overfitting, and supported the robustness of the factor structure.

The test–retest findings provide evidence of temporal stability in E-TmLS scores over the three-week interval. The absence of statistically significant mean differences suggests that group-level scores remained similar across administrations, while the moderate-to-strong Pearson correlations indicate consistency in the relative ordering of participants. Because these statistics assess mean change and association rather than agreement, the findings should not be interpreted as direct evidence of score agreement over time.

The resulting multidimensional structure aligns with previous conceptualizations of techno-mathematical literacy in engineering contexts, which emphasize that techno-mathematical literacy extends beyond mathematical processing skills and encompasses interpretation, evaluation, communication, and technology-assisted reasoning processes (Hoyles et al., 2010; van der Wal et al., 2017, 2023). This consistency provides additional support for the theoretical foundations of techno-mathematical literacy and suggests that the competencies identified in previous conceptual studies can also be represented as measurable dimensions within engineering education. Especially in engineering contexts, where mathematical processes are increasingly mediated through digital technologies, modelling systems, and AI-supported environments, techno-mathematical literacy appears to function as a multidimensional self-perceived competency rather than a collection of isolated skills. In this respect, the study contributes to the literature by addressing techno-mathematical literacy not only in the context of general mathematical competence but also in technology-assisted mathematical applications specific to engineering. The emergence of these six dimensions can be understood as a reflection of the interrelated demands engineering students face when working with technology-mediated mathematics: they must access and interpret data (DL), evaluate the plausibility of outputs (EAE), operate the software or modelling tools that generate them (TSP), represent findings mathematically (MREP), communicate them to others (MCOM), and justify decisions based on them (DMJ), as complementary, interrelated dimensions of technology-mediated mathematical practice rather than as a fixed temporal or causal sequence. For curriculum design, this pattern suggests that techno-mathematical literacy is unlikely to develop through isolated technical-skills training alone, and that engineering curricula may benefit from deliberately connecting data interpretation, software use, and decision-justification within the same technology-enhanced learning tasks. One possible interpretation is that the particularly strong representation of the EAE dimension within the higher-order structure reflects critically identifying and evaluating errors in technology-generated outputs as central to how engineering students experience techno-mathematical literacy, consistent with Hoyles et al.’s (2010) emphasis on the invisibility of mathematics within automated systems. The comparatively weaker, though still substantial, representation of TSP may reflect variability in students’ exposure to specific engineering software across departments and academic years, rather than a weaker conceptual link to the construct.

To further address the conceptual coherence of the empirically derived factor structure, Table 8 presents the item content within each factor alongside a conceptual rationale for their grouping.

Table 8. Conceptual rationale for the empirically derived factor structure.

5.1 Implications

The findings highlight the importance of assessing perceived TmL in engineering education, as these perceptions of TmL may play a critical role in engineering education. Engineering students today are expected not only to perform mathematical operations but also to interpret, evaluate, and integrate mathematical outputs produced in technology-supported environments into data-driven decision-making processes. In this context, the E-TmLS may serve as a useful instrument for assessing engineering students’ perceived techno-mathematical literacy. In addition, the scale may contribute to the evaluation of engineering curricula and educational interventions designed to support students’ perceived techno-mathematical literacy. Beyond its use as a measurement instrument, the E-TmLS may facilitate future research examining the antecedents and outcomes of techno-mathematical literacy in engineering education. For example, the scale may be used to investigate relationships between techno-mathematical literacy and variables such as mathematical modelling competence, computational thinking, digital competence, engineering design performance, and technology-supported problem-solving skills. In addition, the multidimensional structure of the scale enables researchers to examine which specific dimensions of techno-mathematical literacy are most strongly associated with educational outcomes in engineering contexts. Importantly, the availability of a standardized and psychometrically validated instrument may contribute to the accumulation of comparable evidence across engineering education studies and support theory development regarding the nature and development of techno-mathematical literacy. By providing a multidimensional framework for assessing techno-mathematical literacy, the E-TmLS may help advance both empirical research and educational practice in engineering education.

5.2 Limitations and future directions

This study has several limitations that should be considered when interpreting the findings. First, the study was conducted only among engineering students in Türkiye, which may limit the generalizability of the findings across cultural, institutional, and educational contexts. Future studies may examine the validity and reliability of the E-TmLS among engineering students across different countries and educational systems. Second, a substantial proportion of the participants were first-year students, which may have influenced participants’ perceptions of techno-mathematical literacy, given their limited professional and disciplinary experience. This limitation may be particularly relevant for dimensions such as TSP and DMJ, which presuppose exposure to advanced modelling software, simulation tools, and engineering decision-making contexts; first-year students’ responses to these dimensions may partly reflect anticipated rather than experienced competencies, and this should be considered when interpreting the relatively lower first- and second-order loadings observed for TSP. Future research may compare techno-mathematical literacy levels across different grade levels and engineering disciplines. Because the study employed a cross-sectional design, the stability of the factorial structure should be further examined using longitudinal and cross-validation studies. Although the scale demonstrated acceptable temporal stability through test–retest analysis, future studies may investigate the developmental nature of techno-mathematical literacy. Another limitation is that the study relied on self-reported data, which may increase the likelihood of response bias despite acceptable results on Harman's single-factor test. Future research may support self-report findings through performance-based assessments, observational data, or technology-supported task environments. The present study focused primarily on the psychometric development of the E-TmLS. Future studies may examine, within engineering education, the relationships between techno-mathematical literacy and variables such as academic achievement, digital competence, mathematical reasoning, and technology-supported problem-solving skills.

6 Conclusion

This study developed and validated the E-TmLS, a multidimensional scale for assessing perceived techno-mathematical literacy among engineering students. The scale is composed of six dimensions: Data Literacy, Error Analysis and Evaluation, Technology-Supported Processes, Mathematical Representation, Mathematical Communication, and Decision-Making and Justification. Regarding RQ1, the factor structure of the E-TmLS was established through EFA and subsequently validated through PLS-CCA in an independent subsample, confirming techno-mathematical literacy as a multidimensional, second-order reflective construct. Regarding RQ2, the scale demonstrated satisfactory reliability, convergent validity, discriminant validity, acceptable model fit, and temporal stability, supporting its overall psychometric soundness. These indicators should nonetheless be interpreted with appropriate caution, given the borderline Cronbach's alpha values observed for TSP, MCOM, and DMJ, and given that the test–retest findings reflect mean-level stability and association rather than direct score agreement. The E-TmLS is expected to serve as a standardized tool for researchers and educators seeking to assess technology-supported mathematical competencies and inform curriculum development in engineering education. In practice, instructors may use the E-TmLS to identify which techno-mathematical dimensions require greater instructional attention within a given course, curriculum developers may use it to benchmark programme outcomes against technology-supported mathematical competencies, and researchers may use it as a standardized measure for examining relationships between techno-mathematical literacy and other engineering education constructs. These contributions should nonetheless be interpreted in light of the study's cross-sectional, self-report design and its Türkiye-based, first-year-dominated sample. Future research is encouraged to validate the E-TmLS internationally and to test its measurement invariance across engineering disciplines, academic years, and cultural contexts.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2026.1930595/full#supplementary-material

Supplementary material Supplementary material at the source

Footnotes

Edited by: Maja Kerneža, University of Maribor, Slovenia

Reviewed by: Budi Utomo, Universitas PGRI Palembang, Indonesia

Enny Keristiana Sinaga, State University of Medan, Indonesia