Integration of an ontological model of CDIO competencies into the design of educational programs: transforming educational projects into knowledge content

1 Introduction

Engineering education has shifted from transmitting disciplinary knowledge toward developing professional competencies. The Conceive–Design–Implement–Operate (CDIO) initiative reflects that shift. Its central artifact, the CDIO Syllabus, outlines the knowledge, skills, and attitudes expected of beginning engineers across four sections: disciplinary fundamentals; personal and professional skills; interpersonal skills; and the capacity to conceive, design, implement, and operate systems in enterprise and societal contexts (Crawley, 2002; Crawley et al., 2007). The CDIO Standards add an integrated curriculum, design–build experiences, and alignment of outcomes with assessment (O'Connor et al., 2023). Comparative work has also mapped the Syllabus onto engineering-accreditation criteria such as Accreditation Board for Engineering and Technology (ABET) (Forcael et al., 2022).

In CDIO programs, competencies are developed primarily through educational projects, in which students conceive a need, design a solution, implement it, and operate the result. These projects are where personal, interpersonal, and system-building competencies are exercised together. The knowledge a student builds within a project is seldom captured in a structured, reusable form. It is scattered across briefs, design reports, prototypes, rubrics, and reflective journals, and it is expressed in the local vocabulary of each course. As a result, the relationship between what a program declares and what its projects deliver is mediated by expert judgment and spreadsheets.

This mediation is costly and fragile. Curriculum committees find it difficult to demonstrate, at the level of granularity accreditation requires, that each competency is addressed at an adequate level of proficiency and that it is assessed with evidences Coverage gaps and duplication stay hidden until program review. Prerequisite relations among knowledge elements developed across successive projects are rarely modeled, so ordering inconsistencies persist. Because each program encodes competencies and content in free text, comparing programs for credit transfer remains manual. These problems share a cause: competency frameworks such as CDIO offer rich human-readable guidance but no machine-interpretable semantics, while existing educational ontologies tend to model competencies and courses as static catalogs disconnected from the project-based pedagogy through which CDIO competencies are formed.

Ontologies offer a partial remedy. An ontology is an explicit specification of a shared conceptualization (Borgo et al., 2022; Studer et al., 1998) in OWL 2, grounded in description logic; it supports classification, consistency checking, and query answering (Baader, 2011; Horrocks et al., 2003; W3C OWL Working Group, 2012). Prior research has applied these technologies to competency modeling and curriculum analysis, using hierarchical competency representations and prerequisite reasoning (Milosz et al., 2024; Paquette, 2007; Rezgui et al., 2014). These models, however, bind competencies to courses or to pre-existing catalog entries and reason about prerequisites among competencies, without modeling the level at which a competency is attained or how it is generated: Milosz et al. (2024) reason over course-bound competency prerequisites but do not propagate an attained proficiency, Paquette (2007) formalizes competency as a skill applied to a knowledge entity but is not tied to project-based pedagogy, and Rezgui et al. (2014) target learning networks and learning objects rather than engineering projects. What is missing is a procedure that treats the educational project as a transformable source from which structured knowledge content can be derived and bound, with proficiency semantics, to a formal model of the CDIO framework.

We use the term knowledge content to refer to the structured representation of knowledge enacted in a project. A knowledge element is an atomic item: one concept, principle, method, or skill enactment. A knowledge-content unit (KCU) is a coherent cluster of related elements with a topic, an attained proficiency level, prerequisites, and supporting evidence. The KCU is therefore an aggregate rather than an atomic object. Converting a project into knowledge content means recovering these elements and clusters from project artifacts and linking them to the program's competencies. A KCU is deliberately not a reusable learning object in the IEEE LOM sense, nor a static knowledge object: those are authored, pre-packaged units designed for delivery and reuse, whereas a KCU is recovered from project artifacts after the fact, carries an attained proficiency level and a prerequisite structure, and is bound by provenance to the competencies a project actually develops. It is closest to a competency unit, but unlike existing competency-unit models, it makes the project-derived evidence and the attained-versus-required proficiency explicit rather than implicit. We therefore introduce it as a specialization of these prior notions rather than a replacement.

We pursue three research questions.

RQ1: Can the CDIO framework be formalized as a description-logic ontology that preserves its hierarchy and proficiency semantics while remaining decidable and efficient in practice at program scale?

RQ2: Can the conversion of projects into knowledge content be specified as a repeatable, parameterized procedure whose outputs are linked to competencies and learning outcomes?

RQ3: On one program, does the representation support proficiency-aware coverage, gap, and prerequisite analysis with mapping accuracy comparable to expert judgment?

2 Related work

2.1 The CDIO framework and its syllabus

CDIO frames engineering education around the lifecycle of products, processes, systems, and services. The Syllabus lists intended outcomes at several levels of detail and is meant to be customized, with stakeholders setting expected proficiency levels (Crawley, 2002; O'Connor et al., 2023). Section 1 covers disciplinary knowledge; Section 2 covers personal and professional skills such as analytical reasoning, experimentation, and ethics; Section 3 covers interpersonal skills such as teamwork and communication; Section 4 covers conceiving, designing, implementing, and operating systems. Comparative studies relate the Syllabus to accreditation criteria such as ABET (Forcael et al., 2022). For this study, the key fact is that the Syllabus is a natural-language taxonomy with no formal semantics and no computational interface.

2.2 Outcome-based education and constructive alignment

CDIO is a form of outcome-based education that designs curricula backward from the capabilities graduates should show (Yuan et al., 2024). Constructive alignment asks that intended outcomes, learning activities, and assessment form a coherent triad (Pereira et al., 2024), and outcomes are often tagged with cognitive levels from the revised Bloom taxonomy (Ramirez, 2017). Two requirements follow and recur in our method: an outcome should carry an explicit proficiency level, and it should be traceable up to the competency it serves and down to the content and evidence that realize it. In practice, this traceability is asserted informally, so it is neither verifiable nor reusable.

2.3 Ontologies and competing paradigms in educational modeling

Several paradigms address parts of the problem, and each has limits. Ontology engineering, with OWL 2, SWRL, and SPARQL (Antoniou and van Harmelen, 2009; Horrocks, 2004; Seaborne and Prud'hommeaux, 2008; W3C OWL Working Group, 2012) and methodologies such as Ontology Development 101 and NeOn (Noy and Musen, 2002; Suárez-Figueroa et al., 2012), provides formal semantics and reasoning; its cost is manual modeling effort and brittleness when source text is noisy. Educational data mining and curriculum mining infer structure from records and corpora at scale (Kalita et al., 2025), but the patterns they find are statistical and lack the formal, auditable semantics that accreditation needs. Learning analytics focuses on learner behavior and prediction rather than on the formal structure of competencies and content. Knowledge-graph and graph-embedding methods represent course and concept structure and support recommendation and skills-gap analysis (Abu-Salih and Alotaibi, 2024; Clemente et al., 2022; Yang et al., 2023), yet they typically optimize predictive utility over logical consistency, and prerequisite reasoning is implicit in the embedding rather than checkable. Ontology-learning methods extract concepts and relations from textbooks using Natural Language Processing (NLP) (Conde et al., 2016), which reduces modeling effort but introduces extraction error that must be controlled. Our approach sits in the ontology-engineering paradigm but borrows NLP for the extraction and annotation stages, and it keeps a human in the loop precisely because automatic extraction is error-prone.

2.4 Competency ontologies and curriculum analysis

Dedicated competency ontologies exist. Paquette (2007) formalized a competency as a skill applied to a knowledge entity at a performance level and built a management framework around it; the model is expressive but is not tied to project-based pedagogy. Rezgui et al. (2014) proposed an ontology-based approach to competency modeling in learning networks, oriented to training and learning objects rather than to engineering projects. Milosz et al. (2024) built an ontological model of a competency-based curriculum with arbitrarily nested competencies, course–period links, and requires/required by prerequisite dependencies, and implemented deduction over it in OWL/RDF using SPARQL and Python (Lamy, 2017); the model reasons about prerequisites among competencies but binds competencies to courses, not to content derived from projects, and it does not propagate an attained proficiency level. Recent CDIO-oriented ontologies map competencies to courses and content as a static domain model and note, as we do, that CDIO lacks a formal representation; they do not define a transformation from projects to content or proficiency-aware coverage metrics.

A common shortfall in this study is that competencies are linked to pre-existing catalog entries, usually courses, and that the level at which a competency is attained is not modeled. Our methodology differs in three ways. It binds competencies to knowledge-content units derived from projects, so the representation reflects how CDIO competencies are formed. It models an explicit transformation from projects to content and propagates proficiency through it so that the system can reason about attainment level, not just presence. And it adds proficiency-aware metrics computed by the reasoner. We do not claim this is the only way to close the gap; this is one defensible approach evaluated on one program.

3 Materials and methods

3.1 Research approach

The study follows design-science research: knowledge is produced by building and evaluating an artifact for a relevant problem (Hevner et al., 2004). The artifact is the CDIO–CKO methodology: an ontology, a transformation procedure, a metric family, and a reference architecture. Construction was iterative, alternating between formalizing the Syllabus and modeling real project artifacts from a partner program. Evaluation combined reasoner-based verification, a three-expert gold-standard comparison of the automated mapping with confidence intervals, ontology-quality measurement, a sensitivity analysis of method parameters, and a before-and-after coverage analysis. Development followed standard ontology-engineering guidance on scoping, reuse, conceptualization, formalization, and evaluation (Noy and Musen, 2002; Suárez-Figueroa et al., 2012).

3.2 Definitions and a naming convention

We distinguish four notions central to the proposed model. A competency is a program-level disposition to mobilize knowledge, skills, and attitudes in a class of situations, as defined in the CDIO Syllabus and qualified by an expected proficiency level. A learning outcome is a course- or project-level statement, assessable in itself, that operationalizes part of a competency; several outcomes may align with one competency, and a competency is not reducible to any single outcome. A knowledge-content unit carries knowledge: a cluster of knowledge elements with a topic, an attained level, prerequisites, and evidence. Evidence is an artifact that substantiates attainment. In the formal model, class names use upper camel case (for example, KnowledgeContentUnit), the abbreviation KCU denotes an instance of that class in prose, and the knowledge-content layer is the architectural layer (L3) that holds KCUs. Figures and tables use these names consistently.

An ontology is taken as a structure of classes with a subsumption order, relations with their own order, axioms, and individuals, as in Equation 1. The competency hierarchy is class subsumption, so a specific competency entails its ancestor, as in Equation 2. The reasoner uses subsumption to propagate coverage from specialized to general competencies.

O=〈C, ⊆C, R, ⊆R, A, I〉(1)

ci⊆cj⇔∀x(ci(x)→cj(x))(2)

3.3 The CDIO–CKO ontology

The ontological foundation formalizes the Syllabus and the entities that link competencies to projects and content. The four sections map to four top-level competency classes, each refined within the Syllabus hierarchy; leaf competencies are the targets to which knowledge content attaches. Table 1 records this mapping and the decision for each section. Figure 1 shows the schema. Table 2 lists the core classes, including the explicit separation of KnowledgeElement (atomic) from KnowledgeContentUnit (aggregate). Table 3 lists the properties, including their domains, ranges, and characteristics.

Table 1. Mapping of CDIO syllabus sections to CDIO–CKO ontology constructs.

Table 2. Core classes of the CDIO–CKO ontology.

Table 3. Principal object and data properties of the CDIO–CKO ontology.

Two axioms carry weight. The chain covers ° alignsWith ⊆ contributesTo lets the reasoner infer that a knowledge-content unit contributes to a competency when it covers an outcome aligned with that competency; a second chain, produces ° contributesTo ⊆ develops, lifts this to the project, so a project develops a competency when it produces a unit that contributes to it. Transitivity of requires gives the prerequisite closure of any unit, used later for ordering checks. The ontology stays within the SROIQ fragment of OWL 2 DL, so classification and consistency are decidable (Horrocks et al., 2003; W3C OWL Working Group, 2012). The SWRL rules of Section 3.6 are restricted to DL-safe form, which keeps reasoning decidable (Motik et al., 2005); this point is taken up again as a limitation.

3.4 The transformation procedure

The transformation maps an educational project to a set of KCUs and comprises four stages, as in Equation 3: decomposition δ, extraction ε, semantic annotation σ, and aggregation π. We call τ a parameterized procedure rather than a calculus, because its stages depend on language resources and thresholds and because two stages are only semi-automatic; τ is therefore a semi-automatic, human-in-the-loop transformation. Figure 2 shows the layered organization, and Table 4 summarizes the operations per layer.

τ:Proj→2KCUτ=π° σ°ε°δ (3)

Table 4. Operations of the transformation procedure across the architectural layers.

The workflow proceeds sequentially through the transformation layers. Educational projects are first decomposed into CDIO activities, from which candidate knowledge elements are extracted. These elements are annotated with the ontology and aggregated into Knowledge-Content Units (KCUs), forming the semantic knowledge layer. Finally, ontology reasoning and SPARQL-based analytics derive competency coverage, prerequisite relationships, and curriculum quality indicators that support educational decision-making.

3.4.1 Decomposition

Decomposition δ partitions a project into activities along the CDIO cycle, giving a finite activity set, as in Equation 4. The partition is driven by the project brief and rubric, which already name the phases, so it is reproducible across analysts.

δ(p)={a1,a2,…,an}⊆Act(p)(4)

3.4.2 Extraction

Extraction ε derives candidate knowledge elements from each activity and its artifacts, as in Equation 5. An element is a triple consisting of a textual descriptor, a type tag (concept, principle, method, or skill enactment), and a tentative cognitive level inferred from the action verbs in the rubric, following the revised Bloom taxonomy (Ramirez, 2017). Extraction combines rubric parsing with term recognition; multi-word terms are retained because dropping them is a known weakness of frequency-based extraction in specialized curricula (Conde et al., 2016).

ε(a)={e=〈te, typee, ℓe〉}(5)

3.4.3 Semantic annotation

Annotation σ anchors each element to the most similar entity in the ontology signature, as in Equation 6. Similarity is a hybrid measure (Equation 8): a taxonomic component based on Wu–Palmer (Equation 9; Wu and Palmer, 1994), and a distributional component given by the cosine similarity of sentence embeddings (Devlin et al., 2019; Reimers and Gurevych, 2019). Feature-contrast measures in the sense of Tversky (1977) offer an alternative formulation of similarity; we use the depth-based Wu–Palmer form because it operates directly on the subsumption order already present in the ontology. The embedding model is a pretrained multilingual sentence transformer (paraphrase-multilingual-mpnet-base-v2, 768-dimensional), chosen so that descriptors in the program's working language and in English map into one space; we did not retrain it. Preprocessing lower-cases text, strips markup, normalizes domain terms against a controlled vocabulary, and lemmatizes. An element is annotated automatically only if its best cosine similarity exceeds a confidence threshold of 0.62; otherwise it is queued for expert annotation. The taxonomic weight is α = 0.6, set on a development set as reported in Section 4.5 and analyzed for sensitivity in Section 4.6.

σ(e)=arg maxx∈Sig(0)sim(e,x)(6)

k=π(Ek)=〈topic, type, ρ^, Pre, Ev〉(7)

sim(x,y)=αsimtax(x,y)+(1-α)cos(vx,vy)(8)

simwp(ci,cj)=2d(lcs(ci,cj))d(ci)+d(cj)(9)

3.4.4 Aggregation

Aggregation π clusters annotated elements anchored to the same or adjacent concepts and turns each cluster into a KCU, as in Equation 7. The KCU records a topic, an attained level, prerequisite KCUs, and evidence links. The attained level is the provenance-weighted maximum of its elements' cognitive levels, capped by the level actually assessed in the rubric, so incidental exposure is not over-credited. By construction, a KCU links to outcomes and, through the property chain, to competencies.

3.4.5 Properties and complexity of τ

We state what τ does and does not guarantee. Given fixed parameters (α, the confidence threshold, the clustering radius) and a fixed set of expert resolutions for low-confidence cases, τ is deterministic: the same project artifacts yield the same KCUs, because each stage is a function and ties in annotation and clustering are broken by a fixed lexical order. τ is not claimed to be complete: it recovers only knowledge that the artifacts evidence, so sparse documentation yields sparse content. It is not claimed to be optimal or unique, since clustering depends on the radius and embeddings. It is partially automatic: extraction and annotation are assisted, but uncertain items are referred to experts, so τ is a human-in-the-loop procedure, not a closed function. Regarding complexity, let a project yield n candidate elements and let the ontology signature have size m. Decomposition is linear in artifact length. Extraction is linear in artifact length. Annotation is O(n·m) for exhaustive similarity, reduced to roughly O(n·log m) with an embedding index. Aggregation by single-link clustering is O(n²) in the worst case. The dominant cost is therefore annotation or clustering, both polynomial, and in the case study τ ran in well under a minute per project; Table 5 reports the measured per-stage breakdown. We make no claim of invariance to paraphrase beyond what the embedding model provides.

Table 5. Measured mean per-stage wall-clock runtime of the transformation τ on the case-study program (per project, single thread).

Times are machine wall-clock only and exclude expert review of the low-confidence queue, reported separately in Table 13.

3.5 Proficiency-aware metrics

Coverage of a competency is a capped sum of the alignment strengths of contributing units, weighted by a provenance weight that discounts weakly evidenced units, as in Equation 11. Saturation at one encodes diminishing returns: once a competency is well-covered, further exposure adds little, and without saturation a few heavily documented projects could dominate the index. The residual gap is the shortfall of attained below the required level, as in Equation 12. The program coverage index is the share of target competencies that are both covered above a threshold θ and free of residual gap, as in Equation 13. All four are computed by SPARQL aggregation over the inferred graph and recomputed when projects or rubrics change. Here, in Equations 10–13, a(k,c) is the alignment strength between unit k and competency c; β the relevance–depth weight; sim(k,c) the semantic similarity; ρ^(k) and ρ^*(c) the attained and required proficiency levels of unit k and competency c; ρ^(c) = max over k∈K of ρ^(k), the aggregate attained proficiency of competency c (the highest attained level among its contributing units); K the set of knowledge-content units contributing to c; w(k) the provenance weight of unit k; θ the coverage threshold; and Comp_*t* the set of target competencies with Nt = |Comp_*t*| their count.

a(k,c)∈[0,1],a(k,c)=βsim(k,c)+(1-β)ρ^(k)ρ*(c)(10)

Cov(c)=min(1,∑k∈Ka(k,c)w(k))(11)

G(c)=max(0, ρ*(c)-ρ^(c))(12)

CCI=1Nt∑c∈Compt1[Cov(c)≥θ ∧ G(c)=0](13)

The form of Equation 11 is not arbitrary. We require four properties of a coverage measure and choose the simplest function that satisfies them. Boundedness: coverage lies in the unit interval, so values are comparable across competencies. Monotonicity: adding evidence never lowers coverage. Diminishing returns: the marginal effect of further evidence shrinks as coverage rises, so a competency cannot be made to look ever better covered by piling on weakly related units. Proficiency sensitivity: a unit counts for more when it reaches the required level and for less when it falls short. The weighted sum ensures monotonicity; the cap at one provides boundedness; the weights produce diminishing returns; and the level ratio within alignment strength offers proficiency sensitivity. The proficiency ratio in Equation 10 is capped at one, min(ρ^(k)/ρ^*(c), 1), so that a(k,c) remains within [0,1] even when a unit's attained level exceeds the required level. In the language of aggregation-function theory, a bounded, monotone aggregation with a saturation cap is the standard choice when unbounded accumulation is undesirable (Beliakov et al., 2007). Other functions, a noisy-OR or a soft maximum, meet the same four properties; we use the capped sum because it is the easiest to explain to a curriculum committee. The weight β and the threshold θ are policy parameters rather than fitted constants: β says how much depth counts against breadth and θ says what counts as adequate, and both are decisions an institution should make and defend. For the case study, we set β = 0.5, an even split that weights relevance and depth equally as a neutral prior, and θ = 0.70; both were fixed on the 60-unit development set before the test set was scored and were not tuned on the reported results. The sensitivity analysis in Section 4.6 shows the results move smoothly with each.

3.6 Reasoning, rules, and consistency

The reasoning layer runs an OWL 2 DL reasoner to classify the ontology and check global consistency, ensuring that no class is unsatisfiable and that cardinality and functional constraints hold (Baader, 2011; W3C OWL Working Group, 2012). SWRL rules express regularities outside the DL fragment (Horrocks, 2004). All rules are written in DL-safe form, binding only to named individuals, so decidability is preserved (Motik et al., 2005). The rules propagate attained proficiency along prerequisite chains and flag ordering problems. The integrity rule of Equation 14 marks a prerequisite violation when a unit requires another scheduled no earlier than itself; the reasoner materializes violations as instances of a dedicated class. SPARQL queries then realize the metrics of Section 3.5 and retrieve accreditation evidence for any competency.

Viol(k1,k2):requires(k1,k2)∧t(k2)≥t(k1)(14)

3.7 Implementation

The ontology was authored in OWL 2 DL and handled in a desktop editor and through an ontology-oriented Python library that exposes classes and individuals as objects and runs a reasoner for classification (Lamy, 2017). Project artifacts passed through a parsing pipeline that segmented documents, recognized terms, and produced candidate elements; sentence embeddings came from the multilingual transformer named in Section 3.4.3 (Reimers and Gurevych, 2019). Classification and consistency used the HermiT hypertableau reasoner, and analytics were SPARQL queries over the materialized graph (Seaborne and Prud'hommeaux, 2008). The layers (foundation, transformation, content graph, reasoning, application) are separable, so the foundation can serve multiple programs. The implementation is a research prototype, not a production system.

To support reproducibility, the ontology was authored in the Protégé OWL 2 DL editor and manipulated programmatically with the Owlready2 library (Lamy, 2017). Classification and consistency were verified using the HermiT hypertableau reasoner in its default configuration. The proficiency-aware metrics in Section 3.5 were computed via SPARQL 1.1 queries over the materialized graph. Semantic annotation used the pretrained multilingual sentence transformer paraphrase-multilingual-mpnet-base-v2 (768-dimensional). All parameters were fixed on the 60-unit development set before the test set was scored: the taxonomic weight α = 0.6 (Equation 8), the auto-annotation confidence threshold 0.62, the relevance–depth weight β = 0.5 (Equation 10), and the coverage threshold θ = 0.70 (Equation 13). The ontology schema, the DL-safe SWRL rules, the SPARQL metric queries, and the transformation and evaluation scripts, together with a small synthetic instance and the exact library versions, are released as a reference implementation (see Data and Code Availability). The reported timings were measured on a single machine (Intel Core i7-13650HX CPU at 2.60 GHz, 32 GB RAM, and an NVIDIA GeForce RTX 4060 GPU, running 64-bit Windows 11).

3.8 Evaluation design

Mapping accuracy was measured against a gold standard built by three experts: two engineering faculty from the program and one specialist in curriculum design and ontologies. None of the three experts is a co-author of this article, and none contributed to building the ontology or the transformation; the gold standard is therefore independent of the system under evaluation. From the KCUs produced by τ, we drew a stratified random sample of 300 units, stratified by CDIO section and proficiency level. Sixty units formed a development set used only to set the parameters α, β, the confidence threshold, and θ; the remaining 240 formed a held-out test set used once for the reported figures. Each expert independently linked each test unit to its correct competencies. The 300 units were drawn from the KCU set produced by τ after the transformation had completed, so any expert resolution of low-confidence annotations during the annotation stage was separate from, and before, construction of the gold standard; each unit could receive one or more competency labels, and the experts linked units without reference to the system's proposed mappings, so the standard was built blind to the system output. Pairwise inter-rater agreement before any reconciliation was κ = 0.74 on average, which is substantial (Cohen, 1960); the experts then reconciled disagreements to a single adjudicated standard by discussion to consensus rather than by simple majority. For the prerequisite task the same experts additionally conducted a dedicated audit of the curriculum's ordering constraints—separate from the program's routine manual review—and compiled a reference set of 26 genuine ordering issues, against which prerequisite reasoning is scored by precision and recall (reported in Section 4.4). The automated mapping was scored against this standard with precision, recall, and F1, Equation 15; 95% confidence intervals were obtained via bootstrap resampling of the test units with 1,000 replicates. Agreement between the system and the adjudicated standard was measured by Cohen's κ, (Equation 16) (Cohen, 1960). Ontology quality was measured by the inheritance and relationship richness of Equation 17, following metric-based analysis (Tartir et al., 2005). Because the evaluation uses one program, we report descriptive statistics with intervals and make no inferential claim about other programs; the design supports internal comparison (system vs. experts) but not generalization, a point returned to in Section 5.

P=TPTP+FP, R=TPTP+FN, F1=2PRP+R(15)

k=po-pe1-pe(16)

IR=1|C|∑cϵC|subclasses(c)|, RR=|R||R|+|HC|(17)

3.9 Comparative baselines

To compare the proposed methodology with existing approaches, five baseline methods were evaluated on the same 240-unit test set using the same adjudicated gold standard. Performance was measured by precision, recall, and F1 score. The baselines included: (1) manual document-based mapping, (2) Term Frequency Inverse Document Frequency (TF–IDF) with cosine similarity, (3) ontology learning with NLP-based term extraction and entity linking (Conde et al., 2016), (4) knowledge-graph embedding using TransE and Node2Vec models (Bordes et al., 2013; Grover and Leskovec, 2016), and (5) a 5-shot GPT-4o baseline whose five demonstrations were drawn from the 60-unit development set by embedding similarity to each test unit, with the number of shots fixed on the development set and not tuned on the test data. All trainable baselines were tuned using the same 60-unit development set to ensure a fair comparison. These five families were chosen to span the principal approaches to competency–content mapping—a lexical baseline (TF–IDF), an ontology-learning pipeline, knowledge-graph embeddings, a large-language-model prompt, and the manual practice the method aims to replace—so the comparison covers the alternatives a program might realistically adopt.

3.10 Computational complexity, memory, and scalability

Section 3.4.5 stated the cost of τ informally. Proposition 1 makes it precise.

Proposition 1 (Cost of τ). Let a project p have artifacts of total length L_*p*, yield n_p candidate knowledge elements, and be annotated against an ontology signature of size m using an embedding index that answers an approximate nearest-neighbor query in O(log m). Then τ runs in the time and space of Equation 18, where the quadratic term is the worst case of single-link agglomerative clustering in aggregation.

Tτ(p)=O(Lp+nplogm+np2),Sτ(p)=O(np+m)(18)

**Proof.**Decomposition and extraction pass over the project artifacts once each and emit at most n_*p* knowledge elements, so both run in O(L_*p*) time. Annotation issues one approximate-nearest-neighbor query per element against the indexed ontology signature, in O(n_*p* log m); without the index, an exhaustive scan would cost O(n_*p* m). Aggregation groups the n_*p* elements by single-link clustering, which the SLINK algorithm computes in O(np2) time and O(n_*p*) space without enumerating all pairwise edges (Sibson, 1973). Summing the stages gives the time bound of Equation 18, and the resident store holds the n_*p* elements together with the signature index of size O(m), which gives the space bound.

Across a program of P projects, write N = ∑p np for the total number of elements. The graph is then built in O(Ltotal + N log m + ∑p np2). The clustering term is a sum of per-project squares rather than (∑p np)2, because aggregation runs independently within each project; if project size is bounded by a constant n_max, then ∑p np2 ≤ nmax · N = O(N). Aggregation therefore grows linearly in the number of projects once project size is bounded, which is the regime curricula occupy.

Memory is dominated by the materialized graph. As Equation 19 shows, the triple count is linear in the total number of elements, with a fixed terminological offset g₀ and a per-element constant k; in the case study, these gave 9,540 triples for one program. Five comparable programs would hold on the order of 48,000 triples and twenty on the order of 190,000, which are small sizes for current triple stores.

Reasoning cost is separate from transformation cost. Classification and consistency checking in the SROIQ logic underlying OWL 2 DL are worst-case N2ExpTime-complete (Kazakov, 2008), but that bound is rarely approached on the taxonomic, lightly cyclic ontology used here, and practical reasoners do far better in normal cases (Dentler et al., 2011; Glimm et al., 2014). We tested these bounds empirically rather than leaving them as projections: Section 4.10 reports a measured scalability benchmark in which the analytics workload grows close to linearly with graph size and terminological reasoning stays effectively flat, and Table 5 reports the measured per-stage wall-clock runtime of the transformation τ on the case-study program. The benchmark does not cover its extraction and annotation on real artifacts at scale, and it is not a substitute for multi-program field deployment, which Section 5.7 sets out.

4 Results

4.1 Case-study setting

The program is an undergraduate degree in mechatronics and robotics of 240 ECTS over eight semesters that has adopted the CDIO Syllabus. The analysis covered 42 courses and the 18 educational projects within them, from a first-year design–build project to a capstone autonomous-robot project. The program declared 156 leaf competencies across the four sections, each with an expected level on the CDIO 1–5 scale, and 311 project artifacts were available. Table 6 gives the profile.

Table 6. Profile of the case-study program.

4.2 The constructed ontology

The populated ontology had 38 classes, 21 object properties, and 14 data properties in 642 logical axioms and 17 DL-safe SWRL rules. The assertional part had 1,287 individuals in 9,540 Resource Description Framework (RDF) triples. Classification and consistency checking with the HermiT hypertableau reasoner finished in 2.8 s, so the model stayed efficient at the scale of one program (Table 7). This 2.8 s covers classification together with ABox consistency over the fully populated ontology (schema plus 1,287 individuals); the schema-only terminological time, which is independent of instance count, is the ≈ 1.9 s reported in Section 4.10. This addresses the decidability and efficiency part of RQ1; we did not test much larger instances, and we discuss scalability as a limitation.

Table 7. Population and reasoning statistics of the constructed ontology.

4.3 Transformation output and coverage

The procedure converted the 18 projects into 1,043 KCUs, about 58 per project, each with provenance, an attained level, prerequisites, and evidence. Of the candidate elements, 14% fell below the confidence threshold and experts resolved them rather than annotating them automatically. Through the property chain, the reasoner inferred 4,210 contributesTo links. Coverage of every competency was computed by SPARQL aggregation. A baseline used the program's existing document-based competency–course mapping, counting a competency as covered when a course nominally claiming it appears. Figure 3 contrasts the two by section, and Table 8 gives the figures and residual gaps.

Table 8. Competency coverage and residual gaps by CDIO Syllabus section (baseline vs. proposed).

The gains are uneven, and the pattern is informative. Disciplinary competencies were already well covered by the baseline, because they map onto named courses, so the procedure added little there. The larger gains were in personal, interpersonal, and CDIO-process competencies, which the baseline under-represents because they are developed across projects rather than owned by one course. Binding content extracted from project artifacts to these competencies made visible development that the course-level mapping missed, raising overall coverage from 0.59 to 0.84 and bringing 131 of 156 competencies to zero residual gap. Twenty-five competencies retained a gap, mostly leadership and some advanced design competencies that the program evidences weakly; the method flags these rather than hiding them.

4.4 Consistency and prerequisites

Reasoning confirmed global consistency: no class was unsatisfiable and all cardinality and functional constraints held. The integrity rule materialized 23 ordering violations in which a unit depended on a unit produced no earlier in the curriculum. Manual inspection confirmed all 23 as genuine; several were projects assuming analytical or experimental knowledge scheduled only later. The program's prior manual review had not found these, which supports the practical value of automated prerequisite reasoning and the consistency part of RQ3. These 23 are the violations surfaced by the integrity rule. Scored against the experts' dedicated 26-issue audit set (defined in Section 3.8; per-method comparison in Section 4.8), the DL-safe rules achieve a precision of 1.00 (all 23 surfaced are genuine; the integrity rule is deliberately conservative, materializing only violations entailed by the prerequisite closure, so it yields no false positives) and a recall of 23/26 = 0.88 on the prerequisite task, the three missed issues falling outside the DL-safe pattern.

4.5 Mapping accuracy and parameter selection

On the 240-unit test set, the automated mapping scored precision 0.91 (95% CI 0.87–0.94), recall 0.86 (0.81–0.90), and F1 0.88 (0.84–0.91) against the adjudicated standard, with system–standard agreement κ = 0.79. Recall is the weaker figure by design: the procedure refers low-confidence annotations to experts rather than forcing a link, which protects precision at recall's expense, a trade-off we judge appropriate for accreditation use. Parameters were fixed on the 60-unit development set before the test set was touched: α = 0.6, β = 0.5, confidence threshold 0.62, and θ = 0.70. Table 9 reports these results with the manual baseline's efficiency and ontology quality.

Table 9. Mapping accuracy, ontology quality, and efficiency: manual baseline vs. CDIO–CKO.

The expert pre-adjudication inter-rater agreement (κ = 0.74) characterizes the gold-standard construction process and is not attributable to either method; it is defined in Section 3.8. Table 11 reports precision, recall, and F1 for the manual baseline against the same adjudicated standard.

A relationship richness of 0.41 indicates that a large share of relations are non-hierarchical rather than forming a pure taxonomy, and an inheritance richness of 2.7 reflects a moderately deep, branched competency tree. These values fall within the range reported for domain ontologies of comparable scope in the metric-based analysis of Tartir et al. (2005), indicating a schema that is neither a flat taxonomy nor an over-connected graph. Tracing the evidence for one competency took experts about 18 min of document search and was answered by a single SPARQL query in under a minute and a half.

4.6 Sensitivity analysis

Because the metrics are heuristic, we examined how results move with the parameters (see Figure 4). Panel (a) shows F1 against the taxonomic weight α on the development set: F1 is flat near the top, within about 0.02 for α between 0.5 and 0.7, and falls off only toward the extremes where one similarity component is ignored, so the choice of α = 0.6 is not fragile. Panel (b) shows the program coverage index against the threshold θ for two values of β. The index decreases smoothly and monotonically as θ rises, with no discontinuity, and the two β curves stay within about 0.05 of each other. Hence, the index is stable rather than erratic under these parameters. The saturation in Equation 11 is what keeps the index bounded and prevents heavily documented projects from dominating; removing it widened the spread across β by roughly a factor of two on the development set. We did not perform a full factorial sweep, and the analysis is on one program, so these observations bound robustness only for this case.

4.7 Annotation error analysis

We inspected the 34 test units the system mislabeled relative to the standard and grouped the errors (Table 10). The largest source was conflation of near-synonymous competencies in Sections 2 and 3, where the Syllabus vocabulary is close; the next was incorrect proficiency level from rubric verbs that overstate depth; the remainder were missed multi-word terms and a few cases where sparse artifacts gave too little text to anchor. Most errors were therefore precision-preserving omissions or level errors rather than wrong competency links, consistent with the conservative threshold.

Table 10. Categories of annotation error on the test set (34 mislabeled units).

The near-synonym category deserves separate comment, since it accounts for 13 of the 34 errors and is the only group that produces a wrong link rather than an omission. It arises where Sections 2 and 3 of the Syllabus describe the same observable behavior from different standpoints. Attitudes, thought and learning (2.4) and teamwork (3.1) both surface in reflective language about working with others; system thinking (2.3) and communication (3.2) share vocabulary wherever a student explains a design decision to an audience. The evidence sentence is then nearly identical for two competencies that sit in different branches, and the embedding similarity that drives semantic annotation is by construction weak at exactly this discrimination, because it is trained to place near-synonymous phrasings close together.

Three mitigations follow from that diagnosis. The first is to give the annotator taxonomic context it currently ignores: the depth of each candidate node in the Syllabus hierarchy and the identity of its parent branch are known for every candidate, and conditioning the confidence score on agreement at the branch level, not only on leaf similarity, would penalize matches that cross from Section 2 to Section 3 on lexical evidence alone. The second is to encode the distinction in the ontology rather than leave it to the scorer, by asserting disjointness between the intrapersonal and interpersonal branches for the annotation of a single knowledge element, so that a unit linked to both raises a consistency violation and is routed to the expert queue instead of resolved arbitrarily in favor of the higher score. The third is to discriminate on provenance where lexical evidence cannot: DL-safe rules keyed on the artifact type an element was extracted from—an individual reflection, a team log, a presentation rubric—separate the branches using information the pipeline already records but does not currently exploit. None of the three is implemented in the present version, and each would trade recall for precision to a degree that has to be measured rather than assumed; they are the first modifications we would evaluate, and the queue-based design means their effect would be visible as a change in the queue fraction as well as in F1.

4.8 Comparison with alternative methods

Figure 5 and Table 11 report the comparison on the competency-linking task. On F1 the proposed method led at 0.88, narrowly ahead of LLM few-shot extraction at 0.86. The LLM reached the highest recall (0.90), but lower precision (0.82), consistent with a tendency to over-assign competencies, and its outputs carried no checkable justification. Knowledge-graph embedding reached 0.79, strong for link prediction but blind to logical consistency and unable to represent proficiency. Ontology learning and the lexical baseline trailed at 0.73 and 0.70. The manual mapping was precise where it applied but missed cross-project competencies, giving the lowest F1 at 0.65. On the prerequisite task, the proposed DL-safe rules recovered 23 of the 26 expert-identified ordering issues with full precision, against 15 for the LLM, 11 for curriculum mining (with several false positives), and 8 for embedding.

Table 11. Competency-linking accuracy by method family (240-unit test set, same gold standard).

The accuracy comparison tells only part of the story. Table 12 contrasts the capabilities each family provides. Only the proposed method offers full description-logic consistency checking, propagation of an attained proficiency level, and an auditable trail from competency to artifact; ontology learning offers formal semantics but not, on its own, the reasoning and proficiency model. Embedding and LLM methods are competitive or superior on isolated prediction. Yet, neither yields the verifiable, accreditation-ready structure that motivates this study, and the LLM in particular trades precision and auditability for recall. The proposed method is also not dominant everywhere: it depends on documented artifacts, where the LLM can fill gaps by inference. The contribution is therefore not a wide F1 margin, which does not exist, but comparable accuracy together with capabilities the high-accuracy alternatives lack.

Table 12. Capability comparison across method families (✓ supported, ~ partial, × not supported).

4.9 Human effort and annotation burden

A method that only moves work around saves nothing, so we account for the human effort it needs and separate one-time from recurring cost (Table 13). The one-time cost is authoring the CDIO–CKO foundation: formalizing the Syllabus, classes, properties, and DL-safe rules took about twelve person-days for the ontology team, paid once and reused across programs. The recurring per-program cost has two parts. Preparing the project artifacts and rubrics for ingestion took about one person-day for the 18 projects. Expert annotation was confined to the low-confidence queue: of roughly 4,800 candidate elements, 14% (about 670) fell below the confidence threshold and were reviewed by one expert at a median of 40 s each, totaling about 7.5 person-hours, with a second expert spot-checking a 20% sample for a further 1.5 hours. The full automatic mapping plus this review replaced what the program estimated as several person-weeks of manual competency tracing. These figures are from one program and depend on artifact quality: a poorly documented program would push more elements into the queue and raise the burden, which is why we report the queue fraction rather than only the totals.

Because the low-confidence queue is the part of the procedure another team would have to reproduce, it is worth being precise about what the expert does in it. The reviewer does not annotate from scratch. Each queued item arrives with the extracted term, the source sentence it came from, and the ranked candidate competencies with their confidence scores, and the decision is to accept the top candidate, select a lower-ranked one, or reject the element as spurious. That framing is what makes a median of 40 s possible where free annotation takes several minutes, and it also keeps the step auditable: every adjudication is stored with the adjudicator, the timestamp, and the rank of the option chosen, so the manual layer can be replayed and re-examined rather than being tacit. Reliance on a single adjudicator per program is nevertheless the weakest point for reproducibility, since the resulting standard reflects one reading of the Syllabus; this is why the 20% second-expert sample is reported here, and why the external-validation protocol in Section 5.7 requires at least three annotators with a pre-registered adjudication procedure.

Table 13. Human effort: one-time and recurring (per-program) costs.

For scalability, the question is whether the 14% share is a fixed overhead or a declining one. Two of its three components should decline with reuse. Elements queued because a multi-word technical term is absent from the ontology lexicon enter that lexicon once adjudicated, so the same term is resolved automatically thereafter. Adjudicated decisions accumulate as labeled examples that raise confidence for recurring phrasings across programs in the same domain. The third component should not decline, because it originates in sparse or inconsistent documentation—a property of the program's artifacts rather than the method—a poorly documented program will push more elements below the threshold no matter how much prior adjudication exists. The defensible reading is therefore that per-program expert effort falls with reuse but does not approach zero, and that the residual is bounded by artifact quality rather than by the algorithm. We cannot yet attach a number to that decline from a single program; the multi-program study specified in Section 5.7 should track the queue fraction per program as its own outcome, since it determines whether the method remains affordable at institutional scale.

4.10 Measured scalability

Proposition 1 predicts that the analytics workload grows linearly in the number of projects. We tested this on synthetically scaled graphs shaped like the case-study ontology, from 1 to 32 program-equivalents (about 4,500 to 144,000 triples), populated with synthetic individuals on the real schema. Figure 6 shows the result. Property-chain materialization, coverage aggregation, and prerequisite transitive closure together rose from 0.20 s to 2.86 s as the triple count grew thirty-two-fold, tracking the linear reference closely, and peak memory rose from 48 to 309 MB. Terminological classification and consistency with the HermiT reasoner, which depend on the schema and not the instance count, took about 1.9 s and were flat across sizes. The measurements match the linear-in-projects prediction over this range and stay well inside the capacity of a commodity machine; they bound the reasoning and analytics cost but not the extraction and annotation cost, which depends on artifact volume. They also do not stand in for a real multi-program deployment. Every graph beyond the first is a synthetic replication of one program's schema, so the benchmark bounds how computation grows with instance count but says nothing about how genuinely different programs would populate the ontology or about accuracy at that scale. We therefore claim measured computational scalability over this range, not validated scalability across real programs.

To support the informal claim that τ runs in well under a minute per project, Table 5 reports the mean wall-clock time for each transformation stage, measured on the case-study program over its 18 projects on a single-thread commodity workstation. The breakdown follows Proposition 1: the decomposition and extraction pass over artifacts once and contribute little, whereas annotation and single-link aggregation account for the bulk of the cost, yet the per-project total stays below 1 min. Reasoning and analytics costs are separate and are reported in Figure 6; the two should not be conflated, since the transformation runs once per project while the analytics are re-run on demand.

4.11 Ablation study

To see what each component contributes, we removed them one at a time and re-measured on the same test set (Table 14). The two similarity sources are complementary: annotating on taxonomy alone (the α = 1 end of Figure 4a) gave linking F1 0.79, embeddings alone (α = 0) gave 0.71, and the hybrid beat both at 0.88. Stripping the ontology of its structure, so competencies become a flat list of labels with no hierarchy or axioms, barely changed raw matching but cut traceable coverage to 0.68, because coverage no longer propagates up the competency tree, and it found no prerequisite violations. Removing aggregation, so links are made at the level of individual elements rather than clustered units, fragmented the evidence trail and dropped F1 to 0.80 through duplication. Forcing every annotation through instead of referring low-confidence ones to experts raised recall but cut precision, for a net F1 of 0.82; the trade-off the confidence threshold is there to manage. The single largest effect was removing reasoning: with no property-chain inference, the only competency links are those asserted directly, so traceable coverage fell from 0.84 to 0.61, and with no consistency and rule layer, none of the 23 prerequisite violations surfaced. Reasoning and ontology structure together add little to raw annotation accuracy but provide almost all coverage propagation and all consistency checking, which the comparison methods in Section 4.8 cannot reproduce.

Table 14. Ablation: effect of removing each component (similarity rows correspond to the endpoints of Figure 4a).

5 Discussion

The results provide evidence on all three questions, for one program. The Syllabus was formalized as a decidable OWL 2 DL ontology that classified in seconds (RQ1). The conversion of projects into knowledge content was specified as a repeatable, parameterized procedure with stated properties and linked outputs (RQ2). In the case program, the representation supported coverage, gap, and prerequisite analysis, and the mapping matched expert judgment with F1 0.88 and κ 0.79 (RQ3). These findings suggest the approach is feasible; they do not establish that it generalizes.

5.1 Interpretation

The most useful finding is where coverage improved. Course-centric mappings represent disciplinary knowledge well but obscure the personal, interpersonal, and process competencies that are CDIO's distinctive aim, because those competencies are spread across projects. Making the project the unit of transformation and the KCU the carrier of development recovers this part of the curriculum and makes it traceable. In effect, constructive alignment stops being something a committee asserts and becomes something a query can check, and proficiency stops being a label and becomes data the reasoner uses. This extends the competency-ontology literature from static catalogs toward an account of how projects generate the knowledge competencies require, though only one program supports the claim.

5.2 Practical implications

For program designers, the case study suggests the method can turn curriculum review from a periodic audit into a query that re-runs when a project or rubric changes, making the effect of a proposed revision on coverage immediately visible. For accreditation, retrieving the projects, outcomes, units, and evidence behind any competency gives a reproducible evidence trail. For credit transfer, expressing competencies and content in a shared vocabulary could let program comparison be approached as ontology matching, although this was not tested here. These are potential benefits demonstrated on a prototype; none has been tested in a live accreditation cycle.

5.3 Threats to validity

Construct validity. The metrics operationalize coverage and gap as heuristics with chosen parameters; if the parameters or the saturation form misrepresent what stakeholders mean by adequate coverage, the construct is off. We mitigated this with a sensitivity analysis and by separating development and test sets, but the constructs remain our definitions. Fidelity of the formalized Syllabus to the institution's reading of CDIO is also a construct risk.

Internal validity. Three domain experts established the gold standard (pre-adjudication κ = 0.74), and some degree of subjectivity is therefore unavoidable. Parameters were fixed on a separate development set, while the held-out test set was used only for the final evaluation. Because the transformation and the evaluation metrics were designed by the authors, some designer bias cannot be excluded; the three gold-standard experts, however, are not co-authors and did not contribute to the ontology or the transformation (Section 3.8), so the reported accuracy is not inflated by evaluators scoring their own system. Independent validation on additional programs by external evaluators remains part of future studies.

External validity. The study is one program in one engineering domain in one working language. Programs with non-classical structures, such as some postgraduate medical curricula, may not fit the course-and-period model, as prior work notes. Cross-domain, multilingual, and multi-program transfer are untested.

Conclusion validity. We report descriptive statistics with bootstrap intervals on a single program and make no inferential claim across programs. Because assessment units are clustered within the eighteen projects, the unit-level bootstrap treats them as independent. A clustered (hierarchical) bootstrap would be more defensible, so the reported intervals should be interpreted with this non-independence in mind. The efficiency comparison (18 min vs. 90 s) is indicative, measured on a small number of tracing tasks, not a controlled time study. Table 15 summarizes these threats.

Table 15. Systematic summary of threats to validity, mitigations, and residual risk.

5.4 Novelty relative to prior work

Earlier curriculum ontologies bind competencies to courses and reason about prerequisites among competencies (Milosz et al., 2024; Paquette, 2007; Rezgui et al., 2014), and recent CDIO ontologies map competencies to content statically. The difference here is the explicit transformation from projects to content, the propagation of attained proficiency through it, and reasoner-computed proficiency-aware metrics. To our knowledge, this is the first attempt to treat the educational project as a transformable source of competency-linked knowledge content within CDIO. We present it as one workable design, evaluated once, not as a proven best approach.

5.5 Limitations

It is worth being exact about what scales, because three different claims tend to get conflated. Computational scalability has direct evidence: the reasoning and analytics workload was measured on synthetic graphs up to 144,000 triples (Section 4.10) and grew close to linearly. Program-count generalization rests on a single real program, so we cannot yet say the accuracy or coverage figures hold elsewhere. Deployment scalability has no evidence: no real users have run the prototype in production, there is no integration with a student information system, and there is no experience of concurrent editing or long-term maintenance. Only the first of the three is presently supported by measurement, and the article should be read with that distinction in mind.

Three further limitations bound the work beyond the validity threats. The output quality depends on the artifacts: sparse or inconsistent documentation propagates into the content layer, and 14% of elements already needed expert resolution. The DL-safe restriction on rules keeps reasoning decidable but means rules bind only to named individuals, so some general inferences cannot be expressed as rules and must be encoded in the TBox or handled in SPARQL. Scalability is shown only at one program's scale; ABox growth and the O(n²) worst case of clustering would need indexing and incremental reasoning for an institution-wide deployment, which we have not built.

A fourth limitation differs from the three above and should be stated plainly rather than inferred from the evaluation design. This study validates a technical artifact, not an educational outcome. The evaluation establishes that the competency mapping is accurate against an adjudicated standard, that the ontology reasons consistently, and that competency tracing becomes faster and auditable. It establishes nothing about whether students in the program learn more, whether graduates are better prepared, or whether a review panel reaches a different judgment because the evidence was assembled this way. Nor has the method been exercised inside a live curriculum-redesign or accreditation cycle: the 23 prerequisite inconsistencies it surfaced are defects in the declared structure of the curriculum, and repairing them is plausibly beneficial, but no cohort has yet studied under a repaired sequence, and we therefore claim no learning effect. The efficiency figure reported in Section 4.5 is a proxy for effort, not for attainment, and should not be read as one. Section 5.8 specifies the three empirical studies—a controlled accreditation-preparation trial, a curriculum-redesign field study in which a comparable subset of flagged gaps is deliberately center unchanged, and a usability and adoption study with the faculty who would maintain the model—needed to convert the educational case from plausible to demonstrated.

5.6 Future work

The extraction and annotation stages could use larger language models under human validation, with the ontology constraining generation and experts adjudicating low-confidence items (Pan et al., 2024). The method should be replicated across programs, domains, and languages to test transfer and to enable cross-program matching. The proficiency model could be linked to learner-level assessment data so the same ontology supports individual progression, not only program design. The comparison in Section 4.8 uses one program's data; a controlled, multi-program study that re-runs every baseline on independently collected data would confirm whether the accuracy ordering and the capability advantages hold beyond this case.

5.7 Toward external validation

The evaluation here is a single-program proof-of-concept, which journals in this area will reasonably regard as insufficient for general claims. We therefore specify the external-validation study the method needs so that it can be carried out and reported on its own terms. The study should cover at least three programs spanning at least two engineering domains, at least two institutions, and at least two working languages, to separate the method's behavior from the quirks of a single curriculum. The competency gold standard for each program should follow a pre-registered protocol with at least three annotators and a target inter-rater κ of 0.7 before adjudication. The parameters α, β, the confidence threshold, and θ should be fixed on one held-out development program and applied unchanged to the others, so the study tests transfer rather than per-program tuning. Accuracy should be reported per program with confidence intervals and pooled with a random-effects model, so between-program heterogeneity is visible rather than hidden. Each baseline in Section 3.9 should be re-run under identical conditions, and reasoning time and memory should be benchmarked as program count grows, replacing the analytical projections of Section 3.10 with measurements. Only such a study can show whether the accuracy ordering and the capability advantages observed here hold beyond the single case.

5.8 Educational significance and how to validate it

This study validates a technical artifact, not a learning gain. That distinction matters because what an education journal cares about is whether students end up more competent and whether programs improve, not whether an ontology classifies in 2 s. We can say what the method changes in the machinery of a program: competency development that was invisible becomes traceable, the evidence behind a competency can be retrieved in seconds rather than found by hand, and gaps and ordering faults are flagged before a review rather than after. The tracing-time figure (about 18 min down to 90 s) is an efficiency proxy, and a real one, but it is not a measure of learning. None of this shows that students learn more or that graduates are better prepared.

Three studies would supply the pedagogical evidence. The first is a controlled accreditation-preparation study: time and error rates for assembling competency evidence, with and without the method, across several program reviews, which would turn the efficiency proxy into a measured claim. The second is a curriculum-redesign field study: take the gaps the method flags, act on a subset, leave a comparable subset unchanged, and measure over the next cohorts whether attainment on the affected competencies improves relative to the controls, using the program's own assessment data rather than the ontology's coverage score. The third is a usability and adoption study with the faculty who must maintain the model, since a method that only a semantic-web specialist can run will not survive contact with a real department. Until at least the first two are done, the contribution should be read as an instrument for competency-based curriculum engineering whose educational payoff is plausible and motivated but not yet demonstrated.

6 Conclusion

This study addressed the gap between the natural-language competency declarations of CDIO and the unstructured knowledge that students develop through educational projects. It contributed four elements: the CDIO–CKO ontology, a decidable OWL 2 DL formalization of the Syllabus connected to programs, projects, courses, outcomes, and knowledge content; a parameterized transformation that converts projects into competency-linked knowledge-content units with stated properties and complexity; proficiency-aware coverage and gap metrics with a sensitivity analysis; and a six-layer architecture. On one mechatronics and robotics program, the methodology produced a consistent graph of 1,287 individuals and reproduced expert competency mapping with an F1 of 0.88 and system–standard agreement of κ = 0.79. The two κ figures use different references and are not directly comparable: the pairwise inter-annotator agreement before adjudication was κ = 0.74, whereas κ = 0.79 measures the system against the single reconciled standard, so agreement against that cleaner reference can exceed raw pairwise agreement without contradiction. It raised traceable coverage from 0.59 to 0.84, recovering links that course-level mappings miss, and surfaced 23 prerequisite inconsistencies that manual review had overlooked. This evidence comes from a single program and is best read as a proof of concept. It suggests that converting educational projects into formal knowledge content is a feasible route to competency-traceable curriculum engineering, and it identifies the cross-domain, multilingual, and larger-scale studies needed to test the idea properly.

Generative AI statement

The author(s) declared that Generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Ibrahim Arpaci, Bursa Uludag Universitesi, Türkiye

Reviewed by: Phoebe Koundouri, Athens University of Economics and Business, Greece

Guilherme Rockembach, Federal University of Pampa, Brazil