Domain adaptation of large language models for additive manufacturing using retrieval-augmented generation and fine-tuning

General-purpose large language models (LLMs) often struggle to generate reliable responses in specialized engineering domains due to limited domain grounding and insufficient exposure to structured technical knowledge. This study investigates practical strategies for adapting a foundation LLM to the additive manufacturing (AM) domain in order to improve answer accuracy, relevance and usability for expert-level question answering. AM knowledge is distributed across heterogeneous sources such as academic literature, manufacturer documentation, technical standards and procedural guides. Although general LLMs demonstrate strong linguistic capabilities, they frequently fail to retrieve and contextualize such domain-specific information. Two common approaches to address this limitation are domain-specific fine-tuning and retrieval-augmented generation (RAG). A curated AM corpus is constructed and used to evaluate three configurations based on LLaMA-3-8B: (1) a pretrained baseline model, (2) a RAG system that retrieves relevant document chunks from a vector database and (3) a model fine-tuned on raw domain text using quantized low-rank adaptation (QLoRA). Performance is evaluated using 200 expert-designed AM questions assessed by mechanical engineering experts for accuracy, relevance and overall preference. The RAG model consistently outperforms the baseline. Among the 200 questions, 75.5% of RAG responses are judged more accurate, 85.2% are preferred overall and 90.8% are rated more relevant than baseline responses. In contrast, fine-tuning on raw AM text reduces performance, producing more accurate answers in only 5.6% of cases and more relevant answers in 32.5% of cases. These results indicate that retrieval-augmented approaches provide a more effective pathway for adapting LLMs to specialized engineering domains than naive fine-tuning on unstructured technical data. RAG grounds responses in retrieved source documents at inference time, enabling higher accuracy and relevance without requiring model retraining.

Large language models such as OpenAI’s GPT-3 and GPT-4, Google’s PaLM, and Meta’s LLaMA have demonstrated remarkable capabilities in natural language understanding and generation (Touvron et al., 2023). These models, trained on massive internet-scale datasets, perform well on general-purpose benchmarks such as text generation, summarization and open-domain question answering. Scaling these models further has been shown to significantly improve their performance across such tasks. GPT-3, for example, achieved strong results in translation, open-domain question answering and even basic arithmetic-all without task-specific tuning (Brown et al., 2020). However, despite their broad knowledge, general-purpose LLMs often struggle in highly specialized domains. When applied to fields such as AM, medical science or scientific research, these standard LLMs frequently fail to deliver reliable and accurate results.

Several studies highlight this performance gap. For example, GPT-3 was evaluated across 57 subjects in the massive multitask language understanding (MMLU) benchmark and found to underperform on technical topics, sometimes barely exceeding random guessing on niche domains (Hendrycks et al., 2020). In the field of AM, Chandrasekhar et al. (2024) observed that GPT-4 provided only high-level responses, lacking detailed process understanding, while a retrieval-augmented domain-specific model (AMGPT) offered significantly more accurate answers. This was further confirmed by Halsey et al. (2025), who found that GPT-4 could not provide manufacturing-specific details without augmentation, whereas RAG based model significantly improved answer specificity and factuality. Eslaminia et al. (2025) also emphasized that fused deposition modeling (FDM) tasks require specialized interdisciplinary knowledge that general-purpose LLMs fail to capture; they introduced the FDM-Bench to benchmark LLMs on realistic AM scenarios and found limited generalizability without domain adaptation. Similarly, Pak and Barati Farimani (2025) demonstrated that fine-tuning general LLMs on AM defect data (AdditiveLLM) led to substantial performance gains, suggesting that pretrained models alone cannot handle defect classification tasks reliably in AM settings. In clinical applications, DocOA-a specialized assistant built on the GPT-4 architecture demonstrated superior performance in osteoarthritis management compared to general-purpose LLMs, which often failed to incorporate context-specific medical guidelines (Chen et al., 2024). In biomedical natural language processing (NLP), Chen et al. (2025) showed that zero-shot GPT and LLaMA models consistently underperformed compared to domain-trained models like BioBERT across most benchmarks, particularly in information extraction. These findings collectively show that without domain adaptation, general LLMs often fail to meet expert-level performance standards in specialized tasks.

To bridge this gap, researchers have explored two primary strategies for domain adaptation: fine-tuning and RAG (Zhang et al., 2023; Wan et al., 2025; Du et al., 2025; Buehler, 2024; Park et al., 2024; Xiong et al., 2024). Fine-tuning involves updating a model’s parameters using domain-specific information to embed relevant knowledge into the model itself. While this can enhance performance, it is computationally expensive and prone to overfitting, especially when the training data is noisy or limited (Macháček et al., 2025). Methods like low-rank adaptation (LoRA) and other parameter efficient fine-tuning (PEFT) techniques help to reduce adaptation costs yet still depend on the model internalizing domain knowledge to address gaps observed in zero-shot scenarios (Yang et al., 2024). Furthermore, fine-tuned models are static, they cannot easily incorporate new information without retraining and often lack transparency regarding the sources of their claims.

RAG offers a more flexible and scalable alternative. Instead of storing all domain knowledge in model weights, RAG enables the LLM to retrieve relevant documents from a curated knowledge base during inference (Lewis et al., 2020). This retrieved content is then provided as input context, grounding the model’s responses in actual reference material. Studies have shown that this approach improves factual accuracy, reduces hallucinations and allows models to stay current without needing retraining (Borgeaud et al., 2022; Izacard and Grave, 2021). For instance, DeepMind’s RETRO model achieved performance on par with GPT-3 using 25 times fewer parameters by leveraging a large-scale retrieval mechanism (Borgeaud et al., 2022). In domains where correctness, traceability, and up-to-date knowledge are critical such as engineering and medical science, RAG provides a solid foundation for developing expert-level AI systems.

In this study, we investigate how different adaptation strategies influence the performance of a general-purpose LLM in AM question answering (QA). To support this analysis, we first construct a curated AM dataset tailored for both model development and evaluation. Using this dataset, we compare three approaches-a pretrained baseline, a fine-tuned model and a RAG system to understand how each method handles domain-specific engineering queries. Through this comparative study, we examine the effectiveness of retrieval-based knowledge integration and the impact of fine-tuning on raw technical data. The results reveal that not all adaptation strategies are equal, retrieval-based approaches and fine-tuning produce meaningfully different outcomes in domain-specific question answering. The novelty of this work is its unified evaluation of retrieval-based and fine-tuning-based LLM adaptation using the same curated AM knowledge corpus and expert-designed benchmark. Instead of evaluating RAG or fine-tuning separately, this study directly compares how these two adaptation strategies affect AM QA performance under the same experimental conditions. This provides practical insight into which adaptation strategy is more suitable for building reliable LLM-based knowledge assistants for AM.

This section outlines the complete methodological framework used to adapt and evaluate a general-purpose LLM in the AM domain. The methodology integrates domain-specific dataset construction, parameter-efficient model adaptation, retrieval-based knowledge augmentation and expert-driven evaluation.

The overall workflow includes four major stages: domain knowledge collection, benchmark dataset construction, model adaptation strategies and expert-based evaluation. The high-level architecture of the proposed framework is illustrated in Figure 1.

As shown in Figure 1, domain-specific AM documents are first collected to build the AM corpus. The corpus is then used for both fine-tuning the LLM through quantized low-rank adaptation (QLoRA) and constructing a vector database for RAG. The outputs from these components interact with the LLaMA-3-8B language model to produce the final generated responses.

Data collection

To provide the model with relevant domain knowledge, a specialized AM corpus is constructed from multiple sources. These include approximately 250+ peer-reviewed research articles, 30+ book chapters, 150+ AM process guidelines, 50+ international technical standards (e.g., ASTM, ISO) and standard operating procedures (SOPs) from industrial sources and 120+ material datasheets. These materials cover different AM techniques such as FDM, stereolithography (SLA), digital light processing (DLP), directed energy deposition (DED) etc., also subdomains such as material science, machine settings, pre-processing, post-processing, design rules, failure analysis, etc. These techniques and subdomains are identified based on initial literature review and keyword-based search which highlighted the most frequently studied and practically relevant areas in AM domain.

As illustrated in Figure 2, the data collection process begins by aggregating AM knowledge sources such as academic papers, books and technical manuals into a unified corpus. The data acquisition is carried out through a combination of systematic literature search and manual curation using academic databases such as Google Scholar, Scopus, and publisher repositories. Academic publications are collected using keyword-based searches (e.g., “polymer composite,” “DLP,” “direct ink writing (DIW),” “Ceramic AM” etc.) across digital libraries. Technical manuals, standards and industrial documents are obtained from publicly available repositories and manufacturer websites. The collection and curation process are performed by a team of four researchers, who have manually reviewed and filtered the documents to ensure relevance to AM processes, polymer composite materials and design practices. Duplicate and low-quality documents are removed during this stage. The selected documents are then processed through a text preprocessing pipeline that includes optical character recognition (OCR), LaTeX text extraction and text cleaning to obtain machine-readable text. OCR is performed using standard portable document format (PDF) to text extraction tools, while LaTeX-based documents are parsed using automated scripts. Text cleaning includes removal of formatting artifacts, symbols and irrelevant metadata.

The processed text is subsequently segmented into smaller passages using a chunking strategy with fixed token lengths in order to support efficient document retrieval. Each text chunk is converted into a dense vector representation using a sentence-embedding model based on the Sentence-Transformers framework, which has been widely used for semantic similarity and information retrieval tasks in NLP (Reimers and Gurevych, 2019).

The resulting embeddings are stored in a vector database constructed using Facebook AI Similarity Search (FAISS), a similarity search library designed for efficient nearest-neighbor retrieval in large-scale embedding spaces (Johnson et al., 2019). In parallel with corpus construction, a 200-question AM benchmark dataset is developed for model evaluation. The detailed benchmark structure and validation procedure are described in Section 2.5.

Model architecture and foundation

The base model used in this study is LLaMA-3-8B, a transformer-based large language model that provides strong natural language understanding and reasoning capabilities (Grattafiori et al., 2024). Instead of training a model from scratch, the pretrained model is adapted to the AM domain using parameter-efficient adaptation techniques. This model is selected as the base model because it provides a practical balance between model capability, computational accessibility and reproducibility. Using the same open weight base model for the baseline, RAG, and fine-tuned configurations allowed us to compare the effect of different adaptation strategies while keeping the underlying model architecture constant.

To maintain computational efficiency, the majority of the pretrained model parameters remain frozen while a small subset of trainable parameters is introduced through parameter-efficient adaptation layers. This strategy enables the integration of specialized manufacturing knowledge without requiring large-scale retraining of the entire network.

In addition to the base transformer architecture, the framework integrates domain-specific knowledge through structured AM documents and retrieval-based reasoning mechanisms. These mechanisms allow the model to better capture relationships between materials, manufacturing processes and numerical process parameters. As a result, the model can reason about engineering constraints such as valid parameter ranges, material compatibility and manufacturing process limitations.

Adaptation strategies

To analyze the effectiveness of different approaches for incorporating domain knowledge, three model configurations are implemented and compared. The experimental comparison setup is illustrated in Figure 3.

As illustrated in Figure 3, the AM benchmark dataset is evaluated across three different model configurations: baseline, fine-tuned and RAG models. The outputs generated by each model are then evaluated by domain experts according to predefined technical criteria.

Baseline (zero-shot inference)

The baseline configuration evaluates the pretrained LLaMA-3-8B model without domain-specific adaptation. The model answers AM-related questions using only the general knowledge learned during pretraining, providing a reference point for assessing the effects of fine-tuning and retrieval augmentation.

Fine-tuned model

In the fine-tuning configuration, the base model is adapted using the domain-specific dataset described in Section 2.1. The training process used causal language modeling on raw AM-domain text rather than supervised fine-tuning on question-answer pairs. In this autoregressive training objective, the model learns to predict the next token from the preceding context, allowing it to adapt to domain-specific AM terminology and technical language patterns. To reduce computational cost and memory requirements, QLoRA is used (4-bit quantization with LoRA adapters) for parameter-efficient fine-tuning (Alam et al., 2024). QLoRA introduces small trainable low-rank matrices into the transformer attention layers while keeping the majority of the original model parameters unchanged. This approach significantly reduces the number of trainable parameters while maintaining strong performance (Dettmers et al., 2023; Haghighi et al., 2025). For fine-tuning, the domain-specific corpus is divided into 90% training data and 10% validation data. The validation subset is used to monitor validation loss and guide early stopping during training. The benchmark questions are kept separate from the training and validation data and used only for final evaluation. The parameters included in Table 1 are used for fine tuning.

As shown in Table 1, the model is trained for several epochs using the AdamW optimizer and early stopping is applied based on validation loss to prevent overfitting due to the relatively small dataset size. This approach allows the model to embed AM knowledge directly into its internal representations, improving its ability to produce technically accurate responses.

Figure 4 shows the training and validation loss curves recorded during fine-tuning. Both losses decrease progressively across training steps, with the validation loss converging toward the training loss by the later stages. The relatively small and stable gap between the two curves indicates that the model does not exhibit severe overfitting under the applied QLoRA fine-tuning configuration. The residual oscillation in the training loss is consistent with the small batch size and the stochastic nature of token-level language modeling on a heterogeneous AM corpus.

RAG model

The third configuration uses a RAG framework to enhance the language model with external domain knowledge. RAG combines neural language models with document retrieval systems, allowing the model to access relevant documents during inference instead of relying solely on internal model parameters (Lewis et al., 2020; Karpukhin et al., 2020). The RAG workflow consists of two stages: offline document indexing and online inference, which are illustrated in Figure 5.

During the offline stage, domain documents are converted into machine-readable text through OCR or LaTeX extraction, as shown in Figure 5b. The cleaned text is then divided into overlapping segments using a chunking strategy that preserves contextual continuity. Each text segment is converted into a semantic embedding using a sentence embedding model and stored in a FAISS vector database to enable efficient similarity-based retrieval (Johnson et al., 2019).

Also, during online inference, each user query is embedded and matched against the FAISS index to retrieve the most relevant document chunks. These retrieved passages are appended to the query as contextual input for the LLaMA-based language model.

Training strategy and knowledge integration

The model training process follows a staged domain adaptation strategy. Initially, the model is exposed to general technical knowledge related to engineering and materials science to strengthen its understanding of technical vocabulary. Subsequently, training focuses on AM literature to capture relationships between process parameters, materials and performance outcomes.

Special attention is given to representing numerical parameters and process constraints, which are critical for manufacturing applications. The training procedure emphasizes learning valid parameter relationships such as temperature ranges, layer thickness values, curing conditions and process limits associated with different materials and AM processes. This improves the model’s ability to provide realistic and technically consistent parameter recommendations.

Evaluation protocol

The performance of the three model configurations is evaluated using a benchmark dataset consisting of 200 expert-designed AM questions. These questions are developed through a combination of literature review, research experience and practical teaching interactions. Specifically, the dataset is created by the formed team, who curated questions based on commonly discussed topics in AM literature, frequently encountered research problems and real-world queries observed during classroom teaching and online technical forums.

The questions are designed to cover a broad range of AM knowledge areas, including materials selection, process parameter optimization, defect diagnosis, design guidelines and process-specific understanding. To improve transparency and reproducibility, the 200 benchmark questions are categorized according to their primary AM topic, question type and difficulty level. The benchmark includes questions covering general AM principles, FDM, vat photopolymerization, DIW, polymer and composite AM, ceramic AM, two-photon polymerization (2PP), laminated object manufacturing (LOM), binder jetting and 4D printing. Difficulty levels are assigned based on the expected reasoning required to answer each question. Fundamental questions assess basic terminology and process concepts; intermediate questions require explanation of process, material-property relationships; and advanced questions require applied reasoning related to troubleshooting, parameter optimization, defect formation or process-specific limitations. Representative questions include fundamental concepts, such as “What is the typical layer height in FDM printing?” and more advanced process-level inquiries, such as “How does nozzle geometry influence extrusion behavior in DIW printing?”. The resulting benchmark distribution is summarized in Table 2.

Later the benchmark questions are further reviewed to ensure technical correctness, clarity, practical relevance and coverage across major AM subdomains. During this validation process, duplicate, ambiguous, overly broad, or overly narrow questions are revised or removed. After validation, the final set of 200 benchmark questions is used as the input query set for evaluating the three model configurations and is not included in the fine-tuning training or validation data.

Each generated response is assessed using following evaluation criteria:

Accuracy: It measures whether the response is technically correct according to established AM knowledge, consistent with evaluation practices in QA benchmarks (Rajpurkar et al., 2016; Wang et al., 2019).

Contextual correctness: This criterion evaluates whether the statements made in the response are appropriate and factually consistent with known AM and engineering principles for the specific context of the question, even beyond the direct answer to the question. A response may answer the main question correctly but still include unsupported, misleading, or incorrect additional claims. In that case, it may be considered accurate for the direct answer but lower in contextual correctness (Maynez et al., 2020; Lin et al., 2022).

Relevance: It assesses whether the response directly addresses the user’s query and provides meaningful information, as commonly used in information retrieval and QA systems (Chen et al., 2023).

Harmfulness: Whether the response contains potentially unsafe, misleading or harmful recommendations, especially for AM design, material selection, machine operation, or engineering decision-making.

Better answer: This criterion represents the overall expert preference between two model responses. It considers multiple aspects of response quality together, including technical correctness, relevance, completeness, clarity and practical usefulness.

Understandability: Whether the response is clear, logically organized and understandable to the intended audience.

Real-world applicability: Whether the response provides information that is practically useful and realistic for AM research, engineering practice or manufacturing decision-making.

The generated responses are evaluated by AM experts team using a structured qualitative assessment framework. The evaluation is conducted in a blinded manner, meaning that the experts did not know which model generated each response during assessment. Binary judgments, such as Yes/No decisions are used to reduce ambiguity in expert assessment and to obtain a clear question-level decision on whether a response is technically acceptable, contextually correct and relevant to the query. This approach is selected because many AM questions require determining whether a response is acceptable for engineering use rather than assigning a subjective numerical score.

To provide comparative context beyond individual binary judgments, experts also performed pairwise comparisons between responses from two model configurations for each benchmark question. In these comparisons, experts selected the response that better satisfied the specified evaluation criterion. Therefore, the resulting values are interpreted as pairwise expert preference rates rather than absolute standalone model accuracy scores. Because responses are evaluated by multiple experts, the reported percentages are calculated from aggregated expert preference decisions across the 200 benchmark questions rather than from a simple count of correct answers out of 200. For example, an accuracy-based preference value indicates the proportion of expert comparison decisions in which one model’s response is judged more technically correct than the competing model’s response.

This combined use of binary judgments and pairwise preference comparisons provides an intuitive and application-oriented assessment of model performance, reflecting how engineers evaluate the correctness and usefulness of technical information in real-world scenarios. Similar human judgment-based evaluation strategies have been widely adopted in recent LLM studies, particularly for domain-specific tasks where interpretability and practical reliability are critical (Maynez et al., 2020).

Experimental environment

All experiments are implemented using modern deep learning frameworks. The system uses the Hugging Face Transformers library for model implementation and parameter-efficient fine-tuning tools for QLoRA-based training. The retrieval system used in the RAG framework is implemented using FAISS for efficient vector similarity search. Training and inference experiments are conducted on graphics processing unit (GPU) accelerated computing hardware to ensure efficient model optimization and reproducibility of results.

For consistency across evaluation settings, benchmark questions are presented using standardized prompt formats. For the baseline and fine-tuned models, the input consists of the benchmark question alone, without any retrieved external context. For the RAG model, the benchmark question and retrieved document chunks are combined as input using the prompt template shown in Figure 6a. The template contains two placeholders: {question}, which represents the benchmark AM question and {retrieved context}, which represents the top-k document chunks retrieved from the FAISS vector index via semantic similarity search. The instruction line constrains the model to generate responses grounded solely in the retrieved content, reducing the risk of hallucination.

Figure 6b shows a representative example from the benchmark dataset illustrating how this template is populated at inference time. The question “How does nozzle geometry influence extrusion behavior in Direct Ink Writing (DIW)?” is drawn from the benchmark questions of intermediate-difficulty DIW category. The corresponding context passage covers the effect of nozzle geometry on ink flow, extrusion pressure and print fidelity, and the reference answer summarizes how tapered versus cylindrical tip geometries affect performance for different ink viscosities. As demonstrated in Table 2, each benchmark example is annotated with its category, difficulty level and question type to support reproducibility and dataset transparency.

This standardized prompting strategy ensures that all three configurations: baseline, fine-tuned and RAG are evaluated under consistent input conditions, while allowing the RAG model to leverage retrieved AM-domain knowledge during inference.

The entire evaluation focuses on comparative performance across accuracy, relevance and overall response preference.

Comparative performance: RAG vs. baseline

The comparative performance of the RAG model and the pretrained baseline model is presented in Figure 7. The evaluation is based on three criteria: overall answer preference (“Better Answer”), accuracy and relevance, as assessed by domain experts. The “Better Answer” criterion is used as a holistic preference measure in the primary RAG-versus-baseline comparison to evaluate the overall benefit of the proposed retrieval-based framework. This criterion considers correctness, relevance, completeness, clarity and practical usefulness together.

The RAG model consistently outperforms the baseline model across all evaluation metrics. In terms of overall preference, RAG responses are preferred in 85.2% of cases, whereas baseline responses are preferred in 14.8% of cases. This indicates a strong preference among evaluators for answers that are supported by retrieved domain-specific context.

A similar trend is observed in accuracy. In the pairwise accuracy-based comparison, RAG responses are judged more technically correct in 75.5% of cases, whereas baseline responses are preferred in 24.5% of cases. This improvement suggests that incorporating retrieved knowledge helps the model generate technically correct responses by grounding its outputs in relevant AM information.

The difference is even more pronounced in relevance. In the pairwise relevance-based comparison, RAG responses are preferred in 90.8% of cases, whereas baseline responses are preferred in 9.2% of cases. This highlights the effectiveness of retrieval in ensuring that responses directly address the user’s query and remain contextually aligned with the problem.

Performance of fine-tuned model compared to baseline

The performance of the fine-tuned model is compared with the pretrained baseline model in Figure 8, focusing on two key evaluation metrics: accuracy and relevance. This comparison aims to assess whether domain-specific fine-tuning improves the model’s ability to generate technically correct and contextually relevant responses.

As in Section 3.1, the percentages in Figure 8 represent pairwise expert preference rates rather than absolute model accuracy or relevance scores. In this comparison, the baseline model is evaluated against the fine-tuned model; therefore, the baseline percentages differ from those in Figure 7 because the competing model is different.

As shown in Figure 8, the fine-tuned model does not outperform the baseline model. In terms of accuracy, fine-tuned responses are judged more technically correct in only 5.6% of cases, whereas baseline responses are preferred in 94.4% of cases. This indicates that the fine-tuning process, when applied to raw and unstructured domain data, does not effectively enhance the model’s ability to generate correct technical responses.

A similar trend is observed for relevance. Fine-tuned responses are judged more relevant in 32.5% of cases, while baseline responses are preferred in 67.5% of cases. This suggests that the fine-tuned model struggles to consistently produce responses that directly address the user’s query, despite being exposed to domain-specific data during training. Overall, these pairwise preference results show that the fine-tuned model does not outperform the pretrained baseline, indicating that the applied fine-tuning strategy did not improve technical correctness or query alignment for the evaluated AM QA task. The “Better Answer” criterion is not used for the fine-tuned versus baseline comparison because this comparison is designed as a diagnostic evaluation to determine whether fine-tuning improved specific technical dimensions, namely, accuracy and relevance. Since the fine-tuned model showed substantially lower performance in both accuracy and relevance, an additional metric is not included in the original evaluation.

Information quality and usability analysis

The quality and usability of responses generated by the RAG and fine-tuned models are further analyzed in Figure 9, which presents a comparative evaluation across multiple qualitative metrics, including harmfulness, contextual correctness, understandability and real-world applicability. In Figure 9, the “correct context” metric is specific to the RAG model because it evaluates whether the retrieved document chunks provided to the model are relevant and appropriate for answering the given AM question. This metric is not applicable to the fine-tuned model because the fine-tuned model does not receive externally retrieved context during inference. Instead, it generates responses using only the input question and the knowledge encoded in its model parameters.

As illustrated in Figure 9, the RAG model demonstrates consistently strong performance across all evaluation dimensions. The model achieves 100% in producing non-harmful responses, indicating that all generated outputs are safe and aligned with acceptable technical standards. In terms of contextual correctness, 72.4% of the responses are evaluated as containing accurate and relevant information, while 27.6% exhibit some degree of inconsistency. The RAG model also performs well in understandability, with 91.3% of responses being clearly interpretable.

Additionally, 74.0% of the responses are considered applicable in real-world engineering contexts, suggesting that the model is capable generating practically useful guidance for domain specific tasks. In contrast, the fine-tuned model exhibits significantly lower performance across the evaluated metrics, as shown in the lower panel of Figure 9. Only 31.8% of its responses are judged to be understandable, while the majority (68.2%) fail to clearly convey meaningful information. Although the model maintains a relatively high level of safety, with 88.2% of responses classified as non-harmful, its practical utility remains limited as only 8.5% of the responses are considered suitable for real-world use, indicating that most outputs lack actionable or technically reliable content. Overall, these results highlight a clear distinction between retrieval-based and fine-tuning based adaptation strategies. The RAG model consistently achieves higher performance across safety, contextual correctness, understandability and real-world applicability, whereas the fine-tuned model shows comparatively lower response quality and limited practical usability.

Under the experimental conditions evaluated in this study, the RAG model produced more accurate, relevant, and usable responses than both the pretrained baseline and the tested fine-tuned configuration. This result suggests that retrieval-based grounding is beneficial for the AM QA benchmark used in this work. The improved performance of the RAG model may be attributed to its ability to access domain-specific knowledge dynamically during inference. Unlike fine-tuning on raw text, which attempts to encode domain knowledge into model parameters, RAG grounds responses in retrieved source documents. This distinction is particularly relevant for AM, where knowledge is distributed across research articles, standards, process guidelines and material documentation.

These findings are consistent with prior work in retrieval-augmented generation, where integrating external knowledge sources has been shown to improve factual accuracy and reduce hallucinations in knowledge-intensive tasks (Borgeaud et al., 2022). In particular, the strong improvement in relevance observed in this study aligns with the core principle of RAG systems, which retrieve contextually similar documents to guide response generation.

In contrast, the observed performance degradation in the fine-tuned model highlights a known limitation of parameter-based adaptation when applied to unstructured data. Similar observations have been reported in prior studies, where fine-tuning on raw domain text without task-specific supervision can lead to loss of general reasoning capabilities (Chung et al., 2024; Zhang et al., 2020). As shown in Figure 4, the training and validation loss curves converge without divergence, suggesting that the observed performance degradation is not a consequence of overfitting, but rather of the unsupervised causal language modeling objective, which does not directly optimize QA performance. This suggests that simply exposing a model to domain knowledge is insufficient for improving downstream task performance, especially in QA settings. The training objective of predicting the next token from raw text does not teach the model how to answer structured technical queries accurately or concisely. This distinction between domain exposure and task-specific learning is a key factor underlying the performance gap observed between the fine-tuned and RAG configurations.

Another key advantage of the RAG approach is its flexibility and scalability. Unlike fine-tuned models, which require retraining to incorporate new knowledge, RAG systems can update their knowledge base dynamically by adding new documents to the retrieval index. This property is particularly important in engineering domains such as additive manufacturing, where knowledge evolves rapidly and includes both theoretical and procedural components. Furthermore, the strong performance of RAG in real-world applicability and contextual correctness suggests that retrieval-based grounding plays a critical role in bridging the gap between general-purpose language models and domain-specific expertise. This aligns with recent domain-specific LLM studies in manufacturing and biomedical fields, where retrieval mechanisms have been shown to significantly improve reliability and trustworthiness of generated outputs. Overall, the findings of this study reinforce the growing consensus that hybrid approaches combining language models with external knowledge retrieval provide a more effective pathway for deploying LLMs in specialized technical domains.

Despite the promising results, several limitations should be acknowledged. First, the fine-tuning process is conducted using raw textual data without incorporating structured question-answer pairs. This limits the model’s ability to learn how to generate high-quality responses in a QA setting. Second, the evaluation relies on expert judgment, which may introduce subjectivity. Although multiple experts are involved, variations in interpretation and experience could influence evaluation outcomes. Third, the size of the benchmark dataset is relatively limited (200 questions), which may not fully capture the diversity of real-world AM queries. Fourth, the RAG system depends heavily on the quality and coverage of the underlying corpus. Missing or incomplete information in the knowledge base may lead to incomplete or suboptimal responses. Fifth, the study focuses primarily on textual knowledge representation. Numerical reasoning and simulation-based validation, which are important in engineering applications, are not explicitly addressed in this work. Finally, because this study evaluates only LLaMA-3-8B, the relative performance of RAG and fine-tuning may differ for other model families or larger model sizes. Therefore, the results should be interpreted as a controlled comparison of the tested configurations rather than a universal conclusion about all LLM architectures.

Taken together, these results indicate that, for the AM QA benchmark and model configurations evaluated in this study, the RAG-based approach provides more reliable responses than the pretrained baseline and the tested fine-tuned model. It improves response accuracy by grounding outputs in curated source documents, eliminates the need for costly retraining when domain knowledge evolves and consistently outperforms direct fine-tuning on unstructured technical text.

For future work, several directions can be explored. First, developing a larger and more structured domain specific QA dataset could improve fine-tuning effectiveness. In particular, supervised fine-tuning using high quality AM question answer pairs may better teach the model how to generate accurate and concise responses to AM-specific queries. Second, hybrid approaches that combine retrieval and supervised fine-tuning may further enhance performance by preserving the benefits of external knowledge grounding while improving task-specific response generation. Third, future studies should compare open weight models such as LLaMA-3-8B with larger proprietary state-of-the-art models, including ChatGPT and Claude to examine whether smaller domain-grounded RAG systems can achieve competitive performance in AM question answering. Beyond model comparison, future work should also integrate numerical reasoning capabilities and simulation-based validation to improve the applicability of LLMs in engineering decision making. Another important direction is the integration of AM process failure knowledge and real time process monitoring data, such as optical, thermal, acoustic, or layer wise inspection signals, to detect process failures and support real time decisions on parameter adjustment or process interruption. Overall, this work demonstrates that retrieval-based methods offer a practical pathway for building reliable AI assistants in AM and similar technical domains.