Fine-Tuning vs. RAG: A Position Paper from the Perspective of LLM-Based Cybersecurity Modeling

In recent years, the rapid advancement of LLMs has substantially broadened the applications of Artificial Intelligence (AI), particularly in domains such as cybersecurity, where they automate and expedite tasks that previously required considerable human effort. Modern LLMs, with notable examples including Google’s Gemini and OpenAI’s Generative Pre-trained Transformer (GPT) series, have transformed natural language processing (NLP) by achieving advanced capabilities in contextual reasoning, text generation, and comprehension [1]. During pre-training, LLMs are exposed to diverse datasets, enabling broad generalization and domain expertise. Their ability to understand, generate, and reason over complex information makes them valuable tools in cybersecurity, analyzing logs, emails, and network traffic to detect phishing attacks, malware, spam, or insider threats, often faster while reducing manual workload, thereby saving time and costs.

Language models acquire knowledge from diverse datasets [2], but struggle with uncommon, domain-specific, or inaccurate data [3], and cannot incorporate new information or changing facts after training, as their knowledge is static. This can result in temporal degradation, outdated or inaccurate responses, and hallucinations. These issues are concerning in high-stakes fields like cybersecurity, where accuracy, adaptability, and up-to-date threat intelligence are crucial for tasks such as threat detection, anomaly identification, and malware analysis. Researchers have focused on customizing LLMs for specific industries, such as cybersecurity, healthcare, finance, and others, while enhancing their access to current data to address these challenges. Two prominent methods have emerged: fine-tuning and RAG. RAG keeps the model up-to-date without altering its internal parameters by retrieving relevant external knowledge in real time, similar to In-Context Learning (ICL) [4], which boosts responses by adding information to the input rather than adjusting the model’s weights. In contrast, fine-tuning involves retraining the model on specialized datasets to enable it to learn domain-specific knowledge.

RAG is more flexible and scalable, but fine-tuning improves accuracy within a specific range. Both approaches offer distinct advantages and trade-offs in terms of computational cost, data requirements, update frequency, and deployment complexity. In this position paper, the aim is to explore which methodologies are most suitable to enhance LLMs in low-resource, high-stakes domains, such as cybersecurity. Specifically, we focus on:

• Detailed analysis of effective fine-tuning techniques for cybersecurity modeling.

• Highlighting effective RAG strategies for cybersecurity modeling.

• Describe the key challenges and risks within the study of fine-tuning and RAG.

• Analyzing the comparative impact of retrieval-based vs. fine-tuning strategies and the hybrid approach.

Fine-tuning, RAG, and their evolving hybrid approach are key paradigms in the development and use of LLMs. This section examines the background of fine-tuning, RAG, and their hybrid approaches, including the underlying techniques and procedures.

2.1. Fine-Tuning

Fine-tuning is the process of continued training on smaller, more specialized datasets and is used to specialize a pre-trained LLM for domain-specific tasks [5]. It has multiple methodologies, techniques, and applications. This paper discusses the methodologies, techniques, and applications of fine-tuning in cybersecurity.

2.1.1. Learning Paradigms in Fine-Tuning

Supervised Fine-Tuning (SFT): SFT [6] adapts large language models to cybersecurity tasks by training them on labeled datasets that capture real security threats, such as malware samples [7], spam messages [8], network intrusion annotations [9], and others. SFT improves the model’s ability to map inputs to correct outputs for tasks such as threat classification and anomaly detection. Figure 1 illustrates this SFT process.

Unsupervised Fine-Tuning (UFT): UFT [10] adapts pre-trained LLMs to the cybersecurity domain without labeled data by continuously training on large-scale unlabeled sources. This method leverages self-supervised learning and the underlying structure of the data to enable the model to learn cybersecurity terminology, patterns, and context.

Reinforcement Learning (RL): RL, particularly Reinforcement Learning from Human Feedback (RLHF) [11], offers a new frontier for ongoing improvements to LLMs in cybersecurity through iterative feedback aligned with preferences and operational goals. Different RL algorithms, such as reward modeling, Proximal Policy Optimization (PPO), and Direct Preference Optimization (DPO), enable adaptive learning in dynamic threat environments, supporting more effective security decision making.

2.1.2. Fine-Tuning Techniques for Cybersecurity Tasks

Hyperparameter Tuning: Hyperparameter tuning is an effective technique for fine-tuning LLMs to improve model performance, generalization, and training stability. Selecting optimal hyperparameters, such as batch size, epochs, learning rate, and regularization, can significantly improve the model and prevent overfitting, especially in cybersecurity tasks where data are often noisy or imbalanced.

Parameter-Efficient Fine-Tuning (PEFT): PEFT [12] adapts LLMs to various cybersecurity tasks by updating a small number of model parameters, significantly reducing computation and memory resources while preserving pre-trained knowledge. Techniques such as Low-Rank Adaptation (LoRA), adapters, and prompt tuning enable rapid model adaptation, which is useful for keeping pace with continuously evolving cyber threats.

Task-Specific Fine-Tuning: Task-specific fine-tuning is the process of adapting a pre-trained language model to a specific cybersecurity task using domain-specific labeled data to enable the model to identify patterns relevant to the task and improve its performance. Although task-specific fine-tuning can result in catastrophic forgetting due to parameter updates, it enables the model to achieve high accuracy in targeted cybersecurity applications.

Transfer Learning: Transfer learning adapts pre-trained LLMs to cybersecurity tasks using limited data, leveraging domain-specific models, such as SecureBERT [13], achieving superior performance with fewer computational resources than out-of-the-box LLMs for malware, phishing, and vulnerability detection.

Few-Shot and Zero-Shot Fine-Tuning: Few-shot and zero-shot fine-tuning approaches leverage the generalization capabilities of large pre-trained models to perform tasks with few or no labeled examples [14]. Few-shot learning enables the model to infer patterns in new data for detection using a small number of labeled examples. In contrast, zero-shot learning relies solely on prompts and prior knowledge, without using task-specific examples.

2.2. Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) [15] is an emerging paradigm that augments LLMs with knowledge retrieval and generation capabilities. Although LLMs demonstrate remarkable performance, they face issues such as hallucinations and outdated information, especially with Out-of-Distribution (OoD) questions. The hallucination issue arises when a model produces information that appears to be correct and real but is factually incorrect or fabricated. RAG overcomes these challenges by dynamically retrieving up-to-date, domain-specific information from structured and unstructured sources at inference time [16]. Unlike traditional fine-tuning, which encodes task-specific knowledge into model parameters, RAG—particularly in the cybersecurity domain—retrieves information from external sources such as threat databases, security logs, and technical documentation. This approach ensures adaptability and interpretability while enabling the acquisition of new information without modifying or retaining it within the model parameters.

2.2.1. RAG Core Components & Workflow

Core Components: RAG is made up of multiple components that work together to generate context-aware responses. The first component is the retriever, which identifies and collects relevant documents and knowledge snippets from external sources such as databases, search indexes, or structured knowledge bases, with key resources in cybersecurity, including MITRE ATT&CK [17], Common Vulnerabilities and Exposures (CVE) [18], and National Vulnerability Database (NVD) [19]. To support semantic retrieval, embedding models convert both queries and documents into vector representations, enabling dense retrieval of the most contextually relevant information from a knowledge base. The knowledge base is another component that serves as an external repository of domain-specific information, including texts, databases, APIs, and real-time feeds stored in vector databases. Finally, the generator, typically a language model, produces the final response by combining the original query with the retrieved context to generate accurate answers.

RAG Workflow: The RAG workflow is based on the input query, retrieval, and generation process. The workflow begins with a user query, which is converted into a vector that represents its semantic meaning. This vector is used to retrieve the most relevant document fragments from a vector database for the submitted query. After that, the retrieved document chunks are filtered and ranked to select the best matches to the input query. Finally, a language model processes a prompt formed by combining the retrieved context with the original query, generating a coherent and contextually appropriate response. The system then returns a formatted response to the user.

The RAG workflow can be understood through an example of misleading feedback or fake review detection in Small and Medium Enterprises (SMEs) [20]. These false or misleading reviews are a growing concern in the SME industry [21], and the deployment of RAG can play a significant role in this industry. Using the RAG methodology, SME analysts can identify and detect fake reviews with more accuracy. First, the analyst submits a query, such as a product review with the question, “Is this review fake?”, and the system converts the review text into a semantic vector that captures its meaning. The semantic vector is used to retrieve labeled fake reviews, expert analyses, and the most relevant flagged document chunks from the database. Relevant document chunks, such as reviews that show similar patterns, are retrieved. The retrieved context is then combined with the original query and passed to a generation model, which generates an appropriate response. An overview of the RAG workflow is shown in Figure 2 for fake review detection.

2.2.2. Graph Retrieval-Augmented Generation (GraphRAG)

GraphRAG is a graph-based extension of RAG that integrates Knowledge Graphs (KGs) into the RAG pipeline to improve reasoning, contextual understanding, and explainability in LLMs [22]. The GraphRAG framework is designed to overcome the drawbacks of the traditional RAG system, including a lack of understanding of relationships, failure in multi-hop reasoning, and a tendency to hallucinate. The GraphRAG workflow consists of several key steps. For example, in fake review detection for SMEs, when an analyst submits a query—such as a product review accompanied by the question, “Is this product review fake?”—the system first converts the review text into structured entities (e.g., user, product, review, sentiment) and their relationships. These entities and relationships are stored in a Knowledge Graph that can be constructed from both local data sources, such as private reviews, customer histories, and external data sources (public review repositories and other external sources). When the query is processed, the system performs a graph-based retrieval, rather than relying solely on text similarity. It traverses the knowledge graph to identify relational patterns, such as users posting multiple identical reviews, repeated sentiment expressions across unrelated products, or unnatural temporal correlations. The retrieved subgraph representing the most relevant entities and their relationships is aggregated with the original query and submitted to the language model. Finally, the model generates a response by reasoning over the retrieved graph context, enabling context-aware and explainable detection of fake reviews. The example workflow of GraphRAG is shown in Figure 3.

Compared to other domains, this framework is not widely explored in cybersecurity, but it is becoming increasingly popular. For example, in [23], the authors proposed the CyKG-RAG framework, which integrates KGs with the RAG approach to detect and analyze cybersecurity threats. Similarly, in other research areas, such as network security monitoring and analysis, GraphRAG drives demand for context-aware, adaptive solutions.

2.3. Hybrid Approach

The hybrid approach, which combines fine-tuning and RAG, has gained popularity because it addresses the drawbacks of each technique when used independently. Figure 4 provides a conceptual overview of the hybrid approach combining fine-tuning and RAG.

This approach is beneficial when domain-specific knowledge is required alongside a real-time update process. For instance, in cybersecurity modeling, both deep domain adaptation and real-time responsiveness may be needed, and this hybrid methodology can enable a more robust and adaptive approach to cybersecurity modeling. The hybrid approach has also been successfully applied in various other fields, including healthcare [24], multilingual question answering [25], etc.

The hybrid approach provides several key advantages, including increased robustness, lower retraining costs, enhanced explainability, and greater task flexibility. For instance, spam filtering in email, messaging, and other communication channels requires knowledge of both linguistic patterns and dynamic content structures. The fine-tuned model is optimized to detect linguistic and structural characteristics of spam, including anomalous sender behaviour and the excessive use of specific phrases. By acquiring the latest known spam samples, blacklisted IPs (Internet Protocol addresses) and domains, or publicly available spam databases, the RAG module enhances detection and enables real-time, explainable judgments based on both live data and learned patterns. This hybrid approach increases detection accuracy and facilitates better generalization to multilingual or obfuscated spam tactics and zero-day spam patterns.

This section presents several real-world applications within the cybersecurity domain, illustrating the potential of fine-tuning and RAG methods for LLMs to deliver substantial contributions.

Phishing Detection and Analysis: Fine-tuned LLMs effectively detect phishing content by adapting to new phishing patterns via parameter-efficient, few-shot fine-tuning, while hyperparameter optimization further enhances detection accuracy. Consequently, RAG works in scenarios where models need to consider the most recent phishing samples or attack indicators from external sources, such as threat feeds or phishing databases, to adapt to emerging threats.

Intrusion Detection and Log Analysis: Supervised fine-tuning on datasets such as UNSW-NB15, NSL-KDD, and CIC-IDS enables LLMs to detect intrusion activities in system logs or network traffic. Their contextual understanding aids in analyzing data, detecting rare attack vectors, and linking multi-step intrusions. RAG methods assess recent threats by retrieving up-to-date intelligence to enhance detection speed, accuracy, and organizational insights.

Vulnerability Detection and Analysis: Fine-tuned LLMs improve vulnerability management by summarizing complex CVE data and identifying key threats such as buffer overflows and Remote Code Execution (RCE), thereby aiding prioritization. RAG techniques, such as the knowledge-level framework, retrieve relevant information from a vulnerability knowledge base to detect issues and improve accuracy.

Fraud Detection: Fine-tuned LLMs are ideal for identifying known, stable fraud patterns by incorporating temporal and behavioural features from large volumes of transactional data. RAG-based frameworks are useful for detecting fraud or deception across transactions because they can analyze transactions in real time and retrieve relevant trends, rules, and cases from associated fraud history. This approach performs exceptionally well in few-shot, domain-agnostic settings, enabling effective detection of emerging fraud schemes.

SME Cybersecurity Advisor: SMEs are particularly vulnerable to cyber threats due to limited resources, outdated systems, and a shortage of security experts, leading to financial losses, diminished customer confidence, and operational disruptions. Fine-tuned LLMs can act as cybersecurity advisers by learning from domain policies, processes, and event data, while RAG enhances this capability by retrieving real-time threat intelligence, compliance requirement updates, and best-practice mitigations, enabling cost-effective cybersecurity support without retraining.

Even though both fine-tuning and RAG are effective approaches for adapting LLMs to cybersecurity tasks, they present distinct challenges and operational risks. Understanding the limitations of each method is essential for selecting or designing effective LLM-based cybersecurity solutions. The key challenges and risks are summarized below:

Data Dependency: Fine-tuning requires a substantial amount of high-quality labeled data, which is often difficult to obtain in specialized domains such as cybersecurity due to security restrictions or other constraints. RAG depends heavily on the quality and reliability of external retrieval data, raising the possibility of including poisoned, out-of-date, or biased data in its responses.

Security and Adversarial Risks: Fine-tuning faces security risks, such as data poisoning and hidden triggers. Attackers can exploit vulnerabilities in the model’s decision process. RAG is at risk of attacks on its retrieval system, such as the injection of malicious data or tampering with knowledge bases, which can lead to harmful outputs.

Performance and Forgetting: Fine-tuning offers rapid adaptation but may lead to overfitting or catastrophic forgetting of previously learned tasks when data is limited. RAG is slower due to external database retrieval and relies on database quality, risking performance drops if databases are unavailable or slow.

Maintenance and Resource Intensity: Fine-tuning requires periodic retraining, which is resource-intensive, especially for large models. Organizations must weigh the costs of computing and data annotation against the benefits of improved accuracy. RAG systems require continuous maintenance of retrieval databases and robust access controls to prevent data corruption and ensure data integrity. Scaling the retrieval infrastructure for high query volumes can also be challenging.

Deployment Challenges: Fine-tuning large models for commercial applications necessitates substantial computational resources and stringent data governance. Key challenges include model drift, repeatability, privacy compliance, and deployment complexities such as explainability, version control, and domain bias. The commercial deployment of RAG presents data security and governance risks, particularly when internal data sources are employed. Low latency, scalability, and retrieval accuracy are difficult with large, changing datasets. The hybrid workflow also poses challenges related to data freshness, compliance, and explainability.

Based on the comparative analysis of fine-tuning and RAG, in the context of cybersecurity modeling, adapting LLMs requires working with both approaches. Although each method presents distinct advantages, the choice between fine-tuning and RAG primarily depends on the specific use case, available resources, and operational requirements. Moreover, RAG is the preferred option for most enterprise use cases, while fine-tuning serves as the most practical solution for some scenarios. However, this does not mean that RAG is always the right choice over fine-tuning. To determine which method is best for LLMs in cybersecurity modeling, it is essential to understand the different aspects and features.

Knowledge Integration: Fine-tuning absorbs and assimilates domain knowledge during training, resulting in high performance in this domain. However, it is static by nature and quickly becomes outdated without retraining, making it unsuitable for rapidly evolving fields such as cybersecurity. In contrast, RAG maintains up-to-date responses by dynamically accessing external knowledge; however, its reliance on retrieval mechanisms can introduce discrepancies and errors in the outputs.

Adaptability and Flexibility: Fine-tuning adjusts the model’s parameters by retraining on domain-specific data, which incurs an additional cost for organizations. RAG offers superior flexibility by retrieving relevant data from an external knowledge base without changing the model’s internal parameters, though this increases complexity because it relies on retrieval quality.

Latency: RAG is slightly slower due to the heavy data retrieval and generation phases required to ensure up-to-date data, which takes additional time. Fine-tuning is typically faster than RAG because it can generate answers almost instantly without accessing external data sources.

Scalability: RAG is highly scalable, seamlessly handling large volumes of data by updating the external knowledge base without modifying the model itself. Fine-tuning is less flexible and faces challenges when scaling to evolving domains, as it requires periodic retraining. Adapting to new information requires retraining or additional fine-tuning, thereby limiting responsiveness in fast-evolving domains.

Computational Resources: RAG has lower training costs because it leverages the base or pre-trained language model and does not require any modifications during the training phase. However, computational resources are needed for retrieval during inference. Consequently, due to its resource demands and the need for extensive dataset preparation, fine-tuning during the training phase requires substantial memory, significant time, and multiple high-performance GPUs.

Additionally, the hybrid option can be a suitable solution on some occasions. Hybrid methods leverage the fixed-domain knowledge acquired through fine-tuning while also leveraging the dynamic adaptation offered by RAG, resulting in systems that are both knowledgeable and context-aware. However, without careful planning, the hybrid system may potentially inherit the drawbacks of both strategies, including potential latency bottlenecks, difficult implementation, and expensive maintenance. A comparison of fine-tuning, RAG, and the Hybrid Approach is presented in Table 1, focusing on various aspects to choose the most suitable one for a particular use case.

Additionally, another technique named prompt engineering is also used to design and optimize prompts to effectively guide LLMs, particularly in NLP tasks, to generate desired outcomes or responses [26]. It can be an alternative to fine-tuning or RAG in cybersecurity-based experiments. It is a faster, cost-effective, and lightweight approach compared to fine-tuning and RAG, and it works without retaining the model with minimal computational resources. In the cybersecurity domain, prompt engineering can analyze malicious logs, detect phishing attacks, assess disinformation, and perform vulnerability assessments, among other tasks. Despite its numerous advantages, prompt engineering has certain limitations that must be considered. These include the lack of real-time detection and reliance on pre-trained knowledge, meaning the model can only respond based on previously acquired information.

This position paper highlights the role of LLMs in cybersecurity modeling by exploring two prominent approaches: fine-tuning and RAG. This study began by detailing various fine-tuning paradigms and techniques for cybersecurity modeling, highlighting their effectiveness and strengths in adapting LLMs for domain-specific tasks such as detection, classification, and analysis. The study also explored RAG with its core components, working procedure, and a variant. Along with these approaches, a hybrid approach that integrates fine-tuned LLMs with RAG mechanisms was also discussed. The hybrid integration of both paradigms introduces a promising method that combines dynamic reasoning over recently retrieved information with stable core knowledge. Moreover, this study examined the applications, challenges, and risk factors associated with the risk factors of fine-tuning and RAG. Ultimately, the choice between fine-tuning, RAG, or a hybrid approach should be guided by specific cybersecurity use cases, available data, infrastructure constraints, and desired levels of interpretability. We conclude that future cybersecurity systems will benefit most from models that combine the precision of fine-tuning with the adaptability of RAG, providing both strong baseline intelligence and real-time responsiveness to emerging threats.